Trading Strategy Data Manager

The TradingStrategyDataManager (TSDM) is the core component enabling us to progress from slow, hours-long data handling to the V4 vision of automated parameter exploration and scenario-based testing. By leveraging local, columnar data storage, TSDM cuts multi-hour data loads down to seconds, making advanced analyses and seamless experimentation standard practice. For more on the efficiency and capabilities TSDM unlocks, see our Data Performance Documentation.

Equally important, the TSDM separates data retrieval from strategy logic. Instead of getting bogged down in raw file parsing or custom filtering, we can focus purely on trading ideas and their validation. This clear division of concerns reduces cognitive load, enabling rapid iteration, deeper insights, and more confident decision-making.

Key Responsibilities

1. Historical Data Retrieval: Load large historical datasets (OHLCV candles, liquidity snapshots, exchange metadata) from Parquet files, enabling rapid iterative backtesting without reliance on external data sources.

2. Exchange & Pair Identification: Translate parameters (e.g., chain slugs, exchange slugs, token symbols) into internal IDs and addresses, providing a consistent interface to diverse DEX environments.

3. Date Range & Filtering: Allow precise control over time windows and intervals (e.g., daily or hourly data), reducing unnecessary data loading and focusing on the exact historical periods needed.

4. Caching & Performance Optimization: Use caching, chunked reads, and columnar data access to minimize redundant operations and improve execution speed, enabling strategies to test multiple scenarios efficiently.

Data Processing Flow

The TSDM acts as a translator, bridging the gap between our strategy's requests and the complexity of DEX data. When our strategy says, 'I need ETH-USDC price data from last month,' the TSDM figures out how that corresponds to a specific dataset on the hard drive, retrieves the needed information, formats it properly, and then provides that data ready for simulations.

Candle & Liquidity Schemas

• Candle Data is loaded and returned in the structure defined in Candle.md. That doc details the OHLCV fields (open, high, low, close, volume) plus extra metadata (e.g., timestamp, exchange_rate) the strategy might need.

• Liquidity Data is loaded and processed using the schema in XYLiquidity.md. The TSDM fetches daily (or hourly) TVL/reserve data, merges it to candle data if requested, and returns a combined DataFrame with time-based alignment.