Real Data Downloads

SimTrade can replay authentic CME micro contract data in addition to the synthetic GBM paths. Historical candles are downloaded and prepared through download_data.py, then randomly sampled at runtime by engine/real_data_engine.py.

Supported Contracts

Symbol Description Saved As
MES=F Micro E-mini S&P 500 assets/MES_data.csv
MNQ=F Micro E-mini Nasdaq 100 assets/MNQ_data.csv
MGC=F Micro Gold assets/MGC_data.csv
SIL=F Micro Silver assets/SIL_data.csv
MBT=F Micro Bitcoin assets/MBT_data.csv

Each CSV contains OHLCV columns plus all indicators generated by pandas-ta.

Download Workflow

  1. Fetch from Yahoo Finance
  2. download_one() calls yf.download() for a single ticker (default period='59d', interval='5m').

  3. Multi-index columns are flattened and any timezone-aware index is normalized to naive datetimes.

  4. Indicator Pass

  5. The dataframe flows through add_technical_indicators() so synthetic and real datasets share identical indicator names.

  6. A 200-row warm-up slice is dropped to remove initialization bias.

  7. Persist to Assets

  8. Files are written under assets/ using the sanitized ticker name (e.g., MES_data.csv).
  9. Existing files are overwritten so you always have the freshest data.

Run the full batch with:

python download_data.py

You can customize the lookback via flags by editing download_data.py or invoking download_one() from an interactive session.

Sampling Real Scenarios

At runtime the UI can switch to "Real" mode. The flow inside engine/real_data_engine.get_random_scenario() is:

  1. Load the CSV once and cache it in memory by ticker.
  2. If at least periods rows exist, randomly pick a contiguous slice of the requested length (default 400 bars).
  3. If the data is missing or corrupt, gracefully fall back to generate_scenario_data() so the session never stalls.

Use the left panel in the Dash app to select Data Source → Real and pick a symbol. Every skip click will now pull a new real-market snippet.