- Add spot_open field to BacktestDailyPoint for complete OHLC data
- Replace line chart with candlestick chart showing price OHLC
- Add portfolio value line on secondary Y-axis
- Add _chart_options_from_dict for rendering job results
- Update both render_result and render_job_result to use new chart
- Remove async refresh_workspace_seeded_units from date change handlers
- Date changes now only call on_form_change() (updates cost estimates, marks results stale)
- Entry spot is derived only when user clicks Run button
- Form remains responsive during configuration
- No more API errors when changing dates while configuring other fields
- Move option contracts from daily results table to overview cards (constant throughout backtest)
- Fix default dates to March 2026 (2026-03-02 to 2026-03-25)
- Add BT-004 backlog item: candlestick chart with portfolio value line on secondary axis
- Add BT-005 backlog item: defer entry spot derivation to backtest run (not on every date change)
- Add option_contracts field to BacktestDailyPoint (number of contracts held)
- Update engine to calculate total option contracts from positions
- Update job serialization to include underlying_value, option_market_value, net_portfolio_value, option_contracts
- Update both render_result and render_job_result tables to show:
- Low, High, Close (from previous commit)
- Portfolio value (net_portfolio_value)
- Option value (option_market_value)
- Contracts (option_contracts)
- LTV hedged
- Margin call status
- Fix return type of _load_from_cache and _df_to_daily_points to return list[DailyClosePoint]
- Import DailyClosePoint from historical_provider
- Use TYPE_CHECKING pattern for optional databento/pandas imports
- Add back clear_cache method that was accidentally removed
- Add file_count and total_size_bytes to get_cache_stats return value
- Update tests for fixed March 2026 default dates
- Change default backtest date range to 2026-03-02 through 2026-03-25
- Add spot_low and spot_high to BacktestDailyPoint for intraday range
- Update engine to populate low/high from DailyClosePoint
- Update daily results table to show Low, High, Close columns instead of just Spot
- Update job serialization to include spot_low and spot_high
- Extend DailyClosePoint to include low, high, open (optional)
- Update Databento source to extract OHLC data from ohlcv-1d schema
- Update YFinance source to extract Low, High, Open from history
- Modify backtest engine to use worst-case (low) price for margin call detection
This ensures margin calls are evaluated at the day's worst price,
not just the closing price, providing more realistic risk assessment.
Databento can raise RuntimeError for API key issues, but derive_entry_spot
only caught ValueError and KeyError. This ensures Databento errors are
properly caught and displayed to the user.
The job serialization was fixed to use new field names, but the UI render
function was still using old field names (total_pnl, hedging_cost, etc.)
which don't exist anymore. Now uses:
- start_value, end_value_hedged_net, total_hedge_cost from summary_metrics
- template_results[0].daily_path for daily results table
- Added margin call metrics display
- Fix BacktestPageRunResult serialization in jobs.py to correctly access
nested fields from scenario and run_result objects
- Add test_backtest_job.py with comprehensive tests for job execution
- Add conftest_playwright.py with ServerManager that starts FastAPI server
for Playwright tests using uvicorn
- Add test_playwright_server.py with E2E tests using the server fixture
The job serialization bug was causing backtest results to fail silently
because it was trying to access non-existent fields on BacktestPageRunResult.
The backtest engine was always using fixture data (limited to 2024-01-02
through 2024-01-08) regardless of the data_source selection. The fix
fetches historical prices using the specified data source (Databento,
Yahoo Finance, or synthetic) and passes them directly to the engine.
- Pin black version in requirements-dev.txt (was >=24.0.0)
- Update pre-commit to use black 26.3.1 with Python 3.12
- Add language_version: python3.12 to pre-commit black hook
- Reformat files with new black version for consistency
- Remove app/components/ and app/pages/ from ruff/black excludes
- Pre-commit reformatted multi-line strings for consistency
- All files now follow the same code style
The backtest engine uses a fixture provider (synthetic_v1) regardless of
the data_source used for price fetching. We must use the fixture provider's
ID for the scenario, not the data source's ID.
This fixes 'Unsupported provider/pricing combination' error when running
backtests with data_source='databento'.
BREAKING CHANGE: Complete redesign of backtest execution
- Add BacktestJob system with progress stages (validating, fetching_prices, calculating)
- Run backtests in background threads, UI polls for status
- Show progress label with current stage during execution
- Remove synchronous Databento API calls from page load
- Use static default entry spot for initial render (defers API call)
- Make refresh_workspace_seeded_units async with run.io_bound
This fixes:
- 'Connection lost' WebSocket timeout errors
- Slow page load (30s initial load)
- Backtest never completing
The job system provides:
- Non-blocking execution
- Progress tracking with stages
- Error handling with user-friendly messages
- Result caching for retrieval after completion
- Use run.io_bound() from NiceGUI to run Databento API calls in background thread
- Add loading state to Run Backtest button
- Show notification when backtest starts and completes
- Remove loading state on completion/error
This prevents 'Connection lost' errors when the backtest takes longer than the WebSocket timeout.
- on_form_change: Only update cost estimate, skip expensive derive_entry_spot
- Only call derive_entry_spot on date changes (start/end inputs)
- Other inputs (template, units, loan, LTV) just mark results stale
- This reduces lag from constant API polling
- Set default dates to 2024-07-01 to 2024-12-31 (valid for XNAS.BASIC)
- Catch all exceptions during entry spot derivation, not just ValueError
- Don't auto-run backtest on page load - let user configure first
- Use recent GLD price (~30) as fallback
- Add DATABENTO_DATASET_MIN_DATES for XNAS.BASIC (2024-07-01) and GLBX.MDP3 (2010-01-01)
- Validate start date against dataset minimum before running backtest
- Parse Databento API errors and show user-friendly messages
- Update date range hint to show dataset-specific availability
- Catch BentoClientError and show appropriate warning tone
- Pass data_source to derive_entry_spot in backtests.py
- Remove default 'synthetic' value for data_source in derive_entry_spot and validate_preview_inputs
- Update all tests to explicitly pass data_source parameter
- Improve error message with helpful suggestion for Databento/Yahoo Finance
- Change WindowPolicy from EXACT to BOUNDED for backtest fixture
- Pass data_source to run_read_only_scenario so real data can be used
- Fix injected provider identity preservation in BacktestPageService
- Add type: ignore for BacktestHistoricalProvider protocol assignment
- Revert TypedDict change to avoid cascading type issues in pages/
- Update tests to reflect new BOUNDED policy behavior
Critical fixes:
- Add math.isfinite() check to reject NaN/Infinity in _safe_quote_price
- Raise TypeError instead of silent 0.0 fallback in price_feed.py
- Use dict instead of Mapping for external data validation
Type improvements:
- Add PortfolioSnapshot TypedDict for type safety
- Add DisplayMode and EntryBasisMode Literal types
- Add explicit dict[str, Any] annotation in to_dict()
- Remove cast() in favor of type comment validation
- Fix return type annotation for get_default_premium_for_product
- Add type narrowing for Weight|Money union using _as_money helper
- Add isinstance checks before float() calls for object types
- Add type guard for Decimal.exponent comparison
- Use _unit_typed and _currency_typed properties for type narrowing
- Cast option_type to OptionType Literal after validation
- Fix provider type hierarchy in backtesting services
- Add types-requests to dev dependencies
- Remove '|| true' from CI type-check job
All 36 mypy errors resolved across 15 files.
- Remove '|| true' from CI type-check job to enforce strict checking
- Begin type narrowing pattern in units.py with _typed property accessors
- Document all 42 type errors across 15 files in roadmap backlog
- Priority: medium, estimated 4-6 hours to complete
Type errors fall into categories:
- Union types not narrowed after __post_init__ coercion
- float() on object types
- Duplicate method definitions
- Provider interface type mismatches
1. Fix Friday logic edge case comment
- Clarified get_default_backtest_dates() docstring
- Removed confusing 'at least a week old' comment
- Explicitly documented Friday behavior
2. Reorder validation checks in validate_date_range_for_symbol()
- Now checks start > end first (most fundamental)
- Then checks end > today (future dates)
- Finally checks symbol-specific bounds
- Users get most actionable error first
3. Add server-side numeric bounds validation
- New validate_numeric_inputs() function
- Validates units > 0, loan >= 0, 0 < LTV < 1
- Called in run_backtest() before service call
4. Add boundary tests
- Test start_date exactly at SYMBOL_MIN_DATES boundary
- Test same-day date range (start == end)
- Test end_date exactly today
- Test end_date tomorrow (future)
- Test validation order returns most actionable error
- Test near-zero and large values for units calculation
- Test LTV at boundaries (0, 1, 0.01, 0.99)
5. Add tests for validate_numeric_inputs
- Valid inputs, zero/negative values
- LTV boundary conditions
- Changed default date range from 5 days (Jan 2024) to 2 years (2022-2023)
- Added SYMBOL_MIN_DATES constant documenting data availability per symbol
- GLD minimum date: 2004-11-18 (ETF launch)
- GC futures minimum date: 1974-01-01
- XAU index minimum date: 1970-01-01
- Added UI hint showing GLD data availability from ETF launch
- Users can now run backtests across the full historical range
- Changed UI input from 'Underlying units' to 'Initial portfolio value ($)'
- Underlying units are now calculated as initial_value / entry_spot
- Updated default value to workspace gold_value instead of gold_ounces * entry_spot
- Result summary now shows both 'Initial value' and 'Underlying units'
- This allows users to specify how much they invest on day 1, and the system
automatically calculates the maximum purchasable shares/contracts
- BacktestSettingsRepository.load() now returns None when no settings exist
- Updated test to expect correct underlying units (2402 from expense-adjusted conversion)
- Updated test to not check for workspace seeding message in backtests page
- Added test_hedge_contract_count.py and test_backtest_settings.py to CI test suite
- Build job now depends on lint and test passing
- Add DatabentoHistoricalPriceSource implementing HistoricalPriceSource protocol
- Smart caching with Parquet storage and metadata tracking
- Auto symbol-to-dataset resolution (GLD→XNAS.BASIC, GC=F→GLBX.MDP3)
- Cache management with age threshold invalidation
- Cost estimation via metadata.get_cost()
- Add databento>=0.30.0 to requirements.txt
- Add DATABENTO_API_KEY to .env.example
- Full test coverage with 16 tests
