- Fix BacktestPageRunResult serialization in jobs.py to correctly access
nested fields from scenario and run_result objects
- Add test_backtest_job.py with comprehensive tests for job execution
- Add conftest_playwright.py with ServerManager that starts FastAPI server
for Playwright tests using uvicorn
- Add test_playwright_server.py with E2E tests using the server fixture
The job serialization bug was causing backtest results to fail silently
because it was trying to access non-existent fields on BacktestPageRunResult.
The backtest engine was always using fixture data (limited to 2024-01-02
through 2024-01-08) regardless of the data_source selection. The fix
fetches historical prices using the specified data source (Databento,
Yahoo Finance, or synthetic) and passes them directly to the engine.
- Pin black version in requirements-dev.txt (was >=24.0.0)
- Update pre-commit to use black 26.3.1 with Python 3.12
- Add language_version: python3.12 to pre-commit black hook
- Reformat files with new black version for consistency
- Remove app/components/ and app/pages/ from ruff/black excludes
- Pre-commit reformatted multi-line strings for consistency
- All files now follow the same code style
The backtest engine uses a fixture provider (synthetic_v1) regardless of
the data_source used for price fetching. We must use the fixture provider's
ID for the scenario, not the data source's ID.
This fixes 'Unsupported provider/pricing combination' error when running
backtests with data_source='databento'.
BREAKING CHANGE: Complete redesign of backtest execution
- Add BacktestJob system with progress stages (validating, fetching_prices, calculating)
- Run backtests in background threads, UI polls for status
- Show progress label with current stage during execution
- Remove synchronous Databento API calls from page load
- Use static default entry spot for initial render (defers API call)
- Make refresh_workspace_seeded_units async with run.io_bound
This fixes:
- 'Connection lost' WebSocket timeout errors
- Slow page load (30s initial load)
- Backtest never completing
The job system provides:
- Non-blocking execution
- Progress tracking with stages
- Error handling with user-friendly messages
- Result caching for retrieval after completion
- Use run.io_bound() from NiceGUI to run Databento API calls in background thread
- Add loading state to Run Backtest button
- Show notification when backtest starts and completes
- Remove loading state on completion/error
This prevents 'Connection lost' errors when the backtest takes longer than the WebSocket timeout.
- on_form_change: Only update cost estimate, skip expensive derive_entry_spot
- Only call derive_entry_spot on date changes (start/end inputs)
- Other inputs (template, units, loan, LTV) just mark results stale
- This reduces lag from constant API polling
- Set default dates to 2024-07-01 to 2024-12-31 (valid for XNAS.BASIC)
- Catch all exceptions during entry spot derivation, not just ValueError
- Don't auto-run backtest on page load - let user configure first
- Use recent GLD price (~30) as fallback
- Add DATABENTO_DATASET_MIN_DATES for XNAS.BASIC (2024-07-01) and GLBX.MDP3 (2010-01-01)
- Validate start date against dataset minimum before running backtest
- Parse Databento API errors and show user-friendly messages
- Update date range hint to show dataset-specific availability
- Catch BentoClientError and show appropriate warning tone
- Pass data_source to derive_entry_spot in backtests.py
- Remove default 'synthetic' value for data_source in derive_entry_spot and validate_preview_inputs
- Update all tests to explicitly pass data_source parameter
- Improve error message with helpful suggestion for Databento/Yahoo Finance
- Change WindowPolicy from EXACT to BOUNDED for backtest fixture
- Pass data_source to run_read_only_scenario so real data can be used
- Fix injected provider identity preservation in BacktestPageService
- Add type: ignore for BacktestHistoricalProvider protocol assignment
- Revert TypedDict change to avoid cascading type issues in pages/
- Update tests to reflect new BOUNDED policy behavior
Critical fixes:
- Add math.isfinite() check to reject NaN/Infinity in _safe_quote_price
- Raise TypeError instead of silent 0.0 fallback in price_feed.py
- Use dict instead of Mapping for external data validation
Type improvements:
- Add PortfolioSnapshot TypedDict for type safety
- Add DisplayMode and EntryBasisMode Literal types
- Add explicit dict[str, Any] annotation in to_dict()
- Remove cast() in favor of type comment validation
- Fix return type annotation for get_default_premium_for_product
- Add type narrowing for Weight|Money union using _as_money helper
- Add isinstance checks before float() calls for object types
- Add type guard for Decimal.exponent comparison
- Use _unit_typed and _currency_typed properties for type narrowing
- Cast option_type to OptionType Literal after validation
- Fix provider type hierarchy in backtesting services
- Add types-requests to dev dependencies
- Remove '|| true' from CI type-check job
All 36 mypy errors resolved across 15 files.
- Remove '|| true' from CI type-check job to enforce strict checking
- Begin type narrowing pattern in units.py with _typed property accessors
- Document all 42 type errors across 15 files in roadmap backlog
- Priority: medium, estimated 4-6 hours to complete
Type errors fall into categories:
- Union types not narrowed after __post_init__ coercion
- float() on object types
- Duplicate method definitions
- Provider interface type mismatches
1. Fix Friday logic edge case comment
- Clarified get_default_backtest_dates() docstring
- Removed confusing 'at least a week old' comment
- Explicitly documented Friday behavior
2. Reorder validation checks in validate_date_range_for_symbol()
- Now checks start > end first (most fundamental)
- Then checks end > today (future dates)
- Finally checks symbol-specific bounds
- Users get most actionable error first
3. Add server-side numeric bounds validation
- New validate_numeric_inputs() function
- Validates units > 0, loan >= 0, 0 < LTV < 1
- Called in run_backtest() before service call
4. Add boundary tests
- Test start_date exactly at SYMBOL_MIN_DATES boundary
- Test same-day date range (start == end)
- Test end_date exactly today
- Test end_date tomorrow (future)
- Test validation order returns most actionable error
- Test near-zero and large values for units calculation
- Test LTV at boundaries (0, 1, 0.01, 0.99)
5. Add tests for validate_numeric_inputs
- Valid inputs, zero/negative values
- LTV boundary conditions
- Changed default date range from 5 days (Jan 2024) to 2 years (2022-2023)
- Added SYMBOL_MIN_DATES constant documenting data availability per symbol
- GLD minimum date: 2004-11-18 (ETF launch)
- GC futures minimum date: 1974-01-01
- XAU index minimum date: 1970-01-01
- Added UI hint showing GLD data availability from ETF launch
- Users can now run backtests across the full historical range
- Changed UI input from 'Underlying units' to 'Initial portfolio value ($)'
- Underlying units are now calculated as initial_value / entry_spot
- Updated default value to workspace gold_value instead of gold_ounces * entry_spot
- Result summary now shows both 'Initial value' and 'Underlying units'
- This allows users to specify how much they invest on day 1, and the system
automatically calculates the maximum purchasable shares/contracts
- BacktestSettingsRepository.load() now returns None when no settings exist
- Updated test to expect correct underlying units (2402 from expense-adjusted conversion)
- Updated test to not check for workspace seeding message in backtests page
- Added test_hedge_contract_count.py and test_backtest_settings.py to CI test suite
- Build job now depends on lint and test passing
- Add DatabentoHistoricalPriceSource implementing HistoricalPriceSource protocol
- Smart caching with Parquet storage and metadata tracking
- Auto symbol-to-dataset resolution (GLD→XNAS.BASIC, GC=F→GLBX.MDP3)
- Cache management with age threshold invalidation
- Cost estimation via metadata.get_cost()
- Add databento>=0.30.0 to requirements.txt
- Add DATABENTO_API_KEY to .env.example
- Full test coverage with 16 tests
