Options portfolio backtest - Rs 10L / 15L / 25L
Status (2026-11-08): plan written, nothing built. Phase 0 (owner decisions) comes before any code.
Goal: find NIFTY option strategies that hold up out of sample, and combine them into three portfolios (Rs 10 lakh, 15 lakh, 25 lakh) that are replayed together on one timeline with one cash and margin ledger. The aim is a robust combined portfolio, not the best single backtest.
1. The funnel
Sweep (cheap screen: entry time x SL, 720 combinations a day)
-> promote the best cells to full runs (trades and minute curves exist)
-> validate: in sample / out of sample / walk-forward
-> stress: costs, slippage, delayed exit, gap
-> correlation and risk families
-> portfolio replay (one cash ledger, combined margin)
-> three baskets: Rs 10L, 15L, 25L
-> owner approval (never automatic)
2. What we already have (reuse, do not rebuild)
| Piece | File | What it gives |
|---|---|---|
| Engine | backend/tbs/engine.py (run_day) | One day, minute by minute; BUY and SELL legs; all 9 strike rules; stop loss, target, trail, re-entry |
| Bulk sweep | backend/tbs/sweep.py | Entry time x SL grid per day; strike rule, combined SL / target, move SL to cost; costs re-priced afterwards |
| Runs and trades | backend/tbs/runner.py, store.py | bt_run (summary, daily curve) and bt_trade (every leg with times, prices, reason) |
| Re-filter a run | backend/tbs/runview.py | Filter by date, weekday, DTE and re-summarise without a rerun (basis for in sample / out of sample) |
| Baskets | backend/tbs/basket.py, dayview.py | Merge several strategies, weights, basket SL / target, merged minute P&L curve (merge_minute_curves) |
| Exact margin | backend/tbs/spanmargin.py | SPAN + exposure per day of the legs held together; a bought wing lowers margin; BSE not covered |
| Metrics | backend/tbs/metrics.py | Net, win rate, profit factor, drawdown with dates, streaks, by year / month / weekday |
| Job queue | backend/tbs/pool.py, backend/app/api/tbs.py | Queued background jobs, cancel, limits (TBS_MAX_QUEUE), stale jobs failed at start |
| Library | backend/library/store.py | Saved strategies and reports per user |
| Export | backend/tbs/export.py | Excel and PDF |
3. Specification item against status
| Specification item | Status |
|---|---|
| Strategy builder, all strike rules, SL / target / trail / re-entry | Done (TBS) |
| Short straddle and strangle, naked | Done (sweep, strike rule + combined SL) |
| Protected short (wing leg), long straddle, long strangle | In the engine; not in the sweep (New) |
| Staged search (structure, time, strike, management) | Partly: entry x SL only; strike rule fixed per sweep |
| Trade ledger with leg times, prices, costs | Done (bt_trade) |
| Cost model and slippage | Done (costs.py, re-priced in the sweep report) |
| Performance metrics | Partly: no Calmar, Sharpe, p95 / p99 loss, return on margin |
| Rejection gates and risk policy | New |
| In sample / out of sample / walk-forward | New (data exists, no split tool) |
| Stress tests | Partly: cost and slippage re-pricing in the sweep report only |
| Correlation, risk families | New |
| Exact margin per run | Done (spanmargin, end of day file) |
| Combined margin of several runs on one timeline | New |
| Capital and cash ledger | New |
| Optimizer for three baskets | New |
| Portfolio replay | Partly: baskets merge P&L and minute curves, no capital |
| Risk states NORMAL / CAUTION / HALT | New |
| Experiment batch queue and leaderboard | New |
| Dashboards | Partly: Bulk report, Reports, Basket pages exist |
4. Phases and tasks
Owner and size are left for the lead to add on each line.
P0 - Decisions before any code (owner)
- Risk policy values: daily loss, weekly loss, maximum drawdown, per strategy and per expiry concentration, minimum trade count, out of sample thresholds. Frozen and versioned, never taken from the best backtest
- What the 60 percent margin ceiling measures: initial margin, peak intraday margin, or stressed margin
- Margin source: exact SPAN (
spanmargin). The Rs 3 lakh (naked) and Rs 1.5 lakh (protected) per lot figures are planning numbers only - Protective leg rules: strike, quantity ratio, expiry
- Emergency exit order when one leg cannot be filled
- Date split fixed up front: development / validation / final test (final test opened once)
P1 - Sweep v2 (leg templates)
- Templates in
sweep.make_strategy: short straddle, short strangle, long straddle, long strangle, and protected variants with a wing leg. - Result layout per leg count instead of the fixed 4 values (
sweep_store,sweep_report). -
leg_netfor bought legs (entry slipped up, exit slipped down). - Report header and labels follow the template; keep
MAX_COMBOS. - Tests in
tests/test_tbs_sweep.pyfor each template.
P2 - Metrics v2 (backend/tbs/metrics.py)
- Calmar, Sharpe and Sortino on daily P&L, return on peak margin.
- p95 / p99 daily loss, worst 5 day loss, worst expiry day, largest single trade loss.
- Trigger to fill slippage reported apart from entry slippage.
- Follow the report conventions in
CLAUDE.mdrule 11 (return as CAGR from one year, percent on previous close, drawdown as percent of peak, benchmark shown).
P3 - Promote and validate
- "Promote cell to full run": re-run the chosen entry time and SL through
run_backtestso trades and minute curves exist. - In sample / out of sample / walk-forward windows over a stored run's daily series (
runviewdate filter, no rerun). Window sizes are settings. - Gates from the P0 risk policy applied per candidate, with the reason for every rejection stored.
- Stability check: a candidate is judged on its neighbours (entry +/- 10 minutes, SL +/- 10 percent), not on its single best cell.
P4 - Stress
- Cost and slippage x1.5 and x2, re-priced from
bt_trade. - Delayed exit, missed fill, gap and expiry day spike scenarios.
- Higher margin scenario.
- Every stress result reproducible from the stored run and a stated scenario.
P5 - Correlation and risk families
- Daily, weekly and downside correlation, tail co-loss days, from the daily series of the runs.
- Group candidates by risk family: naked short, protected short, long volatility. Variants of one strategy are one family, not independent strategies.
P6 - Portfolio ledger and replay
-
backend/tbs/portfolio/ledger.py: one cash ledger, equity = cash + option premium deployed + margin blocked. Test that cash is never counted twice. - Replay the chosen runs on one timeline using
dayview.merge_minute_curvesandspanmargin.day_sets, so margin is of the legs held together, never a sum of per strategy margins. - Integer lots; a long option's premium comes out of the same cash as short option margin.
- Output: portfolio equity, drawdown, margin used, cash reserve, contribution per strategy.
P7 - Optimizer for Rs 10L / 15L / 25L
- All three baskets from one approved candidate pool.
- Choose strategies, integer lots, naked or protected, long option premium budget; cash may stay unused.
- Hard limits first (margin ceiling, loss limits, concentration), ranking second. If nothing passes: "no feasible portfolio", never a forced allocation.
- Start with a greedy / small search, not a solver.
P8 - Risk engine (backtest only)
- NORMAL / CAUTION / HALT states in the replay, each change logged with its reason.
- HALT follows the P0 exit order, never "close everything at once".
P9 - Screens (extend what exists)
- Bulk report page: strategy template choice and a "Promote" button.
- Reports page: candidate status, leaderboard, rejection reasons.
- Basket page: capital panel, replay, risk events. A separate Experiment Manager page only if P10 is built.
P10 - Automation
- One research pipeline job (configuration in, leaderboard out): sweep, promote the best N per template, validate, stress, correlate, candidate list. It runs on the existing queue (cancel and limits as today).
- Run on demand and, later, monthly when new data arrives.
- Optional: monthly re-score of approved candidates, with a flag when one decays.
P11 - Docs and tests
- A knowledge-base page for each finished phase.
- Tests:
tests/test_tbs_sweep.py(templates), newtests/test_portfolio_ledger.py(cash invariant, integer lots, no feasible portfolio), metrics tests.
5. Better than the specification (and why)
| Change | Reason |
|---|---|
No new research/options_portfolio package and 17 option_* tables | bt_run, bt_trade, usr_report already hold runs, trades and reports. Add only candidate status, risk policy version and portfolio allocation (teudb, one Alembic revision) |
| Replay uses promoted full runs, not sweep cells | A sweep cell keeps only raw leg prices, no exit time or minute curve |
| Judge by neighbourhood and out of sample | Thousands of cells always contain winners; the best in sample cell is mostly luck |
| Exact SPAN margin first | Already built; the Rs 3L / 1.5L figures are only a sanity check. Limits stay: end of day file, no SENSEX / BANKEX |
| NIFTY weekly first | Matches the specification; other underlyings need their own data and margin checks |
| Build order P2, P1, P3 first | One validated candidate is worth more than a portfolio built on unvalidated ones |
6. Risks and open questions
- Overfitting is the main risk: freeze the final test period and open it once.
- Margin: end of day SPAN differs from the broker's intraday figure; BSE is not covered.
- Combined margin of simultaneous legs is the hardest part of P6; check it against a broker margin calculator on a few real examples.
- Expiry day and gap losses dominate short volatility results; report them apart.
- Open: should long option strategies get a premium budget per strategy or one shared cap?
7. Progress log
- 2026-11-08 plan written; nothing built.