Skip to main content

TBS backtester (time based strategies)

Backtests multi-leg option strategies that enter and exit by time (straddles, strangles, spreads, condors) on the 1-minute IEOD store. Package backend/tbs/; plans and feature research in developer/todolist/tbs/ (engine spec, StockMock validation).

Modules​

FileRole
spec.pyStrategy JSON (pydantic): legs, strike rules, SL / target / trail, overall rules, timing, fill and cost options
strikes.pyStrike rules: ATM offset, ATM points, ATM %, closest premium, premium >= / <= / range, straddle width, % of straddle
engine.pyEvent-driven simulator for one trade (run_day); intraday, BTST or positional (sessions joined by concat_days)
data.py / data_bse.py / loaders.pyDay loaders: NSE via Con_Price_Getter, BSE via bseieod.reader (spot synthesized from put-call parity when no spot store exists)
runner.pyschedule() entry / exit dates, run_backtest() with progress, cancel and workers (process pool)
metrics.pyNet / gross / charges, win rate, profit factor, max drawdown, streaks, month and weekday tables
store.pybt_run / bt_trade persistence (models in strategies/models.py, migration c3a91d5e7b42)
templates.pyReady-made specs served by the API
app/api/tbs.py/api/tbs/*

Trade modes​

  • intraday: enter and exit the same day (timing.entry, timing.exit).
  • btst: enter on a run-on day, exit the next trading day at timing.exit.
  • positional: per expiry (positional_expiry weekly or monthly), enter entry_dte trading days before it, exit exit_dte days before it (0 = expiry day). A leg whose contract expires before the exit is flagged and uses its last traded price. Overnight gaps fill at the next open. SL monitoring runs on every bar of every session in the trade.

Fill model (checked against StockMock, see todolist/tbs/stockmock-validation)​

Entry at the open of the entry bar with slippage; time exit at the open of the exit bar; ATM from the previous bar's close (fill.atm_at); SL / target from the slipped entry fill, filled at the level (gap: fill.gap_fill open or level); SL first when both are hit in one bar; square-off-all exits the other legs per fill.square_off_at (close, interp, stockmock). costs.slippage_on: both (default) or entry (StockMock). BTST skips expiry-day entries (timing.skip_expiry_day). The sm_check_* examples carry the StockMock-matching settings. costs.charges full uses an approximate statutory table - verify before relying on it.

Run it​

cd backend
python -m tbs.run --spec tbs/examples/sm_check_b_straddle_sl25.json --from 2025-06-02 --to 2025-06-30 --workers 4

About 2 s per trading day (about 0.7 s with 4 workers). --trades prints every leg, --out file.json saves summary, daily curve and trades.

API (DEVELOPER rank): GET /api/tbs/meta, GET /api/tbs/templates, POST /api/tbs/validate, POST /api/tbs/run (returns run_id; one run at a time, 409 when busy), GET /api/tbs/runs, GET /api/tbs/runs/{id} (status, trades_done / trades_total, summary, daily curve), GET /api/tbs/runs/{id}/trades, POST /api/tbs/runs/{id}/cancel, DELETE /api/tbs/runs/{id}.

Data availability (local, 17 Oct 2026)​

NIFTY and BANKNIFTY options 2016-06 to 2026-07-13; FINNIFTY usable from 2023; MIDCPNIFTY from 2024. SENSEX / BANKEX: only the sparse Fyers store is on this machine (no spot file, so spot is synthesized); sync the vendor eodieod/ store for full history. A day with no data is skipped and counted in skipped_days. The loader drops an expiry that has no bars at all (a shadow contract in scr_master) and uses the next one; positional scheduling keeps one weekly expiry per calendar week. Known open gap: NIFTY 2026-07-08 options (the delivered Option_IEOD - 08-07-2026.rar is a copy of the Future file).

Tests​

backend/tests/test_tbs_engine.py, test_tbs_runner.py, test_tbs_api.py, test_tbs_phase2.py (no database needed; 68 tests).

UI page (/tbs, DEVELOPER rank, sidebar "TBS Backtest")​

frontend/src/pages/TbsBacktest.jsx (+ tbsSpec.js): strategy form (underlying, mode, times, run-on days, positional DTE), leg builder (segment, side, type, lots, expiry, strike rule with its own fields, SL / target / trail / wait-and-trade, start time, monitor), strategy rules, fills and costs, a JSON import / export panel, template and past-run pickers. Run backtest starts a run and polls /api/tbs/runs/{id} every 1.5 s (progress bar, cancel). Results: Summary (stat cards, monthly bars, weekday table, leg exits), Equity (cumulative net and drawdown, drag to zoom), Day-wise (sortable) and Trades (filter by exit reason, CSV). The browser only edits the JSON and displays server numbers; no P&L logic is in the frontend. Verified against the live API with a throwaway no-auth harness (not committed).

Date range rules (From / To)​

  • From can never be earlier than the first day the underlying has a weekly option chain in the store (tbs/availability.py, WEEKLY_FROM: NIFTY 2019-02-11, BANKNIFTY 2016-06-01, FINNIFTY 2022-03-02, SENSEX 2023-08-07, BANKEX 2023-08-11, MIDCPNIFTY 2026-06-08); To can never be later than the last day the data is updated to (looked up live, cached ten minutes). The date pickers enforce both (min / max) and the API answers 400 outside the range (POST /api/tbs/run, POST /api/tbs/sweeps).
  • Default range: the last six months of data, i.e. From = last updated day minus six months (not earlier than the first weekly date), To = the last updated day. It is set again whenever the underlying changes, on the builder (Run section and the results filter bar) and on the Bulk Report page. The All button sets From to the first weekly date.
  • No limit on the number of days of a range: TBS_MAX_RANGE_DAYS and TBS_MAX_SWEEP_DAYS default to 0 (no limit) and can still be set on a server that needs a cap. Long ranges just take longer and wait in the queue.
  • API: GET /api/tbs/availability?underlying=NIFTY returns data_from, data_to and default_from.

Result filters, charts and re-run​

  • Filter bar (above every results tab, as StockMock): From / To dates and Slippage % change what is simulated, so Run again starts a new backtest with them (the server runs it, the new run opens when done). Days (Mon to Fri), DTE (0 to 4), India VIX range, Gap up / down % and Open vs previous day (above PDH, inside, below PDL) only choose among the days already in the run: GET /api/tbs/runs/{id}/view re-summarises the stored legs (tbs/runview.py, nothing is re-simulated) and every tab (metrics, equity, daily profit, day table, trades) follows. Gap = open vs previous close in percent; gap up and gap down are an OR (days that gapped up by at least x or down by at least y); the VIX is the India VIX open of the day; DTE counts trading days from the day to the expiry of the first leg. VIX, gap and previous-day data come from the index EOD rows: scr_nseeq_eod for NIFTY, BANKNIFTY, FINNIFTY, MIDCPNIFTY, scr_bseeq_eod for SENSEX and BANKEX, and the India VIX open from scr_nseeq_eod for all of them. If an underlying ever has no index rows, the bar says that the VIX / gap / previous-day filters would remove every day (it only says so when one of those three is set).
  • Equity tab: the equity curve in two styles to pick from (Neon line with high / low markers, Profit / loss split around the starting capital; the choice is remembered, Neon line is the default), the percent drawdown chart below it and a range selector: 1m, 3m, 6m, YTD, 1y, All buttons and a navigator with draggable handles that zoom both charts together.
  • Day-wise tab: a Daily profit panel above the table with four views, Profit (a bar per day), Cumulative, Drawdown and Distribution (histogram of daily profit with the mean), the same range selector and a row of day statistics (mean, median, std dev, best, worst, win days), then the day table.
  • Defaults: brokerage per order starts at 0, slippage on entry only and "Only that leg" as the square-off mode, all as StockMock. The settings a run used are shown as chips under its title (the "ALL legs together" mode and any brokerage / charges are highlighted).

Bulk Entry x SL report (TBS Bulk Report page)​

A sweep runs a short ATM straddle (both legs sold, each with the same % stop loss, no move-SL-to-cost) over every entry time x SL level x trading day with the same engine as the single backtests (backend/tbs/sweep.py), then saves a standalone HTML report: an Entry x SL heatmap of total net P&L with TOTAL / year / MONTHLY filters, KPI tiles (total P&L, win rate, profitable trades, trading days), a day-by-day curve for the clicked cell and the best and worst 10 combinations. Sidebar: TBS Bulk Report (/tbs-sweep); NIFTY, BANKNIFTY, FINNIFTY, MIDCPNIFTY, SENSEX and BANKEX (SENSEX / BANKEX data lives on the VPS).

  • Settings: underlying, date range (no limit by default, TBS_MAX_SWEEP_DAYS can set one), first and last entry and the step in minutes (defaults 09:17 to 15:25 every 5 minutes, only times before the exit), exit time (default 15:15), SL from / to / step (defaults 10 to 100 step 10), expiry (weekly, next weekly, monthly), DTE filter (0 = expiry day only, 1 = the day before expiry, counted in trading days; empty = all days), what happens when one leg hits SL (only that leg, or square off all) and the report's starting cost values.
  • Engine and speed: each worker process loads a day once (the TBS day cache applies) and runs all combinations on it, about 1 ms per combination (NIFTY, 75 times x 10 SLs x 735 days is 551,000 simulations, a few minutes on 4 workers). Sweeps wait in the same queue as backtests and respect the same worker cap. The fill model is the StockMock-matched one (fixed); lot size is the day's lot size from the EOD data.
  • Skipped sessions: the day list comes from the NSE trading calendar, which includes special sessions (Diwali Muhurat, Saturday / Sunday live sessions) that trade other hours than 09:15-15:30 and have no regular bars. Such days are skipped and listed with a reason (special session HH:MM-HH:MM, special session (weekend), or no data for a real gap): the sweeps list shows the count with the dates on hover, the report footer lists them, and only real gaps count as "without data". Sweeps run before this was added do not have the list.
  • What is stored: per day, entry time and SL level, the raw entry and exit prices of the CE and PE leg (float32; NaN = leg not traded) in backend/tmp/tbs_sweeps/<id>.npz plus a .json with the settings and status (files, owner = the user id; a user sees only their own sweeps). The report embeds the same array (zlib, base64), about 3 MB for 3 years of NIFTY.
  • Costs in the report: the page applies the costs when it loads and again on every change: slippage % (entry only or entry and exit), brokerage per lot per order (2 orders per leg), lots, include other charges, NSE / BSE and every charge rate. Charges follow the Zerodha F&O options calculator: STT 0.15% of sell value, exchange transaction 0.03553% of turnover (NSE; the BSE rate 0.0325% is an assumption to verify), SEBI Rs 10 per crore, GST 18% on brokerage + exchange + SEBI, stamp duty 0.003% of buy value; STT and stamp round to whole rupees, the others to paise (checked on the calculator example: buy 100, sell 110, 400 qty gives net 3850.48). The same formula exists in Python (tbs/sweep.py leg_net) and in the report's JavaScript; a test compares them.
  • One limit: SL trigger levels were fixed in the run with its slippage (SL is a % of the slipped entry fill). Changing slippage in the report re-prices fills and costs but does not move the SL triggers; re-run the sweep for an exact result at another slippage.
  • API: POST /api/tbs/sweeps ({spec, workers}), GET /api/tbs/sweeps, GET /api/tbs/sweeps/{id}, GET /api/tbs/sweeps/{id}/report (inline=true to view, otherwise an attachment), POST .../cancel, DELETE ....

Results page (StockMock layout)​

tbs/metrics.py summarize() produces the figures and the Summary tab shows them as StockMock does: Estimated Margin, Overall Profit, Avg Day Profit, Max Profit, Max Loss, Win% and Loss% (days), Avg Monthly Profit, Avg Profit on Win Days, Avg Loss on Loss Days, Max Drawdown, MDD Days (Recovery Days), Return to MDD Ratio, Max Winning / Losing Streak, Expectancy, 0 / 1 / 2 SL-TP Hit Count (days on which that many legs ended on SL or target), MTM SL / target day counts, the count of each winning and losing streak length (3 or more), the Day Wise Breakup (year by weekday) and the Monthly Breakup (year by month, total, MDD, ROI) with the Show ROI percent switch.

  • Percentages are relative to an estimated margin: 10.5% of the net sold strike notional (futures at the entry price), the largest over all days. This is an approximation without SPAN (StockMock's own margin is also marked beta); it is also the starting capital of the equity chart.
  • Definitions matched to StockMock: Avg Monthly = net / (calendar span / 30); Expectancy = (win% x avg win - loss% x avg loss) / avg loss; Return to MDD = abs(average yearly profit / max drawdown); MDD Days = calendar days from the peak before the deepest drawdown to its recovery (or the last day if still running), in brackets the days from the trough to recovery; the yearly MDD in the monthly table restarts each year.
  • Equity tab: two stacked charts as in AmiBroker: Portfolio Equity (estimated margin plus cumulative net, gradient area, last value tag) and Drawdown in percent below the highest equity so far (blue gradient, red lines marking every new record drawdown, current value tag), values on the right axis, year ticks.

Saved strategies, locked fill model, side panel​

  • Saved strategies (now in the library with folders and saved reports, see library-and-reports; the text below is the first version, table usr_tbs_strategy was replaced by usr_strategy) (usr_tbs_strategy in teudb, the user database; migration a41c7e9d2f35 in backend/alembic_user, applied by pull.sh with the other chains): the builder's Save button (header and side panel) stores the strategy JSON under its name for the signed-in user (usrid links to usr_master, rows go when the user is deleted); saving the same name replaces it. The side panel lists them with a star (favorites first) and a delete button; clicking one loads it into the builder. API: GET/POST /api/tbs/strategies, PATCH /api/tbs/strategies/{id} ({favorite}), DELETE /api/tbs/strategies/{id}. Each user sees only their own. Recent runs (tedb bt_run) stay below the list.
  • Locked fill model: entry price, time-exit price, SL-and-target-in-one-bar, bar-opens-through-SL, square-off-all price, ATM spot and slippage-on are fixed at the StockMock-matched values (open, open, SL first, fill at level, StockMock-fitted, previous bar close, entry only) for everybody except an ADMIN. The builder shows them read-only (GET /api/tbs/meta returns can_edit_fill, fill_locked, costs_locked), and the server overrides them in POST /run and POST /strategies for non-admins, so editing the JSON does not bypass it. Slippage percent, brokerage and statutory charges stay editable. New strategies start with these values for admins too. A signed-in user is judged by their category (only ADMIN edits) whether or not AUTH_ENFORCED is on; only an anonymous caller with authentication switched off (local development without logging in) can edit. The builder keeps the fields locked until /meta has answered. After deploying, restart the backend: a running backend still has the old rule.
  • Side panel: the arrow on its edge collapses it to icons in both Builder and Results (remembered in the browser).

How the lot size is chosen​

There is no hardcoded lot-size table in the engine. For every trading day the loader (tbs/data.py, _lot_size) reads the F&O EOD rows of the contracts it loaded for that day from the F&O history store (nseeod/fo_store, built from NSE F&O bhav copies; the rows carry a lotsize column), keeps rows with lotsize > 0 and takes the most common value (the mode) as the day's lot size. The engine then scales each leg's P&L by lots x lot_size.

  • Follows NSE revisions automatically: when NSE changes a lot size, the EOD data changes on that date and so does the backtest. Whether the dates agree with another tool (StockMock lists NIFTY as 75, 50, 25, 75, 65 on its own dates) depends on our EOD lotsize history, which has not been compared line by line yet.
  • Multi-day trades (BTST, positional): the lot of the entry day is used for the whole trade. A lot-size change in the middle of a trade is not modelled.
  • Fallback: a day with no lotsize rows at all uses the default table in tbs/data.py (DEFAULT_LOT: NIFTY 75, BANKNIFTY 30, FINNIFTY 65, MIDCPNIFTY 120) and the day is flagged "lot size from default table" (it shows as a flagged day in the results). Check the EOD store for that date.
  • Cache: the day cache stores the lot size with the day. After repairing lotsize values in the EOD store run python -m tbs.cache clear from backend/, because the cache signature only watches the IEOD Parquet files.
  • Per-expiry reads: the EOD history of each expiry's contracts is read once per expiry (cached in memory for 10 minutes) and reused for every day of that expiry.

Matching StockMock results​

Validated runs (see todolist/tbs/stockmock-validation) use StockMock-style settings: slippage_on: entry, brokerage_per_order: 0, gap_fill: level, square_off_at: stockmock. The builder defaults (slippage on entry and exit, Rs 20 per order, fill at open, minute close) are deliberately more conservative and show about Rs 160 per day lower P&L on a NIFTY 11:00 straddle with 40 percent stop loss. Check 18 Oct 2026, 1 Jan to 14 Jul 2026, 129 days: StockMock net -10,565; engine with StockMock-style settings -11,915 (median daily gap Rs 1, 111 of 129 days within Rs 100); engine with the defaults -31,656. The remaining gap comes from the price of the other legs when a stop loss squares off all legs (the stockmock factor is a pooled fit).

Server load, queue and limits​

Backtests run on the server (the browser only sends the strategy JSON, polls progress and draws results). Measured about 2 CPU-seconds per trading day; a 1-year NIFTY run is about 8 CPU-minutes. Runs go through a FIFO queue: one run executes at a time and uses up to TBS_MAX_WORKERS processes (default CPU count minus 2, niced), the others show QUEUED with their position in the page. Limits (env): TBS_MAX_WORKERS, TBS_MAX_RANGE_DAYS (default no limit), TBS_MAX_QUEUE (default 20 waiting, 429 above). The VPS (6 cores, 15 GB) gives 4 workers: about 2 minutes for a 1-year run when idle. A queued run can be cancelled and leaves the queue at once. Queued jobs live in memory, so a backend restart drops them (their rows stay RUNNING until deleted).

Speed and the day cache​

Profiling showed the simulator needs about 3 ms per trading day; the time was all in loading (about 2.5-4 s per day). Fixes (results are bit-identical to the old loader, checked on 5 days and a 79-day run):

  • tbs/data.py: the per-contract pandas reindex is replaced by one vectorised NumPy pass (_to_bars_many), and the scr_master / F&O EOD lookups (option rows, per-expiry contracts and EOD history, lot size, futures id) are cached in memory per expiry (10-minute expiry so new listings appear).
  • tbs/cache.py: a disk cache of each loaded day (backend/tmp/tbs_cache, about 0.3 MB per day, TBS_CACHE=0 off, TBS_CACHE_PATH, TBS_CACHE_MAX_MB default 4000, least recently used removed first). A cached day loads in about 25 ms. Each file stores a signature (names, sizes, mtimes) of the underlying's IEOD Parquet folder for that year, so re-ingested data is picked up automatically. A repair that only touches the F&O EOD store or scr_master (lot size, listed expiries) does not change the signature: run python -m tbs.cache clear (from backend/) after such a repair. python -m tbs.cache status shows size.
  • Measured on a 79-day NIFTY SL straddle, 4 workers: 41 s before, 22 s with the vectorised loader (cold cache), 3 s with a warm cache. A strike search cache was not built: the strike rules are part of the 3 ms simulation, not of the load.
  • Part of the remaining cold time here is the Windows localhost Postgres connect (about 90 ms each, 9 per day); Linux on the VPS should be faster.

Phase 2 features (18 Oct 2026)​

All three work on intraday strategies; they are fields in the strategy JSON and controls in the UI.

Re-entry / re-execute (leg.reentry_sl, leg.reentry_target: {mode, count, reverse}; needs overall.square_off = partial, which the UI sets for you)

  • asap (re-execute): the bar after the leg's SL / target a new leg is started and its strike chosen again from the leg's rule at that moment (ATM may have moved). The new leg carries the same SL / target / trail settings.
  • cost (re-enter at cost): the same contract is re-entered when its price comes back to the original entry price (fill at that level, or at the open on a gap).
  • count = maximum number of re-entries (shared along the chain); reverse flips BUY / SELL each time. Re-entries stop after timing.last_entry. Trades carry seq (0 original, 1.. re-entry; shown as L1#1).

Range breakout (range_breakout: {source, end, buffer, buffer_unit, entry_at}, range = entry time to end)

  • source: leg: each leg watches its own premium range; SELL legs enter when the low breaks, BUY legs when the high breaks (strictly beyond the range; fill at the break level or at the open on a gap). A leg that never breaks never trades.
  • source: underlying: all legs wait for the index to break the range high / low (entry_at: high, low or any); strikes are then chosen and the legs are filled at that bar's close. No break before last_entry means no trade that day.

Combined-premium entry (combined_entry: {premium, compare}): all legs are resolved at the entry time but enter together only when the net premium per unit (sold legs positive, bought legs negative, on bar closes) is >= (gte) or <= (lte) the value, filled at that bar's close. timing.last_entry is the deadline.

Not combined: range / combined entry cannot be used with leg start time or wait and trade, and need an intraday strategy. Range breakout (leg) and at-cost re-entry are validated against StockMock (see todolist/tbs/stockmock-validation); combined premium is not (StockMock has no such entry). At-cost re-entry takes trigger: touch (default) or close (candle close through the level, filled at the next open; matches StockMock).

Not built yet​

Delta / greeks strikes, re-execute whole portfolio, execute-other-leg actions, journey, dynamic hedge, profit lock and trail (Phase 3); StockMock validation of Phase 2 features; SENSEX / BANKEX validation; saving named strategies (runs keep their spec; templates are built in).

Report tools: period filter, contract chart, day detail (6 Nov 2026)​

On the results page of a run (and of a saved backtest on Reports):

  • Period filter. From / To in the filter bar cut the report to a period inside the run's own range (the boxes cannot go before the run's first day or after its last). Nothing is re-simulated: GET /api/tbs/runs/{id}/view?date_from=&date_to= (and the same on /api/library/reports/{id}/view) re-summarises the stored legs, together with the other filters. Slippage and other dates still need Run again (it uses the dates of the builder).
  • Contract chart. Hover (or tap) a row of the Trades table: the 1-minute candles of that contract for the day with the buy / sell signals (a sold leg shows SELL at its entry and BUY at its exit), a CSV of the minute bars with the signals. GET /api/tbs/runs/{id}/trade-chart?date=&leg_id=&seq= (report: /api/library/reports/{id}/trade-chart). The bars come from the same loaders and day cache as a run (backend/tbs/dayview.py); a multi-day trade shows every day it was held.
  • Day detail. Click a bar of Daily profit (any tab) or a Day-wise row: the minute by minute equity of that day with its drawdown, the max profit / max drawdown with their times, the legs' entries and exits, a CSV. GET .../day-curve?date=. Each leg is marked to its bar close while open and realised after its exit, charges are taken at the exit, so the curve ends exactly on the day's net P&L. Drawdown is measured from the best point so far (never above zero).
  • Locked fields. The fill-model fields stay locked for everybody but an admin, also right after a page refresh: the browser now renews an expired access token before a request (an expired token used to look anonymous, and the VPS runs with AUTH_ENFORCED=false, where an anonymous caller counted as local development); the server treats an anonymous caller as editable only off the VPS (APP_ENV != VPS).