TBS engine - validation against StockMock
Runs done 17-18 Oct 2026 by logging into StockMock and starting each backtest there, then running the same spec on the engine (
backend/tbs/examples/sm_check_*.json, brokerage 0 because StockMock deducts none: its P&L is exactly (entry - exit) x lot). StockMock credits used across all runs (including the June 2025 check): 6 (36 to 30).
Results
All numbers are rupees, net, StockMock vs engine on the same days. "Within 100" = days where the two P&Ls differ by at most Rs 100.
| Case | Days | StockMock | Engine | Total gap | Median day gap | Mean abs day gap | Within Rs 100 / 300 |
|---|---|---|---|---|---|---|---|
| A. NIFTY 09:20 short straddle, SL 25 % each leg, square off all, weekly (16 Oct 2025 - 13 Jul 2026) | 177 | -19,656 | -16,666 | +2,990 | +16 | 88 | 138 / 171 of 177 |
| B. BANKNIFTY same strategy (2024, weekly era, lot scaled to StockMock's 15) | 225 | -8,824 | -11,292 | -2,468 | +0.4 | 62 | 192 / 218 of 225 |
| C. NIFTY BTST short straddle, enter 15:20, exit next day 09:20, no SL (Jan - 10 Jul 2026) | 98 | 32,299 | 32,782 | +483 | +0.2 | 22 | 94 / 95 of 98 |
| D. NIFTY positional straddle, enter 1 trading day before weekly expiry 09:20, exit expiry day 15:15 (Jan - Jul 2026) | 27 | -26,238 | -25,855 | +383 | +0.2 | 15 | 25 / 27 of 27 |
What matched exactly in all four cases:
- Same trades on the same days (BTST skips expiry days in both), same expiry resolution, same lot sizes (NIFTY 75 then 65), identical exit dates.
- The set of days on which a stop loss triggered: 100 % identical for A and B.
- Strike chosen: 0 of 177 mismatches in A, 2 of 225 in B, 0 of 27 in D, 2 of 98 in C.
- Time exits and entry fills: identical to the paisa on most legs; no-SL days differ by under Rs 1 on most days.
Fill rules learned (now in the engine)
- Entry price = open of the entry bar, then slippage 0.5 % against the trade. Slippage applies to the entry only (
costs.slippage_on: entry). - Time exit = open of the exit bar, no slippage.
- SL level = % of the slipped entry fill; the SL exit fills at that level, no slippage.
- ATM strike uses the spot at HH:MM:00, i.e. the close of the previous bar (
fill.atm_at: prev_close): 525 of 531 strikes match, against 518 for the entry bar's open. For the 15:20 BTST entry the bar's own open matched better (98 vs 96 of 100), so the BTST example setsatm_at: entry_bar. - BTST never enters on a day when a leg's contract expires ("expiry days are excluded",
timing.skip_expiry_day). - Square off all legs: the other legs exit at a price inside the trigger minute. The engine offers
square_off_at:close(trigger minute's close: unbiased, noisy),interp(linear estimate),stockmock(interp x 0.988: StockMock's price sits a steady ~1.2 % under the estimate across about 320 legs). Examples usestockmock; the default isclose. - Gap through the SL level: StockMock fills at the level; the engine default fills at the open (more realistic),
fill.gap_fill: levelmatches StockMock. Over the NIFTY year this made almost no difference because gaps through a 25 % SL are rare.
Remaining differences
- Case A runs about Rs 17 / day less negative than StockMock, case B about Rs 11 / day more negative: the
stockmocksquare-off factor is a pooled fit, so each instrument is a little off in opposite directions. Per-day errors are small (mean about Rs 62-88 on days that swing by thousands). - About 6 % of SL legs trigger in a different minute (StockMock's feed vs our 1-minute bars: a wick that one has and the other does not); these cause the few days with a gap of Rs 500-1,000.
- Different ATM on 2-4 % of days when the spot sits within a point of a strike midpoint.
- StockMock uses lot 15 for BANKNIFTY throughout 2024; the real lot became 30 from 20 Nov 2024. The engine uses the real lot (scaled for this comparison).
Data gaps found while validating (and what was done, 18 Oct 2026)
| Gap | Cause | Status |
|---|---|---|
| NIFTY 2026-07-13 and 07-14 had no index bars / were missing | EODIEOD daily files not yet ingested | Fixed: python -m nseieod.eodieod.run update processed both days; store now complete through 2026-07-14. Engine vs StockMock: 13 Jul 273.5 vs 273, 14 Jul -537 vs -438. |
| NIFTY 2026-07-08 has 6 option contracts instead of about 950 | csvs/EODIEOD2026/Option_IEOD - 08-07-2026.rar is the wrong file: it is byte-identical in size (1,080,619) to Future_IEOD - 08-07-2026.rar, so the converter reported 632 unparsed option files | Open. Needs the correct Option archive for 08-07-2026 from the vendor; then python -m nseieod.eodieod.run pipeline --date 2026-07-08 --no-sequential. The vendor Drive remote (gdrive_vendor) is not configured on this machine, so it could not be fetched here. |
| BANKNIFTY 2024-12-26 to 12-31 reported "no data" | Not missing data: scr_master lists shadow expiries (29 Jan 2025 beside the real 30 Jan, same for Feb) and the weekly rule picked the empty one | Fixed in the loader: an expiry with no bars at all is skipped and the next one used (flag "expiry ... has no bars (shadow contract?)"). The shadow rows themselves are still in scr_master; clean them with nseieod/expiry_mismatch.py (find_shadow_contracts) when convenient. |
| India VIX 2026-07-14 whole day missing | EODIEOD did not deliver it | Filled by the pipeline with a low-confidence restore (fill flag -5); report to the vendor. |
Phase 2 validation (18 Oct 2026)
NIFTY ATM short straddle, entry 09:20, no SL unless stated, 2026 data, slippage on entry only, gap_fill level, square_off_at stockmock. Example specs: backend/tbs/examples/sm_check_f_reentry_cost.json, sm_check_g_range_breakout_leg.json.
| Case | Days | Result |
|---|---|---|
| Range breakout (leg premium range 09:20-09:45, one leg square-off) Jan-Apr 2026 | 79 | Day P&L: median gap Rs 10, MAD Rs 49, 73 of 79 days within Rs 100, 78 within Rs 300. 127 legs entered by both: 116 in the same minute; entry fill within 0.5 for 99, mean abs 0.33. One leg (9 Feb, CE) entered at StockMock but never in the engine (a one-tick break at the range edge). |
| Re-entry at cost on SL (candle-close trigger) Jan-Jul 2026 | 195 | Re-entry minutes match StockMock when the trigger is the candle close (touch triggers re-entered too early). Residual gaps come from feed differences cascading through the chain, not from the rule. |
| Combined premium | - | Not offered by StockMock, so not validated. |
| Re-execute on target | - | Not run (credits). |
Fill rules confirmed: range break fills at the broken level (open on a gap), strict break, range = bars from the entry time to the range end; at-cost re-entry fires on a candle close through the original entry price and fills at the next bar open (reentry.trigger = close).
SENSEX validation (19 Oct 2026)
SENSEX short ATM straddle, weekly, spot ATM, Square Off = Leg (our partial), 4 Nov 2024 to 16 Oct 2026, StockMock-style settings (slippage 0.5 on entry only, brokerage 0), data on the VPS.
| Strategy | StockMock net | Our engine net | Days (both) | Days within Rs 100 | Median day gap |
|---|---|---|---|---|---|
| 09:30, SL 50% each leg | -40,880 | -39,120 | 481 | 473 | Rs 0.1 |
| 09:42, SL 20% each leg | +161,357 | +154,196 | 481 | 474 | Rs 0.1 |
- StockMock's Entry Price column is already slipped: it equals our raw bar price x 0.995. Exit prices agree (904 of 921 within Rs 1).
- ATM from the previous bar close picks StockMock's strike on 98.3% of days (best of the spot references tried: entry-bar open 97.6%, previous open and bar close far lower); the misses are spots within a few points of a strike midpoint and go both ways, so it is feed noise. 8 days with a different strike or a stop-loss minute off by 1-7 minutes cause almost all of the remaining total gap (-6.8k of -7.2k for 09:42); on the other 456 days the mean gap is -Rs 0.8.
- Data bug found and fixed: 17 days (2025-01-08..10, 01-15..17, 01-29..31, 02-05..07, 03-26..28, 2026-01-09, 01-12) were skipped although the vendor archives and the store hold their bars.
scr_bse_masterlists stray expiry dates (Fridays after SENSEX moved its weekly to Tuesday, e.g. 2025-01-10 with 46 contracts and no bars; 2026-01-14 vs 15), the BSE loader took the first listed expiry as "the weekly", found no bars and returned no day.tbs/data_bse.pynow skips an expiry with no bars or fewer than 10 contracts that traded and uses the next one (the NSE loader already did this for shadow contracts). Before the fix: 464 days and totals of -65,205 / +164,712 (the missing days held +26.4k / -10.5k of StockMock P&L); after: 481 days as StockMock. - Square-off mode matters: the same 09:42 / SL 20% strategy with "square off all legs" gives +71.5k (451 stop-loss days: StockMock +100.3k, all-legs run +10.6k; the 30 days without a stop loss agree, +61.1k vs +61.0k), with "Only that leg" +154k. StockMock's "Square Off: Leg" (its default) is our "Only that leg": since 19 Oct 2026 that is also the builder's default, the results page shows the settings a run used as chips under its title (the all-legs mode is highlighted), and the bulk sweep assumes it.
- Still open:
expiries_between(positional schedules) still lists the stray expiries for SENSEX / BANKEX.
Lot sizes and data ranges against StockMock's note (19 Oct 2026)
StockMock prints a note under every backtest: data availability and a lot-size table per index. Ours: the lot size of a day is the most common lotsize of the loaded chain contracts in the EOD data (NSE bhav), the BSE underlyings use a default table because the BSE F&O EOD has no lot size (the day is flagged). Measured with tbs.data.load_day (the loader the backtests use), monthly samples with each change date bisected.
| Underlying | Data from (StockMock / ours) | Lot sizes, StockMock vs ours |
|---|---|---|
| NIFTY | 15 Feb 2019 / 11 Feb 2019 (first weekly chain) | Same, with one typo in StockMock's note: it prints "25 till 26 Dec'25, 75 till 30 Dec'25" but means 26 Dec 2024: its own saved results show 75 for 2025, and ours switches 25 to 75 on 27 Dec 2024. Ours: 75, 50 from 26 Jul 2021 (StockMock 23 Jul), 25 from 26 Apr 2024, 75 from 27 Dec 2024, 65 from 31 Dec 2025. |
| BANKNIFTY | 2 Jan 2017 / 1 Jun 2016 (start of the IEOD store) | Differs on purpose: StockMock simplifies to 25 until 20 Jul 2023 and 15 after. We use the exchange lots: 40 (Jul 2016), 20 (26 Oct 2018), 25 (24 Jul 2020), 15 (24 Jul 2023), 30 (26 Dec 2024), 35 (27 Jun 2025), 30 (31 Dec 2025). Compare P&L with StockMock only where both agree (21 Jul 2023 to 25 Dec 2024), or scale by the lot ratio. |
| FINNIFTY | 19 Oct 2022 / 2 Mar 2022 | Same: 40, 25 from 24 Jul 2024, 65 from 3 Feb 2025 (first sampled day after 28 Jan), 60 from 31 Dec 2025. |
| MIDCPNIFTY | - / 8 Jun 2026 (store has little) | Same within a month of sampling: 75, 50 (Aug 2024), 120 (Feb 2025), 140 (Jul 2025), 120 (Jan 2026). |
| SENSEX | 14 Aug 2023 / 7 Aug 2023 | Same: 10, 20 from 6 Jan 2025 (StockMock "10 till 3 Jan 2025"), default table (flagged). |
| BANKEX | - / 11 Aug 2023 | default table 15, 30 from 2025 (approximate, flagged). |
Weekly options: since 20 Nov 2024 only NIFTY and SENSEX still have weekly expiries (SEBI: one weekly index per exchange); for BANKNIFTY, FINNIFTY, MIDCPNIFTY and BANKEX the leg expiry "weekly" is then the nearest monthly expiry.
Not yet validated
Target, trailing SL, move SL to cost, wait and trade, re-execute on target and combined premium against StockMock; BANKEX; delta strikes. Each needs its own StockMock run.