Skip to main content

bse_derivatives.py — SENSEX/BANKEX Derivatives

Downloads and parses BSE's index-derivatives bhavcopy, resolves contracts against scr_bse_master, and replaces that day's rows in scr_bsefo_eod. Mirrors nseeod/FoBhav.py + the F&O-parsing half of nseeod/utils.py, scoped to BSE's index derivatives only (SENSEX, BANKEX) — per the approved plan, BSE's stock F&O is thin enough to be explicitly out of phase-1 scope.


Two Eras​

Same 2024-07-08 UDiFF cutover as bse_equity.py, confirmed live.

UDiFF (≥ 2024-07-08)​

https://www.bseindia.com/download/BhavCopy/Derivative/BhavCopy_BSE_FO_0_0_0_{YYYYMMDD}_F_0000.CSV

Same UDiFF schema as equity. FinInstrmTp ∈ {IDF, IDO, STF, STO} (index futures/options, stock futures/options) — only IDF/IDO rows are kept here, filtered further to TckrSymb in {SENSEX, BANKEX}. NewBrdLotQty carries lot size directly per row, same as NSE's current format.

Old format (< 2024-07-08)​

https://www.bseindia.com/download/BhavCopy/Derivative/bhavcopy{DD-MM-YY}.zip → bhavcopy{DD-MM-YY}.csv — a completely different, BSE-specific legacy schema, not just old NSE-style column names:

Contract Type,Symbol,Expiry,Strike,Option Type,Open,High,Low,Close,
L T Date,Wt. Avg. Price,Prem/Disc,Prem/Disc(%),Contracts Traded,
Turnover(Rs Lakhs),Open Interest,Change in OI,% Change in OI,
Cost of Carry % p.a.,Margin %,Margin Value,Is Traded

Contract Type ∈ {EF, IF, IO} (equity future, index future, index option) — only IF/IO are kept. Symbol is cryptically encoded (BSX01C42600 for a SENSEX option, BSXJAN2020 for a SENSEX future) — the underlying is recovered from the symbol's 3-letter prefix (prefix_map = {'BSX': 'SENSEX', 'BKX': 'BANKEX'}), not parsed from the contract code directly. No lot-size column at all — unlike NSE's old format (which at least has OI to gcd-derive it from). lotsize is stored as 0 for these rows, the same documented gap NSE's own getfodf has for its old F&O format ("filled later by nseeod.fo_backfill lots (OI gcd)") — a BSE lot-size-by-date reference table is a follow-up, not solved here.

BANKEX didn't always exist

BANKEX derivatives launched after SENSEX's. An old-format day with only SENSEX rows (confirmed live, Jan 2020: 65 total contracts, SENSEX only) is expected, not a parse failure.


Continuous Series — Ranked From Actual Expiries​

build_mnames() assigns the -I/-II/-III continuous-futures names by ranking each day's actual futures expiries per underlying (nearest = -I), never from calendar weekdays — the same lesson nseeod/CLAUDE.md rule 7 documents for NSE (getThursdays broke when NSE moved its monthly expiry weekday in 2025-09).


match_scr_bse_masters_and_fo_update​

Resolves underlying roots (SENSEX/BANKEX, typ=31, underly=0 — the same "this masid is a root" convention as scr_master) first, creating them if missing, then resolves each contract's masid.

New futures and options are inserted in two separate passes​

A new contract's scr_bse_master row is inserted via two separate insertdataframe calls — one for futures-typed rows (columns mname, underly, edate, typ, monthindex), one for options-typed rows (same columns plus strike, opttyp) — never one combined batch. This mirrors how NSE's match_scr_masters_and_fo_update is called once per Opt=True/False.

Combining them was tried first and crashed: a futures row given the options-only opttyp column gets NaN there (pandas aligns the missing column across a pd.concat), which psycopg2 adapts as the 3-character string 'NaN', and Postgres then rejects it against the CHAR(2) column — value too long for type character(2). Confirmed live 2026-10-11.

Day replacement​

Same "delete the whole date, then insert" rule as scr_nsefo_eod (root CLAUDE.md rule 6) — scr_bsefo_eod has no enforced primary key.


BseEodFoUpdater​

Daily sync driver — mirrors nseeod.FoBhav.NseEodFoUpdater, scoped to SENSEX/BANKEX. update_a_day(mdate) returning True with zero rows written is expected and not a failure (no trades that day, or a date before BANKEX existed) — only a genuine parse/write error counts against bse_backfill.py's fo_failed counter.