bse_derivatives.py — SENSEX/BANKEX Derivatives
Downloads and parses BSE's index-derivatives bhavcopy, resolves contracts
against scr_bse_master, and replaces that day's rows in scr_bsefo_eod.
Mirrors nseeod/FoBhav.py + the F&O-parsing half of nseeod/utils.py,
scoped to BSE's index derivatives only (SENSEX, BANKEX) — per the
approved plan, BSE's stock F&O is thin enough to be explicitly out of
phase-1 scope.
Two Eras
Same 2024-07-08 UDiFF cutover as bse_equity.py,
confirmed live.
UDiFF (≥ 2024-07-08)
https://www.bseindia.com/download/BhavCopy/Derivative/BhavCopy_BSE_FO_0_0_0_{YYYYMMDD}_F_0000.CSV
Same UDiFF schema as equity. FinInstrmTp ∈ {IDF, IDO, STF, STO} (index
futures/options, stock futures/options) — only IDF/IDO rows are kept
here, filtered further to TckrSymb in {SENSEX, BANKEX}.
NewBrdLotQty carries lot size directly per row, same as NSE's current
format.
Old format (< 2024-07-08)
https://www.bseindia.com/download/BhavCopy/Derivative/bhavcopy{DD-MM-YY}.zip
→ bhavcopy{DD-MM-YY}.csv — a completely different, BSE-specific legacy
schema, not just old NSE-style column names:
Contract Type,Symbol,Expiry,Strike,Option Type,Open,High,Low,Close,
L T Date,Wt. Avg. Price,Prem/Disc,Prem/Disc(%),Contracts Traded,
Turnover(Rs Lakhs),Open Interest,Change in OI,% Change in OI,
Cost of Carry % p.a.,Margin %,Margin Value,Is Traded
Contract Type ∈ {EF, IF, IO} (equity future, index future, index
option) — only IF/IO are kept. Symbol is cryptically encoded
(BSX01C42600 for a SENSEX option, BSXJAN2020 for a SENSEX future) —
the underlying is recovered from the symbol's 3-letter prefix
(prefix_map = {'BSX': 'SENSEX', 'BKX': 'BANKEX'}), not parsed from the
contract code directly. No lot-size column at all — unlike NSE's old
format (which at least has OI to gcd-derive it from). lotsize is stored
as 0 for these rows, the same documented gap NSE's own getfodf has for
its old F&O format ("filled later by nseeod.fo_backfill lots (OI gcd)")
— a BSE lot-size-by-date reference table is a follow-up, not solved here.
BANKEX derivatives launched after SENSEX's. An old-format day with only SENSEX rows (confirmed live, Jan 2020: 65 total contracts, SENSEX only) is expected, not a parse failure.
Continuous Series — Ranked From Actual Expiries
build_mnames() assigns the -I/-II/-III continuous-futures names by
ranking each day's actual futures expiries per underlying (nearest =
-I), never from calendar weekdays — the same lesson nseeod/CLAUDE.md
rule 7 documents for NSE (getThursdays broke when NSE moved its monthly
expiry weekday in 2025-09).
match_scr_bse_masters_and_fo_update
Resolves underlying roots (SENSEX/BANKEX, typ=31, underly=0 — the same
"this masid is a root" convention as scr_master) first, creating them if
missing, then resolves each contract's masid.
New futures and options are inserted in two separate passes
A new contract's scr_bse_master row is inserted via two separate
insertdataframe calls — one for futures-typed rows (columns mname, underly, edate, typ, monthindex), one for options-typed rows (same
columns plus strike, opttyp) — never one combined batch. This
mirrors how NSE's match_scr_masters_and_fo_update is called once per
Opt=True/False.
Combining them was tried first and crashed: a futures row given the
options-only opttyp column gets NaN there (pandas aligns the missing
column across a pd.concat), which psycopg2 adapts as the 3-character
string 'NaN', and Postgres then rejects it against the CHAR(2) column —
value too long for type character(2). Confirmed live 2026-10-11.
Day replacement
Same "delete the whole date, then insert" rule as scr_nsefo_eod (root
CLAUDE.md rule 6) — scr_bsefo_eod has no enforced primary key.
BseEodFoUpdater
Daily sync driver — mirrors nseeod.FoBhav.NseEodFoUpdater, scoped to
SENSEX/BANKEX. update_a_day(mdate) returning True with zero rows
written is expected and not a failure (no trades that day, or a date
before BANKEX existed) — only a genuine parse/write error counts against
bse_backfill.py's fo_failed counter.
Related
- bse_equity.py — the equity ingestion this rides
alongside in
BseProcessor - Overview —
scr_bsefo_eod