bse_index.py — SENSEX/BANKEX Index Spot
Writes SENSEX/BANKEX index OHLC rows into scr_bseeq_eod, against the same
masid convention as scr_bse_master's indices (typ=TYP_IDX, underly=0)
— the same masids bse_derivatives.py already
creates as futures/options roots. Mirrors
nseeod.EquityBhav.NseEodIndexUpdater.
Source
https://www.bseindia.com/bsedata/Index_Bhavcopy/INDEXSummary_{DDMMYYYY}.csv
Columns: IndexCode, IndexID, IndexName, PreviousClose, OpenPrice, HighPrice, LowPrice, ClosePrice, 52weeksHigh, 52weeksLow, Filler1-4. No
date column in the file itself — sdate comes from the filename. No
volume (indices don't have one, same as NSE's index EOD rows).
IndexID is the short code (SENSEX, BANKEX); IndexCode 1=SENSEX,
12=BANKEX.
The 2025-04-15 Availability Cutoff
This file is only available from 2025-04-15 onward. Every date before 2025-04-14 returns HTTP 404; 2025-04-15 onward returns 200 cleanly. This is a separate, later cutover than the equity/derivatives UDiFF switch (2024-07-08) — not the same discontinuation. BSE's site does not expose an older-format fallback for this specific file the way equity/derivatives have an old-format fallback.
Confirmed by binary-searching HTTP 200/404 responses across 2024 and all of
early 2025, cross-checked at each candidate date against the actual weekday
and the seeded BSE holiday calendar (exch=3) to rule out false negatives
from weekends or holidays — several early test dates that looked like gaps
turned out to be Saturdays, Sundays, or Maharashtra Day, not real 404s. The
one genuinely isolated 404 found within the available range
(2025-04-01, a Tuesday with no holiday-table entry) was retried and
confirmed persistent — a real BSE-side publishing gap for that single date,
not a transient network error.
Practical effect: SENSEX/BANKEX index-spot OHLC history before
2025-04-15 is not obtainable from this source at all. The project's
2020-01-01 phase-1 start applies to equity and derivatives, but not to
index spot — INDEX_DATA_START = datetime.date(2025, 4, 15) is a separate,
later constant. update_a_day() treats any date before it as a known
no-op (returns True, logs nothing as an error), so it never shows up as
a failure in a backfill run — bse_backfill.py's
index_failed counter only increments on a genuine fetch/parse/write
error, not on the expected pre-2025-04-15 gap.
If deeper index history is ever needed, it would require a different data source entirely — not attempted here.
match_scr_bse_index_and_update
Resolves SENSEX/BANKEX to their scr_bse_master root masids (creating
them with typ=TYP_IDX, underly=0 if they don't exist yet — the same
roots bse_derivatives.py creates when it runs
first), then replaces that date's rows for those specific masids only in
scr_bseeq_eod — scoped by masid = ANY(...), never a blanket
DELETE WHERE sdate = ..., since that table also holds every equity
stock's row for the same date.
BseEodIndexUpdater
Best-effort by design — a failure here never blocks the equity/derivatives
sync it rides alongside, mirrored exactly in
BseProcessor: both run_update() and
run_update_for_date() wrap the Index.update()/update_a_day() call in
its own try/except, logging but swallowing any exception.
Related
- bse_derivatives.py — creates the same SENSEX/BANKEX root masids as futures/options underlyings
- Overview — Index-Summary File, a Hard Data Gap