Skip to main content

bse_store.py — Parquet History Store

Per-stock Parquet history store for BSE equity, mirroring nseeod/eq_store.py closely enough that a caller already familiar with that module's interface needs no relearning — same split (TABLE staging in Postgres, RAW Parquet history, CATALOG rollup stats), same export_all/append_days/get_eod/get_day/available_days/verify/ trim_postgres surface.

Postgres scr_bseeq_eod : staging - bseeod's daily/backfill pipeline writes here
BSEEQOD_DATA_PATH/raw/ : history, one extensionless Parquet file per stock
BSEEQOD_DATA_PATH/_catalog/days.parquet : sdate, rows, sum_close, sum_volume
BSEEQOD_DATA_PATH/_catalog/symbols.parquet : masid, file, rows, first, last
cd backend
python -m bseeod.bse_store export # full rebuild from Postgres
python -m bseeod.bse_store append --days 5 # add the last N staging days
python -m bseeod.bse_store verify # Parquet vs Postgres, every shared date
python -m bseeod.bse_store stats
python -m bseeod.bse_store trim --keep 90 # dry run; --apply deletes verified old rows

symbol_path(mname, scrip_code=None)​

def symbol_path(mname: str, scrip_code=None) -> str:
name = mname.strip()
safe = _INVALID.sub("_", name)
if scrip_code:
safe = f"{safe}_{scrip_code}"
...

Always suffixed with scrip_code when given. BSE's ~12-char truncated SC_NAME is not a reliable unique identifier the way NSE's SYMBOL is — confirmed live 2026-10-11, two unrelated companies can truncate to the same name ("SHELTER" at scrip_code 526839 and an unrelated company at 543963 — see bse_equity.py for the full finding). Without the scrip_code suffix, two different companies' Parquet histories would collide on the same file path even after the DB-side matching fix moved them to separate masids — the file name collision is a distinct problem from the DB matching collision, and both needed fixing.

eq_store.symbol_path()'s mixed-case disambiguation (hashing the exact spelling for a name that isn't all-upper-case) is kept as a fallback only for the case scrip_code is unavailable — shouldn't happen for a real equity row, but keeps the function usable as a general-purpose helper.


Write Side​

export_all() and append_days() both resolve (mname, scrip_code) per masid from scr_bse_master (via _masters(), now selecting scrip_code alongside mname) before building each file's path — a change from eq_store.py's single-column lookup, required once symbol_path started taking two arguments.

export_all()'s pre-pass collision check (paths.setdefault(...), raising RuntimeError on a genuine path collision) still exists as a safety net, but should no longer fire in practice now that every path is scrip_code-suffixed and scrip_code is itself UNIQUE at the DB level.


Read Side​

_master_row(key)​

key may be a masid, a scrip_code, or an mname (in that preference order when ambiguous — a bare numeric string could be either a masid or a scrip_code, and masid, being the actual primary key, is tried first, falling back to a scrip_code match if no masid matches). A text mname lookup returns the first match by masid order — since mname is no longer guaranteed unique, this is a convenience only; pass the scrip_code or masid instead when precision matters (e.g. two stocks both named something that truncates to "SHELTER").

get_eod(symbol, frm, to, adjust='none')​

if adjust != "none":
raise NotImplementedError(
"bse_store.get_eod(adjust=...) is not implemented yet - "
"BSE has no corporate-action table (scr_bse_corp_action) to adjust from."
)

Only adjust='none' (raw) is implemented. This is a deliberate choice, not an oversight: eq_store.get_eod(adjust=...) exists for API parity (a caller already familiar with NSE's interface needs no relearning once 'price'/'total' are wired up), but passing anything else today raises loudly rather than silently returning unadjusted data under an adjust flag that implies otherwise. Wiring this up requires reconcile() to exist first — unreconciled AUTO factors from bse_corp_actions.py are usable for SPLIT/BONUS/DIVIDEND but not yet cross-checked against real price gaps the way NSE's factors are.

No follow_renames parameter either, for the same reason — it needs the corporate-actions symbol-change graph (corp_actions.predecessors()'s BSE equivalent), which doesn't exist yet.


What Wasn't Ported From eq_store.py​

price_windows, price_moves, get_returns_panel, rename_families — NSE-specific research tooling built for corp_actions.reconcile() and the ci_* leverage/F&O-effect studies. BSE has neither yet, and this module only covers what bseeod actually uses — adding them speculatively would be unused abstraction (root CLAUDE.md's "don't add features beyond what the task requires").