NSEEOD Package — Overview
backend/nseeod/ is a custom-built Python package that handles the full lifecycle
of NSE end-of-day data: downloading bhav copies from NSE archives, parsing them,
merging with the security master, and persisting to PostgreSQL.
Package File Map
| File | Role |
|---|---|
downloadbhav.py | Orchestrator — BhavProcessor class + standalone utilities |
EquityBhav.py | Equity, Index, and Delivery Volume updater classes |
FoBhav.py | Futures & Options updater class |
NSEDownloader.py | HTTP downloader for new-format NSE archives |
utils.py | Shared helpers — date utils, NSE file downloads, DB write helpers |
pgs.py | PostgreSQL connection and CRUD helpers (psycopg2 + SQLAlchemy) |
bhav.py | AmiBroker export file generator |
check_bhav.py | Debug utility — compares F&O DB vs raw file |
ut_Classes.py | Legacy / research — holiday correction, intraday fill |
Indexinfo.csv | Static NSE index name mapping (Old name → New name → typ) |
High-Level Architecture
Data Flow — Full Sync
Download Modes
| Mode | Method | Trigger |
|---|---|---|
| Full sync | run_update() | POST /api/bhav-downloader/download |
| Single date | run_update_for_date(date) | POST /api/bhav-downloader/download?date=YYYY-MM-DD |
| F&O catch-up | download_missingfoBhav() | Manual / CLI only |
NSE File Format History
NSE changed the bhav copy format in April 2025. The code supports both:
| Era | Format flag | Equity filename | F&O filename |
|---|---|---|---|
| Old (pre-2025) | newformat = False | cmDDMMMYYYYbhav.csv | foDDMMMYYYYbhav.csv |
| New (2025+) | newformat = True | BhavCopy_NSE_CM_0_0_0_YYYYMMDD_F_0000.csv | BhavCopy_NSE_FO_0_0_0_YYYYMMDD_F_0000.csv |
BhavProcessor.newformat = True by default (new format active).
Key Database Tables
| Table | Description | Primary Key |
|---|---|---|
scr_master | Security master — all symbols (EQ, FUT, OPT, Index) | masid |
scr_nseeq_eod | Equity EOD OHLCV + delivery | (masid, sdate) |
scr_nsefo_eod | F&O EOD OHLCV + OI | (masid, sdate) |
scr_indexpepb | Index PE / PB / Div Yield | (masid, sdate) |
scr_missing_bhav | Failed download tracker | (sdate, typ) |
scr_missing_eod | Missing EOD tracker | (sdate, typ) |
holidays | NSE holiday calendar | sdate |
specialdays | Muhurat trading sessions | sdate |
tasks | Retry task queue (JSONB) | typ |
scr_config | System configuration store | mname |
Running Standalone
# Always run as a module from backend/ directory
cd backend
python -m nseeod.downloadbhav
Never run as a script directly
# WRONG — breaks relative imports
python nseeod/downloadbhav.py