JKTV-online/database/README.md

Database Governance

The repository intentionally keeps SQLite for the current private-team scale. This directory separates four different responsibilities:

Layer	Database	Grain	Owner
L1	L1/L1.db	One raw network capture per match	Import pipeline
L2	L2/L2.db	Normalized match, player, round and event facts	L2 Builder
L3	L3/L3.db	Roster features and profile marts	L3 Builder
Web	Web/Web_App.sqlite	Lineups, comments, jobs and editorial data	Flask app

Rules

1. Paths are defined only in database/paths.py.
2. Schemas live next to their owning database.
3. Builders may read the previous layer and write only their own layer.
4. User-generated Web data is never restored as part of an ETL rollback.
5. A full import must run through database/pipeline.py.
6. Pipeline runs are serialized by database/.pipeline.lock.
7. L1/L2/L3 are backed up before a full pipeline run.
8. Missing metrics are stored as NULL, not fabricated zero values.
9. Admin -> Data Integrity is the operational source of truth.
10. Web schema changes increment Config.WEB_SCHEMA_VERSION.

Entry Points

make l1          # Import output_arena JSON into L1
make l2          # Rebuild normalized facts
make l3          # Rebuild active-roster features
make pipeline    # Run L1 -> L2 -> L3 with backup and validation
make check       # Compile and run tests

Directory Policy

• L1/, L2/, L3/, Web/: active code, schema and database.
• backups/: generated rollback snapshots; ignored by Git.
• schema_bkp/: historical schema research only; not used at runtime.
• L1B/: reserved demo-parser integration; not used at runtime.
• L3/Roadmap/: historical design notes; not used at runtime.

Large-scale directory moves are deliberately deferred until the legacy builders no longer depend on their current module layout.