JKTV-online/database/README.md

Database Governance

The repository intentionally keeps SQLite for the current private-team scale. This directory separates four different responsibilities:

Layer	Database	Grain	Owner
L1	L1/L1.db	One raw network capture per match	Import pipeline
L2	L2/L2.db	Normalized match, player, round and event facts	L2 Builder
L3	L3/L3.db	Roster features and profile marts	L3 Builder
Web	Web/Web_App.sqlite	Lineups, comments, jobs and editorial data	Flask app

Rules

1. Paths are defined only in database/paths.py.
2. Schemas live next to their owning database.
3. Builders may read the previous layer and write only their own layer.
4. User-generated Web data is never restored as part of an ETL rollback.
5. A full import must run through database/pipeline.py.
6. Pipeline runs are serialized by database/.pipeline.lock.
7. L1/L2/L3 are backed up before a full pipeline run.
8. Missing metrics are stored as NULL, not fabricated zero values.
9. Admin -> Data Integrity is the operational source of truth.
10. Web schema changes increment Config.WEB_SCHEMA_VERSION.
11. YRTV_DATA_DIR owns runtime databases; code schemas always stay in Git.
12. Container and production deployments must keep runtime data outside the image and source checkout.

Entry Points

make l1          # Import output_arena JSON into L1
make l2          # Rebuild normalized facts
make l3          # Rebuild active-roster features
make pipeline    # Run L1 -> L2 -> L3 with backup and validation
make check       # Compile and run tests
make bootstrap   # Initialize an empty runtime data root
make prepare-data # Copy legacy data into runtime-data

Directory Policy

• Without YRTV_DATA_DIR, L1/, L2/, L3/, Web/ remain the compatible development runtime.
• With YRTV_DATA_DIR, those folders provide code and schemas while databases live under the configured external directory.
• backups/: generated rollback snapshots; ignored by Git.
• schema_bkp/: historical schema research only; not used at runtime.
• L1B/: reserved demo-parser integration; not used at runtime.
• L3/Roadmap/: historical design notes; not used at runtime.

Large-scale directory moves are deliberately deferred until the legacy builders no longer depend on their current module layout.