2.0.0 Alpha: Data Refinery

database/README.md

@@ -0,0 +1,45 @@
+# Database Governance
+
+The repository intentionally keeps SQLite for the current private-team scale.
+This directory separates four different responsibilities:
+
+| Layer | Database | Grain | Owner |
+|---|---|---|---|
+| L1 | `L1/L1.db` | One raw network capture per match | Import pipeline |
+| L2 | `L2/L2.db` | Normalized match, player, round and event facts | L2 Builder |
+| L3 | `L3/L3.db` | Roster features and profile marts | L3 Builder |
+| Web | `Web/Web_App.sqlite` | Lineups, comments, jobs and editorial data | Flask app |
+
+## Rules
+
+1. Paths are defined only in `database/paths.py`.
+2. Schemas live next to their owning database.
+3. Builders may read the previous layer and write only their own layer.
+4. User-generated Web data is never restored as part of an ETL rollback.
+5. A full import must run through `database/pipeline.py`.
+6. Pipeline runs are serialized by `database/.pipeline.lock`.
+7. L1/L2/L3 are backed up before a full pipeline run.
+8. Missing metrics are stored as `NULL`, not fabricated zero values.
+9. `Admin -> Data Integrity` is the operational source of truth.
+10. Web schema changes increment `Config.WEB_SCHEMA_VERSION`.
+
+## Entry Points
+
+```bash
+make l1          # Import output_arena JSON into L1
+make l2          # Rebuild normalized facts
+make l3          # Rebuild active-roster features
+make pipeline    # Run L1 -> L2 -> L3 with backup and validation
+make check       # Compile and run tests
+```
+
+## Directory Policy
+
+- `L1/`, `L2/`, `L3/`, `Web/`: active code, schema and database.
+- `backups/`: generated rollback snapshots; ignored by Git.
+- `schema_bkp/`: historical schema research only; not used at runtime.
+- `L1B/`: reserved demo-parser integration; not used at runtime.
+- `L3/Roadmap/`: historical design notes; not used at runtime.
+
+Large-scale directory moves are deliberately deferred until the legacy
+builders no longer depend on their current module layout.