2.0.0 Alpha: Data Refinery

This commit is contained in:
2026-08-08 21:31:56 +08:00
parent fa75081d4d
commit 562775e5db
48 changed files with 4172 additions and 661 deletions
+19 -10
View File
@@ -1,16 +1,25 @@
L1A 5eplay平台网页爬虫原始数据。
# L1 Raw Match Store
## ETL Step 1:
从原始json数据库提取到L1A级数据库中。
`output_arena/*/iframe_network.json` -> `database/L1A/L1A.sqlite`
L1 stores one complete 5E network capture per match without transforming its
payload.
### 脚本说明
- **脚本位置**: `ETL/L1A.py`
- **功能**: 自动遍历 `output_arena` 目录下所有的 `iframe_network.json` 文件,提取原始内容并以 `match_id` (文件夹名) 为主键存入 `L1A.sqlite` 数据库的 `raw_iframe_network` 表中。
## Runtime Files
### 运行方式
使用项目指定的 Python 环境运行脚本:
- Database: `database/L1/L1.db`
- Builder: `database/L1/L1_Builder.py`
- Input: `output_arena/<match_id>/iframe_network.json`
- Primary key: `raw_iframe_network.match_id`
## Commands
```bash
C:/ProgramData/anaconda3/python.exe ETL/L1A.py
make l1
make pipeline
```
Normal ingestion is incremental. `--force` re-reads every capture currently
present in `output_arena`.
`L1A.db` and the historical `database/L1A/L1A.sqlite` path are retired. L1B is
reserved for a future demo-parser source and is not part of the runtime
pipeline.