diff --git a/docs/project.md b/docs/project.md index eec1f71..815edb6 100644 --- a/docs/project.md +++ b/docs/project.md @@ -6,6 +6,8 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to a configurable 1-10 zone workflow with numeric region labels and editable trash ROI calibration. All v1.1 items are part of one batch; backend, API, frontend, and documentation are implementation workstreams inside that same batch. +The `v1.2 轨迹识别` batch adds source-zone trajectory evidence for disposal confirmation. The first implementation uses lightweight motion tracking and keeps YOLO disabled, while preserving an evidence contract that a later trained YOLO product detector can enrich. + ## Architecture - Backend package: `src/cold_display_guard/` @@ -15,6 +17,7 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to - `manage_api.py`: standard-library HTTP management API. - `main.py`: RTSP runtime loop connecting frame capture, vision, state engine, and JSONL sinks. - `vision.py`: heuristic ROI occupancy and trash-motion detection. + - v1.2 adds trajectory evidence between vision and engine: `MotionTrajectoryBackend` emits source-zone-to-trash evidence; `BatchEngine` consumes the backend-neutral `disposal_evidence` contract. - Frontend package: `web/` - Vite + vanilla JavaScript management console. - Default web port `23000`. @@ -45,6 +48,13 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to - Trash ROI: - Stored under `[trash] roi`. - Does not use a food zone number. +- v1.2 trajectory settings: + - `trajectory_enabled`: enables source-zone trajectory evidence. + - `trajectory_window_seconds`: seconds after a zone clears where movement can confirm disposal. + - `trajectory_sample_interval_seconds`: faster runtime delay while a candidate is active. + - `trajectory_min_confidence`: minimum confidence before evidence can close pending disposal. + - `trajectory_backend`: first valid value is `"motion"`. + - `yolo_enabled`: defaults to `false`; reserved for a future trained model backend. ## Event Model @@ -59,6 +69,8 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to Events should include `zone_id`, `zone_index`, `zone_label`, `started_at`, `dwell_seconds`, and relevant alarm/removal/deadline timestamps when available. +In v1.2, `batch_discarded` can be triggered by zone-scoped `disposal_evidence` before falling back to generic `trash_deposit_count`. Evidence must match the pending batch's `source_zone_id`. + ## Runbook - Python tests: @@ -79,6 +91,7 @@ Events should include `zone_id`, `zone_index`, `zone_label`, `started_at`, `dwel ## Known Risks - The current vision detector is heuristic and reports binary occupancy, not item counts. +- v1.2 motion tracking improves disposal matching but can still miss movement if the hand/object path is occluded or sampled too sparsely. - If food is already present during baseline collection, those regions may be treated as empty baseline until visual changes occur. - Changing calibration while the runtime process has active batches can create operational ambiguity; v1.1 should document or enforce a pause/restart expectation. - Historical events must keep the zone index at the time of emission so later region reordering does not reinterpret old logs. diff --git a/docs/superpowers/plans/2026-05-29-v1.2-trajectory-recognition.md b/docs/superpowers/plans/2026-05-29-v1.2-trajectory-recognition.md new file mode 100644 index 0000000..43e0d84 --- /dev/null +++ b/docs/superpowers/plans/2026-05-29-v1.2-trajectory-recognition.md @@ -0,0 +1,169 @@ +# v1.2 Trajectory Recognition Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add source-zone trajectory evidence so alarmed items moved to the trash are discarded by their actual source zone, while keeping YOLO as a future optional backend. + +**Architecture:** Extend `Observation` with backend-neutral `disposal_evidence`, make `BatchEngine` consume matching evidence before generic trash fallback, then add a no-dependency motion trajectory tracker in the vision layer. Runtime writes diagnostics and uses faster sampling only while trajectory candidates are active. + +**Tech Stack:** Python 3.11+ standard library, existing `Frame` RGB bytes, `unittest`, Vite/Node tests only if frontend files change. + +--- + +### Task 1: Data Contract And Engine Evidence Handling + +**Files:** +- Modify: `src/cold_display_guard/models.py` +- Modify: `src/cold_display_guard/engine.py` +- Test: `tests/test_engine.py` + +- [ ] **Step 1: Write failing tests** + +Add tests for: +- `Observation.from_dict()` normalizes `disposal_evidence`. +- Matching evidence discards the pending batch for the same source zone. +- Evidence for zone 4 does not discard pending zone 1. +- Same-observation removal plus evidence closes the newly pending batch. +- Low-confidence evidence is ignored. + +- [ ] **Step 2: Run RED tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_engine -v` + +Expected: FAIL because `Observation` has no `disposal_evidence` and engine ignores evidence. + +- [ ] **Step 3: Implement minimal contract and engine logic** + +Add a `DisposalEvidence` dataclass and `Observation.disposal_evidence`. In `BatchEngine.process()`, apply evidence to matching `pending_disposal` before generic trash deposits and again after zone transitions for same-frame removals. + +- [ ] **Step 4: Run GREEN tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_engine -v` + +Expected: PASS. + +- [ ] **Step 5: Commit phase** + +Run: + +```bash +git add src/cold_display_guard/models.py src/cold_display_guard/engine.py tests/test_engine.py +git commit -m "feat: add disposal evidence engine handling" +``` + +### Task 2: Lightweight Motion Trajectory Backend + +**Files:** +- Modify: `src/cold_display_guard/vision.py` +- Test: `tests/test_vision.py` + +- [ ] **Step 1: Write failing tests** + +Add synthetic RGB-frame tests for: +- Motion from source zone to trash ROI emits evidence. +- Motion that starts away from source zone is rejected. +- Motion that never reaches trash ROI is rejected. +- One-frame reflection flash is rejected. +- Multiple active candidates do not cross-close each other. + +- [ ] **Step 2: Run RED tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_vision -v` + +Expected: FAIL because no trajectory tracker exists. + +- [ ] **Step 3: Implement minimal motion tracker** + +Add trajectory settings, candidate state, motion blob extraction from frame deltas, confidence scoring, and diagnostics. Keep the implementation standard-library only. + +- [ ] **Step 4: Run GREEN tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_vision -v` + +Expected: PASS. + +- [ ] **Step 5: Commit phase** + +Run: + +```bash +git add src/cold_display_guard/vision.py tests/test_vision.py +git commit -m "feat: add lightweight trajectory tracking" +``` + +### Task 3: Runtime Configuration And Diagnostics Integration + +**Files:** +- Modify: `src/cold_display_guard/main.py` +- Modify: `src/cold_display_guard/vision.py` +- Modify: `config/example.toml` +- Test: `tests/test_vision.py` +- Test: `tests/test_main.py` + +- [ ] **Step 1: Write failing tests** + +Add tests that verify runtime defaults include trajectory settings with YOLO disabled and diagnostics rows include emitted evidence when present. + +- [ ] **Step 2: Run RED tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_vision tests.test_main -v` + +Expected: FAIL because runtime does not pass evidence into `Observation` or expose trajectory sampling state. + +- [ ] **Step 3: Implement runtime integration** + +Return `disposal_evidence` from vision observation, write it to diagnostics, pass it to `Observation`, and use `trajectory_sample_interval_seconds` while candidates are active. + +- [ ] **Step 4: Run GREEN tests** + +Run: `PYTHONPATH=src python3 -m unittest tests.test_vision tests.test_main -v` + +Expected: PASS. + +- [ ] **Step 5: Commit phase** + +Run: + +```bash +git add src/cold_display_guard/main.py src/cold_display_guard/vision.py config/example.toml tests/test_vision.py tests/test_main.py +git commit -m "feat: integrate trajectory runtime diagnostics" +``` + +### Task 4: Documentation, Full Verification, And Deployment Prep + +**Files:** +- Modify: `README_zh.md` +- Modify: `docs/project.md` +- Modify: `task_plan.md` +- Modify: `findings.md` +- Modify: `progress.md` +- Modify: `memories.md` + +- [ ] **Step 1: Update docs** + +Document v1.2 trajectory settings, evidence semantics, tests, and remote deployment notes. + +- [ ] **Step 2: Run full verification** + +Run: + +```bash +PYTHONPATH=src python3 -m unittest discover -s tests -v +node --test web/test/zone-state.test.js +cd web && pnpm build +``` + +Expected: PASS for all commands. If frontend files did not change, frontend commands still provide regression coverage for the management console. + +- [ ] **Step 3: Commit phase** + +Run: + +```bash +git add README_zh.md docs/project.md task_plan.md findings.md progress.md memories.md docs/superpowers/plans/2026-05-29-v1.2-trajectory-recognition.md +git commit -m "docs: document v1.2 trajectory recognition" +``` + +- [ ] **Step 4: Prepare remote deploy** + +Use rsync excluding `config/example.toml`, rebuild runtime/API, and verify Docker services. Record the exact commands and results in `progress.md`. diff --git a/findings.md b/findings.md index 6901ad9..a76a45f 100644 --- a/findings.md +++ b/findings.md @@ -52,6 +52,31 @@ Use `阶段 x` only as a workflow stage inside the same `v1.1 优化改造` batch. +## v1.2 轨迹识别 Findings + +## Architecture + +- `main.py` 当前每轮从 RTSP 截一帧，调用 `ZoneOccupancyDetector.observe()` 得到 `zone_counts`、`trash_deposit_count`、`diagnostics`，再构造 `Observation` 给 `BatchEngine.process()`。 +- `vision.py` 当前只有区域占用和垃圾桶动作计数，没有从“来源区域到垃圾桶”的轨迹证据。 +- `engine.py` 当前先处理 pending 过期和旧 trash deposit，再处理区域转移，最后对同帧剩余 trash deposit 做兜底；这为同帧 evidence 处理提供了插入点。 +- `models.py` 的 `Observation` 是最适合扩展统一 evidence contract 的位置。 +- v1.2 设计规格已保存并提交：`docs/superpowers/specs/2026-05-29-lightweight-trajectory-yolo-ready-design.md`，commit `ac6d368`。 + +## Constraints + +- 第一版必须在无 YOLO 依赖下运行。 +- 轨迹检测只应在区域变空后的短窗口活跃，避免持续 CPU 压力。 +- 垃圾桶内部不可见；确认点是进入垃圾桶口 ROI，不是看到桶内物品。 +- 不能让一个来源区域的 evidence 关闭另一个区域的 pending batch。 +- 远端部署必须继续排除 `config/example.toml`，以保留 RTSP 和现场标定。 + +## Decisions + +- 新字段命名为 `disposal_evidence`，元素使用 `source_zone_id`、`target`、`confidence`、`method`、`track_points`、可选 `item_class`、`detector_score`。 +- `trash_deposit_count` 保留兼容和兜底，但 engine 先使用 zone-scoped evidence。 +- 轨迹诊断必须记录 active、emitted、rejected、expired，方便现场调参。 +- 后续 YOLO 模型只作为 evidence 增强输入，不能修改 `BatchEngine` 的业务语义。 + ## Backend Planning Notes - `EngineSettings.zone_ids` should remain config driven; numeric zones are preferred for new configs, but old `r1c1` style IDs should continue loading. diff --git a/memories.md b/memories.md new file mode 100644 index 0000000..12152a7 --- /dev/null +++ b/memories.md @@ -0,0 +1,24 @@ +# Memories + +## User and Workflow + +- User wants Chinese responses. +- Project path for subagent headers is `/Users/yoilun/Code/cold_display_guard`. +- Current workflow batch is `v1.2 轨迹识别`. +- User requested following `/Users/yoilun/Code/goal-subagents-workflow-prompt.md`. +- Every subagent task must begin with: + +```text +[项目: /Users/yoilun/Code/cold_display_guard] +[工作流批次: v1.2 轨迹识别] +[阶段: 阶段 x] +[角色: 对应智能体角色] +``` + +## Technical Direction + +- First implement lightweight motion trajectory detection without YOLO dependencies. +- Preserve a stable `disposal_evidence` contract for a future trained YOLO product detector. +- Keep ROI occupancy timing as the source of zone occupied/empty state. +- Use trajectory evidence before generic trash-motion FIFO fallback. +- Remote deployment target is `xiaozheng@192.168.5.206:/home/xiaozheng/cold_display_guard`; preserve remote `config/example.toml`. diff --git a/progress.md b/progress.md index e32793c..8a2620f 100644 --- a/progress.md +++ b/progress.md @@ -183,3 +183,23 @@ | Runtime restart can drop threshold-edge timers | Runtime restore used raw threshold recompute instead of stable `occupied`, so a restart during a one-frame raw dip could lose an active timer | Restore now uses stable `occupied` when present and keeps raw recompute only as a fallback | Resolved; regression test covers zone 2-style flicker | | Zone 1 timer reset | Zone 1's ROI had too few samples with the default stride `8`; dark evidence jumped between `0.0714` and `0.0357/0`, causing two occupied frames followed by two empty frames and repeated short batches | Reduced default and remote `sample_stride_pixels` to `4` so small ROI/object evidence is less quantized | Resolved in current verification; zone 1 remains continuously occupied after deploy | | Zone 1/4 disposal missed after zone 2 discard | Zone 2 generated a trash deposit at `14:32:13`; zone 1/4 cleared at `14:32:19`, but their strong trash motion was inside the old 8-second cooldown, and one same-frame deposit could only discard one newly pending batch | Reduced trash cooldown to 3 seconds and let one same-frame trash motion discard all newly pending alerted batches from that observation | Resolved for future events; historical zone 1/4 rows are not rewritten | + +## 2026-05-29 v1.2 轨迹识别 + +### Session Log + +| Time | Phase | Actor | Action | Result | +| --- | --- | --- | --- | --- | +| 2026-05-29 | Setup | Main Agent | Created Goal for `v1.2 轨迹识别` using `/Users/yoilun/Code/cold_display_guard` as the real project path | Active goal tracks lightweight trajectory detection plus YOLO-ready evidence contract | +| 2026-05-29 | Setup | Main Agent | Read `/Users/yoilun/Code/goal-subagents-workflow-prompt.md` | Workflow requires task files, stage-based coding/testing agents, bug loop limit, and standard subagent context header | +| 2026-05-29 | Setup | Main Agent | Read existing `task_plan.md`, `findings.md`, `progress.md`, and `docs/project.md` | v1.1 state is complete; v1.2 plan can start on branch `lightweight-trajectory-tracking` | + +### Test Results + +| Time | Command | Result | Notes | +| --- | --- | --- | --- | + +### Bug Loop + +| Phase | Bug | Fix Attempt | Retest Result | +| --- | --- | --- | --- | diff --git a/task_plan.md b/task_plan.md index d2d9d91..a63d448 100644 --- a/task_plan.md +++ b/task_plan.md @@ -76,3 +76,42 @@ Create and evolve an independent git project under `~/Code` for monitoring food ``` 其中 `阶段 x` 表示同一 `v1.1 优化改造` 批次内的工作阶段，不代表拆分成独立批次。 + +## v1.2 轨迹识别 + +### Goal + +在 `/Users/yoilun/Code/cold_display_guard` 中完成轨迹识别改造：保留现有 ROI 占用计时和垃圾桶动作兜底，新增轻量轨迹移动检测，输出可被未来 YOLO 物品识别模型复用的统一 `disposal_evidence`，让报警后移出的物品按来源区域确认是否进入垃圾桶。 + +### Stop Conditions + +- [ ] v1.2 所有阶段完成。 +- [ ] 必要 Python 测试通过。 +- [ ] 前端测试或构建在受影响时通过。 +- [ ] `docs/project.md` 记录 v1.2 架构、配置、运行方式和关键决策。 +- [ ] 没有 blocking bug 或未处理的高风险问题。 +- [ ] 如果同一问题连续 3 次修复仍失败，暂停并报告原因、已尝试方案和建议下一步。 + +### Phases + +| Phase | Status | Goal | Acceptance Criteria | +| --- | --- | --- | --- | +| 1 | pending | 建立 `disposal_evidence` 数据契约并让状态机优先按来源区域丢弃 | `Observation` 支持 evidence；engine 能按 `source_zone_id` 精确关闭 pending batch；同帧移除+evidence 有回归测试；旧 `trash_deposit_count` 仍可兜底 | +| 2 | pending | 实现无 YOLO 依赖的轻量轨迹检测 | synthetic frame 测试覆盖源区域到垃圾桶、非源区域运动、未到垃圾桶、单帧反光、多候选互不串扰；不引入模型依赖 | +| 3 | pending | 集成 runtime 配置、诊断和候选窗口加速采样 | `main.py` 写入 `disposal_evidence` 与 trajectory diagnostics；配置默认 `trajectory_enabled=true`、`yolo_enabled=false`；候选活跃时使用更短采样间隔 | +| 4 | pending | 文档、全量验证和部署准备 | README/project/progress 更新；Python 全量测试通过；前端测试/构建按影响范围验证；远端部署命令和风险记录清楚 | + +### v1.2 Decisions + +- 第一版使用 `MotionTrajectoryBackend`，不安装 YOLO、PyTorch、ONNX Runtime 或 OpenVINO。 +- YOLO 作为后续 `YoloDetectionBackend` 接入统一 evidence contract，不能绕过轨迹校验直接关闭业务事件。 +- 状态机只消费 `disposal_evidence`，不依赖具体视觉后端。 +- 轨迹 evidence 优先级高于 FIFO 垃圾桶动作兜底。 +- 子 agent 派发必须使用标准上下文头： + +```text +[项目: /Users/yoilun/Code/cold_display_guard] +[工作流批次: v1.2 轨迹识别] +[阶段: 阶段 x] +[角色: 对应智能体角色] +```