docs: plan v1.2 trajectory recognition workflow

This commit is contained in:
Yoilun
2026-05-29 15:18:58 +08:00
parent ac6d368810
commit 5f518991bf
6 changed files with 290 additions and 0 deletions

View File

@@ -6,6 +6,8 @@
The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to a configurable 1-10 zone workflow with numeric region labels and editable trash ROI calibration. All v1.1 items are part of one batch; backend, API, frontend, and documentation are implementation workstreams inside that same batch.
The `v1.2 轨迹识别` batch adds source-zone trajectory evidence for disposal confirmation. The first implementation uses lightweight motion tracking and keeps YOLO disabled, while preserving an evidence contract that a later trained YOLO product detector can enrich.
## Architecture
- Backend package: `src/cold_display_guard/`
@@ -15,6 +17,7 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to
- `manage_api.py`: standard-library HTTP management API.
- `main.py`: RTSP runtime loop connecting frame capture, vision, state engine, and JSONL sinks.
- `vision.py`: heuristic ROI occupancy and trash-motion detection.
- v1.2 adds trajectory evidence between vision and engine: `MotionTrajectoryBackend` emits source-zone-to-trash evidence; `BatchEngine` consumes the backend-neutral `disposal_evidence` contract.
- Frontend package: `web/`
- Vite + vanilla JavaScript management console.
- Default web port `23000`.
@@ -45,6 +48,13 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to
- Trash ROI:
- Stored under `[trash] roi`.
- Does not use a food zone number.
- v1.2 trajectory settings:
- `trajectory_enabled`: enables source-zone trajectory evidence.
- `trajectory_window_seconds`: seconds after a zone clears where movement can confirm disposal.
- `trajectory_sample_interval_seconds`: faster runtime delay while a candidate is active.
- `trajectory_min_confidence`: minimum confidence before evidence can close pending disposal.
- `trajectory_backend`: first valid value is `"motion"`.
- `yolo_enabled`: defaults to `false`; reserved for a future trained model backend.
## Event Model
@@ -59,6 +69,8 @@ The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to
Events should include `zone_id`, `zone_index`, `zone_label`, `started_at`, `dwell_seconds`, and relevant alarm/removal/deadline timestamps when available.
In v1.2, `batch_discarded` can be triggered by zone-scoped `disposal_evidence` before falling back to generic `trash_deposit_count`. Evidence must match the pending batch's `source_zone_id`.
## Runbook
- Python tests:
@@ -79,6 +91,7 @@ Events should include `zone_id`, `zone_index`, `zone_label`, `started_at`, `dwel
## Known Risks
- The current vision detector is heuristic and reports binary occupancy, not item counts.
- v1.2 motion tracking improves disposal matching but can still miss movement if the hand/object path is occluded or sampled too sparsely.
- If food is already present during baseline collection, those regions may be treated as empty baseline until visual changes occur.
- Changing calibration while the runtime process has active batches can create operational ambiguity; v1.1 should document or enforce a pause/restart expectation.
- Historical events must keep the zone index at the time of emission so later region reordering does not reinterpret old logs.

View File

@@ -0,0 +1,169 @@
# v1.2 Trajectory Recognition Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add source-zone trajectory evidence so alarmed items moved to the trash are discarded by their actual source zone, while keeping YOLO as a future optional backend.
**Architecture:** Extend `Observation` with backend-neutral `disposal_evidence`, make `BatchEngine` consume matching evidence before generic trash fallback, then add a no-dependency motion trajectory tracker in the vision layer. Runtime writes diagnostics and uses faster sampling only while trajectory candidates are active.
**Tech Stack:** Python 3.11+ standard library, existing `Frame` RGB bytes, `unittest`, Vite/Node tests only if frontend files change.
---
### Task 1: Data Contract And Engine Evidence Handling
**Files:**
- Modify: `src/cold_display_guard/models.py`
- Modify: `src/cold_display_guard/engine.py`
- Test: `tests/test_engine.py`
- [ ] **Step 1: Write failing tests**
Add tests for:
- `Observation.from_dict()` normalizes `disposal_evidence`.
- Matching evidence discards the pending batch for the same source zone.
- Evidence for zone 4 does not discard pending zone 1.
- Same-observation removal plus evidence closes the newly pending batch.
- Low-confidence evidence is ignored.
- [ ] **Step 2: Run RED tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_engine -v`
Expected: FAIL because `Observation` has no `disposal_evidence` and engine ignores evidence.
- [ ] **Step 3: Implement minimal contract and engine logic**
Add a `DisposalEvidence` dataclass and `Observation.disposal_evidence`. In `BatchEngine.process()`, apply evidence to matching `pending_disposal` before generic trash deposits and again after zone transitions for same-frame removals.
- [ ] **Step 4: Run GREEN tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_engine -v`
Expected: PASS.
- [ ] **Step 5: Commit phase**
Run:
```bash
git add src/cold_display_guard/models.py src/cold_display_guard/engine.py tests/test_engine.py
git commit -m "feat: add disposal evidence engine handling"
```
### Task 2: Lightweight Motion Trajectory Backend
**Files:**
- Modify: `src/cold_display_guard/vision.py`
- Test: `tests/test_vision.py`
- [ ] **Step 1: Write failing tests**
Add synthetic RGB-frame tests for:
- Motion from source zone to trash ROI emits evidence.
- Motion that starts away from source zone is rejected.
- Motion that never reaches trash ROI is rejected.
- One-frame reflection flash is rejected.
- Multiple active candidates do not cross-close each other.
- [ ] **Step 2: Run RED tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_vision -v`
Expected: FAIL because no trajectory tracker exists.
- [ ] **Step 3: Implement minimal motion tracker**
Add trajectory settings, candidate state, motion blob extraction from frame deltas, confidence scoring, and diagnostics. Keep the implementation standard-library only.
- [ ] **Step 4: Run GREEN tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_vision -v`
Expected: PASS.
- [ ] **Step 5: Commit phase**
Run:
```bash
git add src/cold_display_guard/vision.py tests/test_vision.py
git commit -m "feat: add lightweight trajectory tracking"
```
### Task 3: Runtime Configuration And Diagnostics Integration
**Files:**
- Modify: `src/cold_display_guard/main.py`
- Modify: `src/cold_display_guard/vision.py`
- Modify: `config/example.toml`
- Test: `tests/test_vision.py`
- Test: `tests/test_main.py`
- [ ] **Step 1: Write failing tests**
Add tests that verify runtime defaults include trajectory settings with YOLO disabled and diagnostics rows include emitted evidence when present.
- [ ] **Step 2: Run RED tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_vision tests.test_main -v`
Expected: FAIL because runtime does not pass evidence into `Observation` or expose trajectory sampling state.
- [ ] **Step 3: Implement runtime integration**
Return `disposal_evidence` from vision observation, write it to diagnostics, pass it to `Observation`, and use `trajectory_sample_interval_seconds` while candidates are active.
- [ ] **Step 4: Run GREEN tests**
Run: `PYTHONPATH=src python3 -m unittest tests.test_vision tests.test_main -v`
Expected: PASS.
- [ ] **Step 5: Commit phase**
Run:
```bash
git add src/cold_display_guard/main.py src/cold_display_guard/vision.py config/example.toml tests/test_vision.py tests/test_main.py
git commit -m "feat: integrate trajectory runtime diagnostics"
```
### Task 4: Documentation, Full Verification, And Deployment Prep
**Files:**
- Modify: `README_zh.md`
- Modify: `docs/project.md`
- Modify: `task_plan.md`
- Modify: `findings.md`
- Modify: `progress.md`
- Modify: `memories.md`
- [ ] **Step 1: Update docs**
Document v1.2 trajectory settings, evidence semantics, tests, and remote deployment notes.
- [ ] **Step 2: Run full verification**
Run:
```bash
PYTHONPATH=src python3 -m unittest discover -s tests -v
node --test web/test/zone-state.test.js
cd web && pnpm build
```
Expected: PASS for all commands. If frontend files did not change, frontend commands still provide regression coverage for the management console.
- [ ] **Step 3: Commit phase**
Run:
```bash
git add README_zh.md docs/project.md task_plan.md findings.md progress.md memories.md docs/superpowers/plans/2026-05-29-v1.2-trajectory-recognition.md
git commit -m "docs: document v1.2 trajectory recognition"
```
- [ ] **Step 4: Prepare remote deploy**
Use rsync excluding `config/example.toml`, rebuild runtime/API, and verify Docker services. Record the exact commands and results in `progress.md`.