135 lines
9.5 KiB
Markdown
135 lines
9.5 KiB
Markdown
# Cold Display Guard Project Documentation
|
|
|
|
## Goal
|
|
|
|
`cold-display-guard` monitors refrigerated display food batches by camera region. It tracks how long each configured food region remains occupied, raises a configurable time alarm, and escalates alarmed food to a warning if it is removed without a matching trash-bin deposit.
|
|
|
|
The `v1.1 优化改造` batch upgrades the product from a fixed 8-zone layout to a configurable 1-10 zone workflow with numeric region labels and editable trash ROI calibration. All v1.1 items are part of one batch; backend, API, frontend, and documentation are implementation workstreams inside that same batch.
|
|
|
|
The `v1.2 轨迹识别` batch adds source-zone trajectory evidence for disposal confirmation. The first implementation uses lightweight motion tracking and keeps YOLO disabled, while preserving an evidence contract that a later trained YOLO product detector can enrich.
|
|
|
|
## Architecture
|
|
|
|
- Backend package: `src/cold_display_guard/`
|
|
- `models.py`: settings, observations, and batch dataclasses.
|
|
- `engine.py`: deterministic batch state machine.
|
|
- `config.py`: TOML config load/save, calibration merge, and project path resolution.
|
|
- `manage_api.py`: standard-library HTTP management API.
|
|
- `main.py`: RTSP runtime loop connecting frame capture, vision, state engine, and JSONL sinks.
|
|
- `vision.py`: heuristic ROI occupancy and trash-motion detection.
|
|
- v1.2 adds trajectory evidence between vision and engine: `TrajectoryTracker` emits source-zone-to-trash evidence; `BatchEngine` consumes the backend-neutral `disposal_evidence` contract.
|
|
- Frontend package: `web/`
|
|
- Vite + vanilla JavaScript management console.
|
|
- Default web port `23000`.
|
|
- API proxy target `http://127.0.0.1:19080`.
|
|
- Runtime home view falls back to clearly marked demo data when no real events exist, so the operational layout still shows summary cards, dwell timers, and event rows.
|
|
- Runtime data:
|
|
- Events JSONL default path `logs/events.jsonl`.
|
|
- Diagnostics JSONL default path `logs/runtime_diagnostics.jsonl`.
|
|
- v1.2 diagnostics include root-level `disposal_evidence` plus `diagnostics.trajectory`.
|
|
- Deployment:
|
|
- Root Python Docker image for API/runtime.
|
|
- `web/Dockerfile` for static web console.
|
|
- `deploy/docker-compose.yml` wires API, runtime, and web services.
|
|
|
|
## Configuration
|
|
|
|
- Main config path: `config/example.toml`.
|
|
- Camera identity: `camera_id`.
|
|
- Timezone default: `Asia/Shanghai`.
|
|
- RTSP input: `[stream] rtsp_url`.
|
|
- Thresholds:
|
|
- `max_dwell_seconds`: v1.1 time-alarm threshold. Default can remain 10800 seconds; users can set values such as 1200 seconds for 20 minutes.
|
|
- `trash_confirmation_seconds`: window after an alarmed batch is removed where a trash deposit must be observed before warning escalation.
|
|
- Food zones:
|
|
- v1.1 food zone IDs are numeric strings from `"1"` through `"10"`.
|
|
- The configured zone count must be between 1 and 10.
|
|
- If both `zone_count` and numeric `zone_ids` are present, they must agree.
|
|
- Each `[[zones]]` polygon must have at least 3 normalized points.
|
|
- Trash ROI:
|
|
- Stored under `[trash] roi`.
|
|
- Does not use a food zone number.
|
|
- v1.2 trajectory settings:
|
|
- `lighting_shift_guard_enabled`: freezes occupancy changes when many regions shift brightness in the same direction.
|
|
- `lighting_shift_min_regions`, `lighting_shift_region_fraction`, `lighting_shift_mean_delta`: tune the global lighting/exposure guard.
|
|
- `trajectory_enabled`: enables source-zone trajectory evidence.
|
|
- `trajectory_window_seconds`: seconds after a zone clears where movement can confirm disposal.
|
|
- `trajectory_sample_interval_seconds`: faster runtime delay while a candidate is active.
|
|
- `trajectory_min_points`: minimum sampled motion points required before evidence can emit.
|
|
- `trajectory_segmented_enabled`, `trajectory_segmented_min_points`: allow a source point plus trash-entry point to confirm disposal when the middle of the path is occluded.
|
|
- `trajectory_min_confidence`: minimum confidence before evidence can close pending disposal.
|
|
- `trajectory_motion_delta`: frame-difference threshold for trajectory motion points.
|
|
- `trajectory_min_blob_area`: minimum connected motion area to keep as a point.
|
|
- `trajectory_max_blob_area_fraction`: rejects overly broad frame motion as ambiguous.
|
|
- `trajectory_trash_entry_margin`: margin for treating a track point as entering the trash ROI.
|
|
- `trajectory_backend`: first valid value is `"motion"`.
|
|
- `yolo_enabled`, `yolo_model_path`, `yolo_min_confidence`: reserved for a future trained model backend. Current v1.2 keeps YOLO disabled.
|
|
|
|
## v1.2 Runtime Flow
|
|
|
|
1. `RTSPFrameSource` captures a resized RGB frame.
|
|
2. `ZoneOccupancyDetector` updates per-zone binary occupancy and generic trash-motion count from calibrated ROIs.
|
|
3. `TrajectoryTracker` watches zones that just cleared, follows lightweight motion points toward the trash ROI, and emits source-specific `DisposalEvidence` when confidence passes the configured threshold. If the middle of the path is occluded, a segmented source-to-trash track can still emit evidence.
|
|
4. `BatchEngine` processes `Observation(zone_counts, trash_deposit_count, disposal_evidence)`.
|
|
5. For pending disposal, matching `disposal_evidence.source_zone_id` confirms `batch_discarded` before generic FIFO `trash_deposit_count` fallback is used.
|
|
6. Runtime writes events to `logs/events.jsonl` and diagnostics to `logs/runtime_diagnostics.jsonl`.
|
|
|
|
The current tracker is a motion backend only. A later trained YOLO detector should plug in as another backend that enriches or replaces the evidence producer while preserving the same `disposal_evidence` contract consumed by the engine.
|
|
|
|
## Diagnostics
|
|
|
|
- Runtime diagnostics JSONL records one item per runtime iteration.
|
|
- Root `disposal_evidence` is the exact evidence list passed into the engine for that iteration.
|
|
- `diagnostics.zones` contains occupancy metrics used to derive `zone_counts`.
|
|
- `diagnostics.lighting_shift` reports whether global brightness drift suppressed occupancy transitions.
|
|
- `diagnostics.trash` contains generic trash-motion metrics and cooldown state.
|
|
- `diagnostics.trajectory` contains v1.2 candidate counts, emitted evidence count, motion point count, segmented-track flags, and per-candidate emitted/rejected/expired records.
|
|
- Capture failures still keep the v1.2 schema with root `disposal_evidence: []` and `diagnostics.trajectory.reason = "frame_capture_failed"`.
|
|
|
|
## Event Model
|
|
|
|
- `batch_started`: a food region changes from empty to occupied.
|
|
- `time_alarm`: an active batch reaches `max_dwell_seconds` while still in the display region.
|
|
- `batch_count_changed`: count decreases while the region remains occupied.
|
|
- `mixed_batch_violation`: count increases before the region clears.
|
|
- `batch_consumed`: a non-alarmed batch clears before the threshold.
|
|
- `batch_pending_disposal`: an alarmed batch clears and waits for trash confirmation.
|
|
- `batch_discarded`: a pending alarmed batch is matched to a trash deposit.
|
|
- `warning_escalated`: a pending alarmed batch is not matched to a trash deposit before the confirmation deadline.
|
|
|
|
Events should include `zone_id`, `zone_index`, `zone_label`, `started_at`, `dwell_seconds`, and relevant alarm/removal/deadline timestamps when available.
|
|
|
|
In v1.2, `batch_discarded` can be triggered by zone-scoped `disposal_evidence` before falling back to generic `trash_deposit_count`. Evidence must match the pending batch's `source_zone_id`.
|
|
|
|
## Runbook
|
|
|
|
- Python tests:
|
|
- `PYTHONPATH=src python3 -m unittest discover -s tests -v`
|
|
- Management API:
|
|
- `scripts/run_manage_api.sh`
|
|
- Health: `curl http://127.0.0.1:19080/api/manage/health`
|
|
- Web console:
|
|
- `scripts/run_web.sh`
|
|
- Build: `cd web && pnpm build`
|
|
- Frontend logic tests: `node --test web/test/zone-state.test.js`
|
|
- Docker web URL: `http://127.0.0.1:23000`
|
|
- Runtime monitor:
|
|
- `scripts/run_runtime.sh`
|
|
- One-frame smoke test when camera and `ffmpeg` are available:
|
|
- `PYTHONPATH=src python3 -m cold_display_guard.main --config config/example.toml --once`
|
|
- v1.2 operating notes:
|
|
- Keep `trajectory_backend = "motion"` and `yolo_enabled = false` unless a trained YOLO backend has been explicitly deployed.
|
|
- Confirm `logs/runtime_diagnostics.jsonl` contains top-level `disposal_evidence` and `diagnostics.trajectory` before judging trajectory behavior from events alone.
|
|
- When `TrajectoryTracker` has active candidates, runtime sampling uses `trajectory_sample_interval_seconds`; this can temporarily be faster than the normal `sample_interval_seconds`.
|
|
- On remote deployments, preserve the remote `config/example.toml` calibration and stream settings when syncing code.
|
|
|
|
## Known Risks
|
|
|
|
- The current vision detector is heuristic and reports binary occupancy, not item counts.
|
|
- The lighting-shift guard suppresses multi-zone brightness/exposure jumps; if operators intentionally fill most zones at once under a large lighting change, diagnostics should be reviewed before treating that interval as clean data.
|
|
- v1.2 motion tracking improves disposal matching but can still miss movement if the hand/object path is occluded, too broad, too small, or sampled too sparsely.
|
|
- YOLO config fields are present for compatibility, but no trained YOLO model is part of the current runtime.
|
|
- If food is already present during baseline collection, those regions may be treated as empty baseline until visual changes occur.
|
|
- Changing calibration while the runtime process has active batches can create operational ambiguity; v1.1 should document or enforce a pause/restart expectation.
|
|
- Historical events must keep the zone index at the time of emission so later region reordering does not reinterpret old logs.
|