cold_display_guard/docs/project.md

Cold Display Guard Project Documentation

Goal

cold-display-guard monitors refrigerated display food batches by camera region. It tracks how long each configured food region remains occupied, raises a configurable time alarm, and escalates alarmed food to a warning if it is removed without a matching trash-bin deposit.

The v1.1 优化改造 batch upgrades the product from a fixed 8-zone layout to a configurable 1-10 zone workflow with numeric region labels and editable trash ROI calibration. All v1.1 items are part of one batch; backend, API, frontend, and documentation are implementation workstreams inside that same batch.

The v1.2 轨迹识别 batch adds source-zone trajectory evidence for disposal confirmation. The first implementation uses lightweight motion tracking and keeps YOLO disabled, while preserving an evidence contract that a later trained YOLO product detector can enrich.

Architecture

• Backend package: src/cold_display_guard/
  • models.py: settings, observations, and batch dataclasses.
  • engine.py: deterministic batch state machine.
  • config.py: TOML config load/save, calibration merge, and project path resolution.
  • manage_api.py: standard-library HTTP management API.
  • main.py: RTSP runtime loop connecting frame capture, vision, state engine, and JSONL sinks.
  • vision.py: heuristic ROI occupancy and trash-motion detection.
  • v1.2 adds trajectory evidence between vision and engine: TrajectoryTracker emits source-zone-to-trash evidence; BatchEngine consumes the backend-neutral disposal_evidence contract.
• Frontend package: web/
  • Vite + vanilla JavaScript management console.
  • Default web port 23000.
  • API proxy target http://127.0.0.1:19080.
  • Runtime home view falls back to clearly marked demo data when no real events exist, so the operational layout still shows summary cards, dwell timers, and event rows.
• Runtime data:
  • Events JSONL default path logs/events.jsonl.
  • Diagnostics JSONL default path logs/runtime_diagnostics.jsonl.
  • v1.2 diagnostics include root-level disposal_evidence plus diagnostics.trajectory.
• Deployment:
  • Root Python Docker image for API/runtime.
  • web/Dockerfile for static web console.
  • deploy/docker-compose.yml wires API, runtime, and web services.

Configuration

• Main config path: config/example.toml.
• Camera identity: camera_id.
• Timezone default: Asia/Shanghai.
• RTSP input: [stream] rtsp_url.
• Thresholds:
  • max_dwell_seconds: v1.1 time-alarm threshold. Default can remain 10800 seconds; users can set values such as 1200 seconds for 20 minutes.
  • trash_confirmation_seconds: window after an alarmed batch is removed where a trash deposit must be observed before warning escalation.
• Food zones:
  • v1.1 food zone IDs are numeric strings from "1" through "10".
  • The configured zone count must be between 1 and 10.
  • If both zone_count and numeric zone_ids are present, they must agree.
  • Each [[zones]] polygon must have at least 3 normalized points.
• Trash ROI:
  • Stored under [trash] roi.
  • Does not use a food zone number.
• v1.2 trajectory settings:
  • lighting_shift_guard_enabled: freezes occupancy changes when many regions shift brightness in the same direction.
  • lighting_shift_min_regions, lighting_shift_region_fraction, lighting_shift_mean_delta: tune the global lighting/exposure guard.
  • trajectory_enabled: enables source-zone trajectory evidence.
  • trajectory_window_seconds: seconds after a zone clears where movement can confirm disposal.
  • trajectory_sample_interval_seconds: faster runtime delay while a candidate is active.
  • trajectory_min_points: minimum sampled motion points required before evidence can emit.
  • trajectory_segmented_enabled, trajectory_segmented_min_points: allow a source point plus trash-entry point to confirm disposal when the middle of the path is occluded.
  • trajectory_min_confidence: minimum confidence before evidence can close pending disposal.
  • trajectory_motion_delta: frame-difference threshold for trajectory motion points.
  • trajectory_min_blob_area: minimum connected motion area to keep as a point.
  • trajectory_max_blob_area_fraction: rejects overly broad frame motion as ambiguous.
  • trajectory_trash_entry_margin: margin for treating a track point as entering the trash ROI.
  • trajectory_backend: first valid value is "motion".
  • yolo_enabled, yolo_model_path, yolo_min_confidence: reserved for a future trained model backend. Current v1.2 keeps YOLO disabled.

v1.2 Runtime Flow

1. RTSPFrameSource captures a resized RGB frame.
2. ZoneOccupancyDetector updates per-zone binary occupancy and generic trash-motion count from calibrated ROIs.
3. TrajectoryTracker watches zones that just cleared, follows lightweight motion points toward the trash ROI, and emits source-specific DisposalEvidence when confidence passes the configured threshold. If the middle of the path is occluded, a segmented source-to-trash track can still emit evidence.
4. BatchEngine processes Observation(zone_counts, trash_deposit_count, disposal_evidence).
5. For pending disposal, matching disposal_evidence.source_zone_id confirms batch_discarded before generic FIFO trash_deposit_count fallback is used.
6. Runtime writes events to logs/events.jsonl and diagnostics to logs/runtime_diagnostics.jsonl.

The current tracker is a motion backend only. A later trained YOLO detector should plug in as another backend that enriches or replaces the evidence producer while preserving the same disposal_evidence contract consumed by the engine.

Diagnostics

• Runtime diagnostics JSONL records one item per runtime iteration.
• Root disposal_evidence is the exact evidence list passed into the engine for that iteration.
• diagnostics.zones contains occupancy metrics used to derive zone_counts.
• diagnostics.lighting_shift reports whether global brightness drift suppressed occupancy transitions.
• diagnostics.trash contains generic trash-motion metrics and cooldown state.
• diagnostics.trajectory contains v1.2 candidate counts, emitted evidence count, motion point count, segmented-track flags, and per-candidate emitted/rejected/expired records.
• Capture failures still keep the v1.2 schema with root disposal_evidence: [] and diagnostics.trajectory.reason = "frame_capture_failed".

Event Model

• batch_started: a food region changes from empty to occupied.
• time_alarm: an active batch reaches max_dwell_seconds while still in the display region.
• batch_count_changed: count decreases while the region remains occupied.
• mixed_batch_violation: count increases before the region clears.
• batch_consumed: a non-alarmed batch clears before the threshold.
• batch_pending_disposal: an alarmed batch clears and waits for trash confirmation.
• batch_discarded: a pending alarmed batch is matched to a trash deposit.
• warning_escalated: a pending alarmed batch is not matched to a trash deposit before the confirmation deadline.

Events should include zone_id, zone_index, zone_label, started_at, dwell_seconds, and relevant alarm/removal/deadline timestamps when available.

In v1.2, batch_discarded can be triggered by zone-scoped disposal_evidence before falling back to generic trash_deposit_count. Evidence must match the pending batch's source_zone_id.

Runbook

• Python tests:
  • PYTHONPATH=src python3 -m unittest discover -s tests -v
• Management API:
  • scripts/run_manage_api.sh
  • Health: curl http://127.0.0.1:19080/api/manage/health
• Web console:
  • scripts/run_web.sh
  • Build: cd web && pnpm build
  • Frontend logic tests: node --test web/test/zone-state.test.js
  • Docker web URL: http://127.0.0.1:23000
• Runtime monitor:
  • scripts/run_runtime.sh
  • One-frame smoke test when camera and ffmpeg are available:
    • PYTHONPATH=src python3 -m cold_display_guard.main --config config/example.toml --once
• v1.2 operating notes:
  • Keep trajectory_backend = "motion" and yolo_enabled = false unless a trained YOLO backend has been explicitly deployed.
  • Confirm logs/runtime_diagnostics.jsonl contains top-level disposal_evidence and diagnostics.trajectory before judging trajectory behavior from events alone.
  • When TrajectoryTracker has active candidates, runtime sampling uses trajectory_sample_interval_seconds; this can temporarily be faster than the normal sample_interval_seconds.
  • On remote deployments, preserve the remote config/example.toml calibration and stream settings when syncing code.

Known Risks

• The current vision detector is heuristic and reports binary occupancy, not item counts.
• The lighting-shift guard suppresses multi-zone brightness/exposure jumps; if operators intentionally fill most zones at once under a large lighting change, diagnostics should be reviewed before treating that interval as clean data.
• v1.2 motion tracking improves disposal matching but can still miss movement if the hand/object path is occluded, too broad, too small, or sampled too sparsely.
• YOLO config fields are present for compatibility, but no trained YOLO model is part of the current runtime.
• If food is already present during baseline collection, those regions may be treated as empty baseline until visual changes occur.
• Changing calibration while the runtime process has active batches can create operational ambiguity; v1.1 should document or enforce a pause/restart expectation.
• Historical events must keep the zone index at the time of emission so later region reordering does not reinterpret old logs.