yangyl/cold_display_guard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a4657a4bdf3dc7c97ef39ceadbd373b29981fe5a
cold_display_guard/docs/project.md

9.6 KiB
Raw Blame History

Cold Display Guard Project Documentation

Goal

cold-display-guard monitors refrigerated display food batches by camera region. It tracks how long each configured food region remains occupied, raises a configurable time alarm, and escalates alarmed food to a warning if it is removed without a matching trash-bin deposit.

The v1.1 优化改造 batch upgrades the product from a fixed 8-zone layout to a configurable 1-10 zone workflow with numeric region labels and editable trash ROI calibration. All v1.1 items are part of one batch; backend, API, frontend, and documentation are implementation workstreams inside that same batch.

The v1.2 轨迹识别 batch adds source-zone trajectory evidence for disposal confirmation. The first implementation uses lightweight motion tracking and keeps YOLO disabled, while preserving an evidence contract that a later trained YOLO product detector can enrich.

Architecture

  • Backend package: src/cold_display_guard/
    • models.py: settings, observations, and batch dataclasses.
    • engine.py: deterministic batch state machine.
    • config.py: TOML config load/save, calibration merge, and project path resolution.
    • manage_api.py: standard-library HTTP management API.
    • main.py: RTSP runtime loop connecting frame capture, vision, state engine, and JSONL sinks.
    • vision.py: heuristic ROI occupancy and trash-motion detection.
    • v1.2 adds trajectory evidence between vision and engine: TrajectoryTracker emits source-zone-to-trash evidence; BatchEngine consumes the backend-neutral disposal_evidence contract.
  • Frontend package: web/
    • Vite + vanilla JavaScript management console.
    • Default web port 23000.
    • API proxy target http://127.0.0.1:19080.
    • Runtime home view falls back to clearly marked demo data when no real events exist, so the operational layout still shows summary cards, dwell timers, and event rows.
  • Runtime data:
    • Events JSONL default path logs/events.jsonl.
    • Diagnostics JSONL default path logs/runtime_diagnostics.jsonl.
    • v1.2 diagnostics include root-level disposal_evidence plus diagnostics.trajectory.
  • Deployment:
    • Root Python Docker image for API/runtime.
    • web/Dockerfile for static web console.
    • deploy/docker-compose.yml wires API, runtime, and web services.

Configuration

  • Main config path: config/example.toml.
  • Camera identity: camera_id.
  • Timezone default: Asia/Shanghai.
  • RTSP input: [stream] rtsp_url.
  • Thresholds:
    • max_dwell_seconds: v1.1 time-alarm threshold. Default can remain 10800 seconds; users can set values such as 1200 seconds for 20 minutes.
    • trash_confirmation_seconds: window after an alarmed batch is removed where a trash deposit must be observed before warning escalation.
  • Food zones:
    • v1.1 food zone IDs are numeric strings from "1" through "10".
    • The configured zone count must be between 1 and 10.
    • If both zone_count and numeric zone_ids are present, they must agree.
    • Each [[zones]] polygon must have at least 3 normalized points.
  • Trash ROI:
    • Stored under [trash] roi.
    • Does not use a food zone number.
  • v1.2 trajectory settings:
    • lighting_shift_guard_enabled: freezes occupancy changes when many regions shift brightness in the same direction.
    • lighting_shift_min_regions, lighting_shift_region_fraction, lighting_shift_mean_delta: tune the global lighting/exposure guard.
    • trajectory_enabled: enables source-zone trajectory evidence.
    • trajectory_window_seconds: seconds after a zone clears where movement can confirm disposal.
    • trajectory_sample_interval_seconds: faster runtime delay while a candidate is active.
    • trajectory_min_points: minimum sampled motion points required before evidence can emit.
    • trajectory_segmented_enabled, trajectory_segmented_min_points: allow a source point plus trash-entry point to confirm disposal when the middle of the path is occluded.
    • trajectory_min_confidence: minimum confidence before evidence can close pending disposal.
    • trajectory_motion_delta: frame-difference threshold for trajectory motion points.
    • trajectory_min_blob_area: minimum connected motion area to keep as a point.
    • trajectory_max_blob_area_fraction: rejects overly broad frame motion as ambiguous.
    • trajectory_trash_entry_margin: margin for treating a track point as entering the trash ROI.
    • trajectory_backend: first valid value is "motion".
    • yolo_enabled, yolo_model_path, yolo_min_confidence: reserved for a future trained model backend. Current v1.2 keeps YOLO disabled.

v1.2 Runtime Flow

  1. RTSPFrameSource captures a resized RGB frame.
  2. ZoneOccupancyDetector updates per-zone binary occupancy and generic trash-motion count from calibrated ROIs.
  3. TrajectoryTracker caches recent source-region motion while a zone is occupied, watches zones that just cleared, follows lightweight motion points toward the trash ROI, and emits source-specific DisposalEvidence when confidence passes the configured threshold. If the middle of the path is occluded, a segmented source-to-trash track can still emit evidence.
  4. BatchEngine processes Observation(zone_counts, trash_deposit_count, disposal_evidence).
  5. For pending disposal, matching disposal_evidence.source_zone_id confirms batch_discarded before generic FIFO trash_deposit_count fallback is used.
  6. Runtime writes events to logs/events.jsonl and diagnostics to logs/runtime_diagnostics.jsonl.

The current tracker is a motion backend only. A later trained YOLO detector should plug in as another backend that enriches or replaces the evidence producer while preserving the same disposal_evidence contract consumed by the engine.

Diagnostics

  • Runtime diagnostics JSONL records one item per runtime iteration.
  • Root disposal_evidence is the exact evidence list passed into the engine for that iteration.
  • diagnostics.zones contains occupancy metrics used to derive zone_counts.
  • diagnostics.lighting_shift reports whether global brightness drift suppressed occupancy transitions.
  • diagnostics.trash contains generic trash-motion metrics and cooldown state.
  • diagnostics.trajectory contains v1.2 candidate counts, emitted evidence count, motion point count, source-seeded and segmented-track flags, and per-candidate emitted/rejected/expired records.
  • Capture failures still keep the v1.2 schema with root disposal_evidence: [] and diagnostics.trajectory.reason = "frame_capture_failed".

Event Model

  • batch_started: a food region changes from empty to occupied.
  • time_alarm: an active batch reaches max_dwell_seconds while still in the display region.
  • batch_count_changed: count decreases while the region remains occupied.
  • mixed_batch_violation: count increases before the region clears.
  • batch_consumed: a non-alarmed batch clears before the threshold.
  • batch_pending_disposal: an alarmed batch clears and waits for trash confirmation.
  • batch_discarded: a pending alarmed batch is matched to a trash deposit.
  • warning_escalated: a pending alarmed batch is not matched to a trash deposit before the confirmation deadline.

Events should include zone_id, zone_index, zone_label, started_at, dwell_seconds, and relevant alarm/removal/deadline timestamps when available.

In v1.2, batch_discarded can be triggered by zone-scoped disposal_evidence before falling back to generic trash_deposit_count. Evidence must match the pending batch's source_zone_id.

Runbook

  • Python tests:
    • PYTHONPATH=src python3 -m unittest discover -s tests -v
  • Management API:
    • scripts/run_manage_api.sh
    • Health: curl http://127.0.0.1:19080/api/manage/health
  • Web console:
    • scripts/run_web.sh
    • Build: cd web && pnpm build
    • Frontend logic tests: node --test web/test/zone-state.test.js
    • Docker web URL: http://127.0.0.1:23000
  • Runtime monitor:
    • scripts/run_runtime.sh
    • One-frame smoke test when camera and ffmpeg are available:
      • PYTHONPATH=src python3 -m cold_display_guard.main --config config/example.toml --once
  • v1.2 operating notes:
    • Keep trajectory_backend = "motion" and yolo_enabled = false unless a trained YOLO backend has been explicitly deployed.
    • Confirm logs/runtime_diagnostics.jsonl contains top-level disposal_evidence and diagnostics.trajectory before judging trajectory behavior from events alone.
    • When TrajectoryTracker has active candidates, runtime sampling uses trajectory_sample_interval_seconds; this can temporarily be faster than the normal sample_interval_seconds.
    • On remote deployments, preserve the remote config/example.toml calibration and stream settings when syncing code.

Known Risks

  • The current vision detector is heuristic and reports binary occupancy, not item counts.
  • The lighting-shift guard suppresses multi-zone brightness/exposure jumps; if operators intentionally fill most zones at once under a large lighting change, diagnostics should be reviewed before treating that interval as clean data.
  • v1.2 motion tracking improves disposal matching but can still miss movement if the hand/object path is occluded, too broad, too small, or sampled too sparsely.
  • YOLO config fields are present for compatibility, but no trained YOLO model is part of the current runtime.
  • If food is already present during baseline collection, those regions may be treated as empty baseline until visual changes occur.
  • Changing calibration while the runtime process has active batches can create operational ambiguity; v1.1 should document or enforce a pause/restart expectation.
  • Historical events must keep the zone index at the time of emission so later region reordering does not reinterpret old logs.