191 lines
5.9 KiB
Markdown
191 lines
5.9 KiB
Markdown
# Hik Cloud Download Analysis Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add Hik Cloud Storage recording download as a configurable multi-device source, then feed downloaded videos into the existing model analysis pipeline.
|
|
|
|
**Architecture:** Keep the current local-folder pipeline intact. Add a cloud acquisition module that plans one-hour chunks, calls the Hik download-address API, downloads videos to local output storage, records a download manifest, and returns local file records for the existing probe/frame/clip/inference/aggregate stages.
|
|
|
|
**Tech Stack:** Python standard library, existing `unittest` suite, existing JSONL manifest helpers, FFmpeg/vLLM pipeline already in `video_ai_analysis_poc`.
|
|
|
|
---
|
|
|
|
### Task 1: Config Schema And Time Chunking
|
|
|
|
**Files:**
|
|
- Modify: `video_ai_analysis_poc/config.py`
|
|
- Create: `video_ai_analysis_poc/hik_cloud.py`
|
|
- Modify: `tests/test_config.py`
|
|
- Create: `tests/test_hik_cloud.py`
|
|
|
|
- [ ] **Step 1: Write failing config tests**
|
|
|
|
Add tests that load:
|
|
|
|
```yaml
|
|
source:
|
|
mode: hik_cloud
|
|
hik_cloud:
|
|
access_token_env: HIK_CLOUD_ACCESS_TOKEN
|
|
devices:
|
|
- device_serial: EXAMPLE_DEVICE_SERIAL
|
|
channel_no: 1
|
|
name: front
|
|
time_ranges:
|
|
- begin: "2026-02-03 09:00:00"
|
|
end: "2026-02-03 10:30:00"
|
|
```
|
|
|
|
Expected: `source.mode == "hik_cloud"`, `devices` is a list of dicts, and `time_ranges` is a list of dicts.
|
|
|
|
- [ ] **Step 2: Write failing chunk tests**
|
|
|
|
Test that `build_download_chunks(...)` converts the range above into chunks with `timeEnd - timeBegin <= 3600`.
|
|
|
|
- [ ] **Step 3: Run red tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
python3 -B -m unittest tests.test_config tests.test_hik_cloud -v
|
|
```
|
|
|
|
Expected: fail because list-of-mapping parsing and `hik_cloud.py` do not exist yet.
|
|
|
|
- [ ] **Step 4: Implement minimal parser/defaults/chunking**
|
|
|
|
Extend the simple YAML parser only enough for list items shaped as mappings. Add defaults for `source` and `hik_cloud`. Implement date-time parsing with `zoneinfo.ZoneInfo`.
|
|
|
|
- [ ] **Step 5: Run green tests**
|
|
|
|
Run the same unittest command. Expected: pass.
|
|
|
|
### Task 2: Hik Download Address API Client
|
|
|
|
**Files:**
|
|
- Modify: `video_ai_analysis_poc/hik_cloud.py`
|
|
- Modify: `tests/test_hik_cloud.py`
|
|
|
|
- [ ] **Step 1: Write failing API client tests**
|
|
|
|
Mock the HTTP function and verify:
|
|
|
|
- URL is `api_base_url.rstrip("/") + download_path`.
|
|
- Headers include `Authorization: bearer TOKEN`.
|
|
- JSON body includes `deviceSerial`, `channelNo`, `timeBegin`, `timeEnd`.
|
|
- Success returns URL and actual begin/end.
|
|
- Code `80438027` returns a structured `no_recording` result.
|
|
- Other non-zero codes return `address_failed`.
|
|
|
|
- [ ] **Step 2: Run red tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
python3 -B -m unittest tests.test_hik_cloud -v
|
|
```
|
|
|
|
Expected: fail because the client is missing.
|
|
|
|
- [ ] **Step 3: Implement client**
|
|
|
|
Use `urllib.request` and injectable callables for tests. Do not log or persist the token.
|
|
|
|
- [ ] **Step 4: Run green tests**
|
|
|
|
Run the same command. Expected: pass.
|
|
|
|
### Task 3: Download Files And Manifest
|
|
|
|
**Files:**
|
|
- Modify: `video_ai_analysis_poc/hik_cloud.py`
|
|
- Modify: `video_ai_analysis_poc/paths.py`
|
|
- Modify: `tests/test_hik_cloud.py`
|
|
|
|
- [ ] **Step 1: Write failing downloader tests**
|
|
|
|
Mock address results and download bytes. Verify downloaded files are written under `downloads/hik_cloud/<device>/ch<channel>/`, filenames contain requested timestamps, manifest rows are written, token/query signatures are not in filenames, and resume skips already downloaded files.
|
|
|
|
- [ ] **Step 2: Run red tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
python3 -B -m unittest tests.test_hik_cloud -v
|
|
```
|
|
|
|
Expected: fail because downloader/manifest behavior is missing.
|
|
|
|
- [ ] **Step 3: Implement downloader**
|
|
|
|
Write `download_hik_cloud_recordings(config, output_dir, *, address_client=None, download_url=None)` returning downloaded video records with cloud metadata.
|
|
|
|
- [ ] **Step 4: Run green tests**
|
|
|
|
Run the same command. Expected: pass.
|
|
|
|
### Task 4: CLI Cloud Source Integration
|
|
|
|
**Files:**
|
|
- Modify: `video_ai_analysis_poc/cli.py`
|
|
- Modify: `tests/test_cli.py`
|
|
|
|
- [ ] **Step 1: Write failing CLI tests**
|
|
|
|
Add tests that:
|
|
|
|
- `source.mode: local` still uses `discover_videos`.
|
|
- `source.mode: hik_cloud` calls the cloud downloader and probes returned downloaded paths.
|
|
- `--dry-run` in cloud mode requests download addresses and writes the download manifest, but does not download video files, probe, call FFmpeg, call VLM, or aggregate.
|
|
- `--until clips` in cloud mode produces video/frame/clip manifests from mocked downloaded video records.
|
|
|
|
- [ ] **Step 2: Run red tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
python3 -B -m unittest tests.test_cli -v
|
|
```
|
|
|
|
Expected: fail because CLI has no source mode branch.
|
|
|
|
- [ ] **Step 3: Implement CLI branch**
|
|
|
|
Keep local behavior unchanged. In cloud mode, call downloader before probe and carry cloud metadata into `video_manifest.jsonl`.
|
|
|
|
- [ ] **Step 4: Run green tests**
|
|
|
|
Run the same command. Expected: pass.
|
|
|
|
### Task 5: Docs, Example Config, And Full Verification
|
|
|
|
**Files:**
|
|
- Modify: `config/local_batch.yaml`
|
|
- Modify: `docs/project.md`
|
|
- Modify: `findings.md`
|
|
- Modify: `progress.md`
|
|
- Modify: `memories.md`
|
|
|
|
- [ ] **Step 1: Update docs/config**
|
|
|
|
Add a commented or safe example for `source.mode: hik_cloud`, token env var, devices, and time ranges. Do not include a real token.
|
|
|
|
- [ ] **Step 2: Run full tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
python3 -B -m unittest discover -s tests -v
|
|
python3 -B -m py_compile video_ai_analysis_poc/*.py
|
|
```
|
|
|
|
Expected: all pass.
|
|
|
|
- [ ] **Step 3: Run local mock smoke**
|
|
|
|
Use test mocks or a temporary local HTTP fixture to verify cloud mode can produce downloaded files and continue to `--until clips` without a real Hik token.
|
|
|
|
- [ ] **Step 4: Record results**
|
|
|
|
Update `progress.md` with commands, results, files changed, and remaining risk. Real Hik API verification is skipped until a real AccessToken/device/time range is provided.
|