Skip to main content

Contents Capture integration

Contents Capture is the field-capture platform (an iOS app backed by a Cloudflare Workers API, D1 metadata database, and a private R2 media bucket). ContentsVision integrates with it as a pull-based media source: users browse a connected Capture org like a folder tree and import selected media into claims.

Architecture

ContentsVision PULLS media. All Capture traffic is server-to-server from the backend; the browser never talks to the Capture API and never sees the org token.

ContentsVision UI
| /api/integrations/... (team-scoped)
ContentsVision backend --Bearer cci_ token--> Contents Capture API
| |
| <-- short-lived presigned R2 GET URLs ------+
+--> downloads bytes, creates claims/rooms/jobs

Key properties:

  • Org-level access. A Contents Capture org admin mints an org integration token (cci_...) in the Capture admin console. A ContentsVision team admin pastes it under Settings, then Integrations. The token is read-only on the Capture side (its principal is an org-wide observer) and is validated against GET /integrations/whoami before it is stored.
  • The Capture API is the source of truth. Only media with syncState == "synced" has its original durably stored; browse and import filter on it. ContentsVision holds no R2 credentials: bytes are fetched through presigned GET URLs the Capture API signs per media id (1 hour TTL, signed in batches immediately before download).
  • Entity mapping. Capture assignment -> claim (the assignment reference becomes the claim number), location -> room (matched by name, created when missing), group -> staged photo group_index (grouping arrives pre-done), photo -> StagedPhoto in one photo job per room, video/audio -> one job per file.
  • Billing is never bypassed. Imported photo jobs park at awaiting_grouping and video/audio jobs at awaiting_confirmation; the user starts them through the normal quote/process step.

Data model

Three tables, declared in backend/app/models/integration.py and registered in database.py:init_db() (whole new tables, so no ALTER lines):

TablePurpose
storage_connectionsOne configured provider per team (kind = contents_capture). Holds the Capture org id (external_account_id) and the org token (api_token, following the Team.team_token precedent; secret_ref is the upgrade path to a secret manager).
uploadsOne piece of source media with provenance: (connection_id, external_media_id) is the idempotency key, sha256/size_bytes come from the Capture variant record, meta snapshots the capture tree (assignment/location/group ids). Lifecycle: `registered -> importing -> ready
import_batchesOne user-initiated import run: the selection manifest verbatim, progress counters (total/imported/skipped/failed), the claim it landed in, and the job ids it created. The UI polls this row.

Provider layer

backend/app/services/integrations/ holds the adapter layer:

  • base.py defines the StorageProvider interface (browse a virtual tree, stat one item, stream its bytes) plus ProviderCapabilities, so future providers (S3, Dropbox, ...) plug in without touching callers.
  • registry.py maps StorageConnection.kind to a provider class via @register_provider / build_provider.
  • contents_capture.py implements the Capture provider: CaptureClient is a thin async httpx client over the Capture read API (same house style as adjust_square.py), and ContentsCaptureProvider is registered under contents_capture.
  • importer.py is the import engine (see below).

API endpoints

All under /api/integrations, team-scoped via the standard Actor dependency (backend/app/api/routes/integrations.py):

EndpointPurpose
GET /integrations/connectionsThe team's connections plus contents_capture_available (false when the server has no base URL configured, which hides the integration in the UI).
POST /integrations/contents-capture/connectValidate a pasted cci_ token via whoami and store/replace the team's connection.
DELETE /integrations/connections/{id}Disconnect.
GET .../connections/{id}/teams, .../teams/{tid}/assignments, .../assignments/{aid}/tree, .../assignments/{aid}/mediaBrowse proxy over the Capture read API. The media page passes Capture's presigned thumbnail URLs through to the grid.
POST .../connections/{id}/importSave a selection manifest on an ImportBatch and enqueue run_capture_import.
GET /integrations/imports/{batch_id}Poll import progress.

The selection manifest

The import request is selector-based, never a flat list of media ids, so selecting "everything" stays a tiny request and the server expands it:

{
"assignment_id": "...",
"target": {"mode": "new_claim" | "existing_claim" | "room", "claim_id": "...", "room_id": "..."},
"select": [{"scope": "assignment"} , {"scope": "location", "id": "..."},
{"scope": "group", "id": "..."}, {"scope": "media", "ids": ["..."]}],
"exclude": [ ...same shapes... ],
"filters": {"types": ["photo"], "submitted_only": false}
}

The import job

run_capture_import runs as an ARQ function (Stage 2) or a BackgroundTask (Stage 1), registered in worker.py and core/queue.py. Per batch it:

  1. expands the manifest against the Capture API (synced media only, cursor-paginated), applies excludes and filters, deduplicates;
  2. skips media already imported on this connection (idempotency key), counting them as skipped;
  3. resolves the target claim and rooms, creating them as needed;
  4. signs original URLs in batches of 200 immediately before downloading (bounded concurrency, CONTENTS_CAPTURE_IMPORT_CONCURRENCY), normalizes photos to JPEG (falling back to the Capture proxy variant when the original is not Pillow-readable, e.g. HEIC), seeds staged_photos with group indices carried over from Capture groups;
  5. updates the batch counters as it goes and finishes complete, partial (some files failed; re-running the import retries just those), or failed.

Configuration

SettingMeaning
CONTENTS_CAPTURE_API_BASE_URLThe Capture API base URL. Empty (default) disables the integration entirely. Dev/stage: https://contents-capture-api.dev-908.workers.dev.
CONTENTS_CAPTURE_WEBHOOK_SECRETHMAC secret for Capture webhooks. Optional; webhooks are a later phase.
CONTENTS_CAPTURE_IMPORT_CONCURRENCYParallel downloads per import run (default 4).

Per-team org tokens live on storage_connections.api_token, not in the environment.

Frontend

  • Settings, Integrations tab (frontend/src/components/IntegrationsCard.tsx): connect (token paste + validate), connected state, disconnect.
  • Import browser (frontend/src/components/CaptureImportModal.tsx): a folder-style explorer (assignment folders on the left, media grid on the right) with tri-state folder checkboxes, shift-click range select, type and submitted-only filters, selector-based selection, and a polling progress view. Opened from the Dashboard ("new claim" imports) and the claim page ("into this claim" imports); the buttons render only when the team has an active connection.

Production cutover

The integration currently points at the Capture dev/stage stack. Moving to production means: provision the Capture production environment (Worker, D1, R2, Clerk live), set CONTENTS_CAPTURE_API_BASE_URL to the production host, and have org admins mint production tokens. No code changes.