Skip to main content

Testing & quality

ContentsVision has one firm, non-negotiable rule, written at the top of the repo's CLAUDE.md:

The hard rule

Run the full test suite after any change to the app, and all tests must pass before the work is considered done. A change with no test is incomplete.

This page explains how to run the suite and why it is built the way it is.

Running the tests​

One command, from the repo root:

./run_tests.sh            # run everything
./run_tests.sh -k billing # only tests matching "billing"
./run_tests.sh -x         # stop at the first failure

run_tests.sh activates the backend virtualenv, installs the test-only dependencies once (pytest, pytest-asyncio), and runs pytest from backend/. Any extra arguments pass straight through to pytest.

Offline by design​

The suite must stay fast, offline, and safe to run anytime. Two guarantees make that true (backend/tests/conftest.py):

  1. A throwaway in-memory database. DATABASE_URL is forced to sqlite+aiosqlite:///:memory: before the app is imported, and each test gets a fresh engine. The real dev database at storage/inventory.db is never opened.
  2. No third-party calls. The suite never reaches the AI on OpenRouter, transcription on Deepgram, the Adjust Square API, or PostHog. Anything that needs an external service mocks that layer. PostHog analytics is simply off in tests (no key is set), so core/analytics.py no-ops without even importing the SDK.

The test client (conftest.py) seeds a team and a user and mints a valid AdjustSquare-format token with make_test_token, so auth-protected endpoints work without hitting real SSO. The get_db dependency is overridden to share the in-memory engine, so data seeded directly is visible to API calls.

What the suite covers​

The backend/tests/ directory has broad coverage: claim and room APIs, the inventory API, uploads, jobs, billing (unit and API), auth, auto-approve, numbering, the Adjust Square client, observability, prompt settings, the activity log and service, audio and video helpers, audio chunking, and an end-to-end pipeline test with the AI layer mocked. Tests are configured in pytest.ini (async auto-mode, verbose one-line-per-test output driven by each test's docstring).

Conventions​

  • Add or update tests with every change. A feature or fix includes its test.
  • Show real failures. If a test fails, show the actual failing output. Never claim green when it is red.
  • Keep the default suite offline. Anything that needs a real external service must be mocked, or gated behind an explicit opt-in marker so ./run_tests.sh never touches the network.

Mocking the external layers​

Because the pipeline and Adjust Square client are isolated services, tests mock at their boundaries:

  • The AI analyzer functions (claude_analyzer) are patched to return canned items, so the pipeline can be exercised end to end without a real model call.
  • The Adjust Square client functions are patched to capture the arguments they are called with (for example, asserting the right bearer token and adjuster id are passed) and to return fake responses.

This keeps tests deterministic and instant while still covering the real control flow. For the broader engineering conventions (UI, motion, writing style), see conventions.

Last updated on