diff --git a/backend-spec.md b/backend-spec.md new file mode 100644 index 0000000..0f34019 --- /dev/null +++ b/backend-spec.md @@ -0,0 +1,268 @@ +# Backend Specification: Household Multi-Tenancy (v2) + +This document has been validated against the current codebase (v1) to ensure the plan captures all required changes. It starts with a concise baseline of what exists today, then details the v2 multi-tenancy/auth refactor with concrete, file-scoped steps and acceptance criteria. + +Date reviewed: 2025-11-01 + +Repo modules checked: `main.py`, `api/*`, `persons/*`, `meals/*`, `ingredients/*`, `recipes/*`, `products/*`, `shopping/*`, `common.py`, `db.py`, `settings.py`, tests in `tests/*`. + +Key conventions in v1: +- Response shape uses camelCase aliases (via `ApiModel` in `common.py`). +- RFC7807 Problem Details are returned for 400/404/422 with `application/problem+json` (handlers in `main.py`). +- Pagination uses a `Page` envelope: `{ items: T[], nextCursor?: string, prevCursor?: string, total: number }`. +- Some endpoints require a `user_id` cookie; missing cookie generally yields 422 (validation), except one special-case mapping to 401 (see below). + +--- + +## 0. Current v1 Baseline (validated) + +### 0.1 Auth (prototype) +- Mechanism: `user_id` cookie containing a Person ID. +- Endpoints (`api/auth.py`): + - POST `/api/v1/auth/login` body `{ username: string }` → sets `user_id` cookie if person exists; 404 ProblemDetails if not. + - POST `/api/v1/auth/refresh` → returns the current Person (requires cookie). +- Dependencies (`api/deps.py`): + - `cookie_person`: requires cookie, loads Person by id; 401 if unknown id, 422 if cookie missing (FastAPI validation). + - `error_response`: builds RFC7807 responses. + +Special-case 401: In `main.py`, missing `user_id` cookie on POST `/api/v1/shopping` is mapped from 422 → 401 for “Unauthorized”. All other protected endpoints surface 422 when the cookie is missing. + +### 0.2 API surface +- Persons (`api/persons.py`) + - GET `/api/v1/persons` → `Page` with optional name filter `q`, cursor pagination. + - POST `/api/v1/persons` → create Person; sets `Location` header. + +- Recipes (`api/recipes.py`) + - GET `/api/v1/recipes` → `Page`; loads ingredients per page. + - GET `/api/v1/recipes/{id}` → full recipe (ingredients + createdBy). + - GET `/api/v1/recipes/parse?url=...` (auth required) → scrape/parse a recipe; 400 if not found. + - GET `/api/v1/recipes/ingredients/parse?ingredients=...&ingredients=...` → parse raw ingredient lines (no auth); matches existing products. + - POST `/api/v1/recipes` (auth required) → validate (≥1 ingredient), perform versioning (hide base if id≥0), set createdById, insert ingredients; sets `Location` header. + - DELETE `/api/v1/recipes/{id}` (auth required) → soft-delete (hide) recipe. + +- Meals (`api/meals.py`) + - GET `/api/v1/meals/upcoming?from&to` → upcoming unconsumed meals in range; bulk loads participants, recipes, extra ingredients. + - GET `/api/v1/meals/{id}` → meal with participants/recipes/extra ingredients. + - POST `/api/v1/meals` → validate meal, insert participants, recipes, extra ingredients; sets `Location` header. + - PUT `/api/v1/meals/{id}` → validate/update and return current state. + - POST `/api/v1/meals/{id}/consumed` (auth required) → marks consumed; requires timezone in `consumed_date` if provided; removes any shopping requests for the meal. + - DELETE `/api/v1/meals/{id}` (auth required) → soft-delete meal via shopping cleanup then update. + +- Shopping (`api/shopping.py`) + - GET `/api/v1/shopping/current` → aggregate of outstanding ingredient requests (no auth), requested meals, purchased items, plus lookup maps (`meals`, `recipes`, `ingredients`, and any referenced purchased lists). Outward storeName enum uses `"home"` for internal empty string. + - GET `/api/v1/shopping/{listId}` → purchased shopping list + lookups; 404 if not found. + - POST `/api/v1/shopping` (auth required) → purchase list with items; validates invariants in repository; returns full list + lookups. Missing/invalid cookie maps to 401 (special-case). + - GET `/api/v1/shopping/current/me/ingredients` (auth required) → outstanding ingredient requests for me (personId from cookie). + - POST `/api/v1/shopping/current/me/ingredients` (auth required) → sync my ingredient requests (add missing, remove extra). Matches existing entries by `id` or `line`. + - POST `/api/v1/shopping/current/meals/me` (auth required) → request a meal; 404 if meal not found; prevents duplicates. + - DELETE `/api/v1/shopping/current/meals/{mealId}` (auth required) → unrequest a meal; 404 if meal not found. + +### 0.3 Data model (SQLite, created by `db.create()`) +- Person(id PK, name UNIQUE) +- Recipe(id PK, name, link, serves, image_urls TEXT JSON, based_on_recipe FK, date_created, created_by_id FK NOT NULL, date_hidden, hidden_by_id FK) +- Ingredient(id PK, name, line, preparation, unit, quantity REAL, product_id FK, recipe_id FK, meal_id FK) +- Product(id PK, product_id UNIQUE, shop_code, link, name, quantity, unit, img_small, img_large, raw_data TEXT) + ProductTag(food_item_id, tag) +- Meal(id PK, suggested_date, consumed_date NULL, deleted_date NULL, purchase_date NULL) + - MealParticipant(meal_id, person_id, role) + - MealRecipe(meal_id, recipe_id, servings) +- ShoppingList(id PK, created_date, store_name, purchased_by_id FK) + - ShoppingListItem(id PK, ingredient_id FK, list_id FK NULL for requests, person_id FK, meal_id FK, recipe_id FK, created_date) + +### 0.4 Validated behaviors and invariants +- ProblemDetails content-type returned for 400/404/422. +- Pagination: `cursor` is treated leniently (invalid → start). `prevCursor` is computed by a DB helper; `total` is non-null integer. +- Recipe creation requires ≥1 ingredient; soft-deletes older version when updating. +- Meals validation (`api.meals.validate_meal`): + - Must have ≥1 chef, ≥1 cleanup, ≥1 consumer. + - Must have either at least one recipe or at least one extra ingredient. + - No duplicate participants by role. + - Each MealRecipe.servings must be > 0. + - POST `{id}` and body.id must match for update. +- Shopping invariants (`shopping.repository`): + - Purchase requires `purchased_by_id` and ≥1 item. + - Requests: exactly one of (ingredient|meal); person required; ingredient requests must have valid ingredient id (or be inserted when syncing); meal request must not already exist; purchased meal auto-updates `purchase_date` and clears requests when all meal ingredients are covered by list items. + - Outward `storeName` uses `home|woolworths|coles` (maps internal empty string to `home`). +- Auth cookie missing: + - Most protected endpoints → 422 from FastAPI validation. + - Special case: POST `/api/v1/shopping` → 401 via custom handler. + +--- + +## 1. Objective + +Refactor the backend from a single-tenant architecture to a robust, multi-tenant system based on "Households". This requires evolving the data model to enforce data isolation, overhauling the authentication system to support standard credential types, and introducing an invitation mechanism for household management. This plan is adapted to the existing codebase. + +## 2. Core Concepts & Data Model + +- **Data Isolation**: All primary resources (`recipes`, `meals`, `shopping_lists`, etc.) MUST be strictly scoped to a `household_id`. No API endpoint should ever return data from a household the authenticated user does not belong to. +- **User vs. Person**: The existing `persons` table and `Person` model will be replaced by a `users` table and a `HouseholdMember` link table. +- **Authentication**: Replace the prototype `user_id` cookie with a standard JWT-based system. + +### New & Modified Data Models + +- **`households`** (New Table): + - `id`: Primary Key + - `name`: `TEXT NOT NULL` + - `slug`: `TEXT NOT NULL UNIQUE` +- **`users`** (New Table, replaces `persons`): + - `id`: Primary Key + - `email`: `TEXT NOT NULL UNIQUE` + - `display_name`: `TEXT NOT NULL` + - `profile_photo_url`: `TEXT` +- **`household_members`** (New Table): + - `user_id`: FK to `users.id` + - `household_id`: FK to `households.id` + - `role`: `TEXT NOT NULL` (e.g., 'admin', 'member') + - `PRIMARY KEY (user_id, household_id)` +- **`local_credentials`** (New Table): + - `user_id`: PK, FK to `users.id` + - `hashed_password`: `TEXT NOT NULL` +- **`oauth_credentials`** (New Table): + - `user_id`: FK to `users.id` + - `provider`: `TEXT NOT NULL` (e.g., 'google') + - `provider_user_id`: `TEXT NOT NULL` + - `PRIMARY KEY (provider, provider_user_id)` +- **`household_invitations`** (New Table): + - `id`: Primary Key + - `household_id`: FK to `households.id` + - `email`: `TEXT NOT NULL` + - `invited_by_user_id`: FK to `users.id` + - `token`: `TEXT NOT NULL UNIQUE` + - `expires_at`: `DATETIME NOT NULL` + - `status`: `TEXT NOT NULL` ('pending', 'accepted', 'expired') + +## 3. API & Logic Changes + +### 3.1. Authentication API + +- **Refactor `api/auth.py`**: + - Replace existing login/refresh logic with JWT-based authentication. + - **`POST /api/v1/auth/register`**: Body `{ email, password, displayName }`. Creates `User` and `LocalCredentials`. Returns JWT. + - **`POST /api/v1/auth/login`**: Body `{ email, password }`. Validates credentials. Returns JWT. + - **`POST /api/v1/auth/google`**: Body `{ token }`. Validates Google ID token, finds/creates `User` and `OAuthCredentials`. Returns JWT. +- **JWT Structure**: Payload must contain `user_id`. Use short-lived access tokens with a long-lived refresh token stored in an `HttpOnly` cookie. + +Notes from v1 baseline to carry forward: +- Replace the `user_id` cookie dependency (`cookie_person`) with `get_current_user` (JWT) and update all protected endpoints in: `api/recipes.py`, `api/meals.py`, `api/shopping.py`. +- Preserve RFC7807 error semantics and the existing OpenAPI augmentation in `api/openapi.py`. The special-case 401 mapping for POST `/api/v1/shopping` will become unnecessary—JWT middleware should standardize 401/403. + +### 3.2. Household & Tenancy API + +- **Create `api/households.py`**: + - **`GET /api/v1/users/me/households`**: (Auth: JWT) Returns a list of the user's households. + - **`POST /api/v1/households`**: (Auth: JWT) Body `{ name }`. Creates a `Household` and adds the user as the first 'admin' member. +- **Household-Scoped Routes**: + - All data-related routes must be prefixed: `/api/v1/households/{householdSlug}/...`. + - **Create a new dependency in `api/deps.py`**: `get_household_from_slug` will: + 1. Take `householdSlug` from the path. + 2. Verify the authenticated user (from JWT) is a member of that household. + 3. Raise `HTTPException(403, "Forbidden")` if not a member. + 4. Return the `household` object (or just `household_id`) for use in endpoints. + +Impact on existing routes (exact files to refactor): +- `api/recipes.py`, `api/meals.py`, `api/shopping.py`, `api/products.py` → move under `APIRouter(prefix="/api/v1/households/{householdSlug}")` or mount via a `household_router` in `main.py`. +- Repository functions must accept `household_id` and filter by it in all SELECT/INSERT/UPDATE. See `meals.repository`, `recipes.repository`, `ingredients.repository`, `shopping.repository`, `products.repository`. +- `persons` is replaced by `users + household_members`. Endpoints that currently return `Person` should instead resolve household-scoped `User` references. + +### 3.3. Invitation API + +- **Add to `api/households.py`**: + - **`POST /api/v1/households/{householdSlug}/invitations`**: (Auth: JWT, household membership). Body `{ email }`. Creates `HouseholdInvitation`, sends email. +- **Add to `api/auth.py`**: + - **`POST /api/v1/invitations/accept`**: (Auth: JWT). Body `{ token }`. Validates token, adds user to household. + +## 4. Actionable Implementation Steps + +1. **[ ] Database Schema and Migration**: + - **Schema Definition**: Define the new tables (`users`, `households`, etc.) in new repository files (e.g., `users/repository.py`, `households/repository.py`). The `create` function in each will contain the `CREATE TABLE` SQL. + - **Migration Logic**: Create a new migration script (e.g., in `scripts/migration_to_households.py`). This script will: + - Connect to the database (reusing logic from `db.py`). + - Call the `create()` function for each new repository to create the tables. + - Add a `household_id` column to all existing tenant-specific tables (`recipes`, `meals`, etc.). + - **Data Porting**: + - Create a single default "My Household". + - Read all records from the `persons` table. + - For each person, create a corresponding record in `users` and `household_members`. + - Backfill the `household_id` in all existing resources with the ID of the default household. + - **Bootstrap Update**: Modify `db.py` so that a fresh database bootstrap (`db.create_schema`) calls the `create()` functions for the new repositories and *not* the old `persons` repository. + - **Indices/Constraints**: Add unique `households.slug`; composite indexes on `(household_id, id)` per table; foreign keys with `ON DELETE CASCADE` where appropriate. + - **Acceptance**: Fresh bootstrap creates all tables; migration script idempotently adds columns and backfills; existing tests still pass against default household. + +2. **[ ] Implement New Authentication System**: + - **Refactor `api/auth.py`**: Gut the existing cookie-based logic. Implement the new `/register`, `/login`, and `/google` endpoints. + - **Create `users/` package**: Add `models.py` and `repository.py` for the new `User` entity. + - **Update `api/deps.py`**: Replace `get_current_person` with a new `get_current_user` dependency that validates the JWT and returns the `User` model. + - **Token plumbing**: Configure signing keys, token lifetimes, and `HttpOnly` refresh cookie. Consider `Authorization: Bearer` for access tokens. + - **OpenAPI**: Update `api/openapi.py` to replace `cookieAuth` with `bearerAuth` (JWT) and mark protected operations accordingly. + - **Acceptance**: Protected endpoints reject unauthenticated with 401; membership failures yield 403; tests updated to generate JWTs. + +3. **[ ] Implement Household Scoping**: + - **Create `households/` package**: Add `models.py` and `repository.py` for `Household`, `HouseholdMember`, and `HouseholdInvitation`. + - **Implement `get_household_from_slug`** in `api/deps.py`. + - **Refactor `main.py`**: + - Create a new `APIRouter` for household-scoped routes, e.g., `household_router = APIRouter(prefix="/api/v1/households/{householdSlug}")`. + - Mount the existing routers (`recipes_api`, `meals_api`, etc.) onto this new `household_router` instead of the main `app`. + - **Update Repositories**: Modify all repository functions (e.g., `recipes.repository.get_all`, `meals.repository.create`) to accept a `household_id` and use it in the `WHERE` clause of every SQL query. + - **Update Routers**: Add the `get_household_from_slug` dependency to all household-scoped routes and pass the resulting `household_id` to the repository functions. + - **Acceptance**: The same queries as v1, when run under different household slugs and users, return isolated data sets; cross-household access yields 403. + +4. **[ ] Implement Household & Invitation Logic**: + - Create the `api/households.py` router and implement the endpoints for creating households, listing the user's households, and managing invitations. + - Add the logic for sending invitation emails (this may require a new utility/service for sending emails). + - **Acceptance**: Admins can invite by email; invite accept adds user to household; listing households shows membership with roles. + +5. **[ ] Update OpenAPI Specification**: + - Modify the script `scripts/export_openapi.py` to correctly generate the new specification, or manually update `openapi.json`. This is crucial for the frontend team. + - Replace cookie auth doc with JWT bearer auth. Keep RFC7807 components. Ensure shopping outward enum uses `home|coles|woolworths`. + - Add household path parameter and `403` responses where applicable. + - Ensure all array properties are present (even if empty) as in v1. + +6. **[ ] Refactor and Test**: + - Update `tests/` to reflect the new API structure and authentication. Tests will need to be updated to handle the `{householdSlug}` path parameter and provide a valid JWT. + - Manually test all API flows to ensure data is strictly isolated between households. + - **Cleanup**: Once all tests pass, remove the `persons/` package and any lingering references to it. + - Keep existing v1 behavior parity (ProblemDetails, Location headers on create, pagination envelopes, ingredient parsing behavior). Update/extend tests in: `tests/test_main.py`, `tests/test_v1.py`, `tests/test_shopping*.py`. + +--- + +## 5. Gaps vs Current Codebase (summary) + +What’s missing today (must be implemented in v2): +- JWT-based authentication (register/login/google) and `users` domain. Replace `persons` entirely. +- Household domain: `households`, `household_members`, `household_invitations` tables, repos, and APIs. +- Household scoping: path prefixes, membership checks, repository filtering by `household_id` across all data tables. +- OpenAPI security scheme update to JWT bearer; `403` responses for membership violations. + +What to preserve from v1: +- CamelCase response keys, non-null collection properties, `Page` envelope and cursor semantics, RFC7807 responses, `Location` header on create. +- Shopping outward storeName normalization ("home" instead of empty string). + +Nice-to-have carryovers (already partially implemented as per `tighten-api-spec.md`): +- Outward union models for shopping list items to reduce nullability (implemented in API without changing DB schema). + +--- + +## 6. Edge Cases and Error Semantics +- Unauthenticated → 401 (JWT), not 422 (replace the v1 cookie-missing 422/401 hybrid behavior). +- Not a member of household → 403 with ProblemDetails. +- Validation stays 400 with ProblemDetails (e.g., meal validation failures; recipe without ingredients; shopping invariants). +- `422` reserved for request model validation errors. + +--- + +## 7. Rollout & Migration Plan +- Phase A: Introduce `users` alongside `persons`; dual-write in migration; add JWT while keeping cookie for limited time (behind feature flag) if needed. +- Phase B: Add households and backfill default household; gate routes to household prefix with a migration shim for old clients if necessary. +- Phase C: Remove cookie auth and `persons` package; require JWT everywhere; finalize OpenAPI to v2. + +Data migration acceptance: +- All existing data appears under a default household and is accessible to the migrated user accounts. + +--- + +## 8. Acceptance Criteria (v2) +- Authentication: JWT bearer with register/login/google flows; refresh via HttpOnly cookie; `get_current_user` dependency replaced everywhere. +- Tenancy: All read/write queries filter by `household_id`; cross-household access returns 403; invitation flow works. +- API parity: v1 behavior preserved aside from auth/paths; ProblemDetails and pagination semantics intact; Location headers set on creates. +- OpenAPI: Security scheme is JWT; household path param present; arrays required and enums normalized; 400/404/422 standardized; 403 added on household routes. +