diff --git a/README.md b/README.md index ba6fbb4..1e0d548 100644 --- a/README.md +++ b/README.md @@ -1,132 +1,142 @@ -Meal planner backend +## Doof Backend (aka Munch Ease) 🍽️ + +FastAPI backend for collaborative meal planning, recipe wrangling, and grocery shopping. Household-scoped, JWT-secured, SQLite-fast. Bring your recipes, we’ll do the rest. + +## Why you’ll love it + +- πŸš€ Fast and modern API with FastAPI + Pydantic v2 +- 🏠 Household-scoped everything (recipes, meals, shopping) for clean multi-user isolation +- πŸ” JWT access tokens + HttpOnly refresh cookie (Argon2 password hashing) +- πŸ§ͺ 100% test-friendly: ephemeral SQLite, deterministic APIs, RFC7807 errors +- 🧠 Ingredient NLP parsing with product matching +- πŸ›’ One-click shopping lists: request meals or individual ingredients, dedup done for you +- 🧾 OpenAPI on tap for your frontend and SDKs +- πŸ”— Built-in recipe parsing from URLs (with BeautifulSoup + httpx) + +## Stack + +- Runtime: Python 3.11+ +- Web: FastAPI, Starlette +- Data models: Pydantic v2 (camelCase JSON via custom `ApiModel`) +- Database: SQLite (aiosqlite), schema bootstrapped in each `repository.py` +- Auth: Custom HMAC-SHA256 JWTs + Argon2 password hashing +- Parsing/Scraping: `ingredient-parser-nlp`, `beautifulsoup4`, `httpx` +- Tooling: ruff (lint+format), mypy (typecheck), pytest, pre-commit, uvicorn + +## Features at a glance + +- πŸ”‘ Auth: Register, login, refresh, logout; access bearer token + HttpOnly refresh cookie +- 🏑 Households: Create, list, invite members; accept invitations via shareable links +- 🧾 Recipes: Create, list, paginate, delete (hide) with actor attribution; parse-from-url helper +- πŸ§ͺ Ingredients: NLP parse one or many lines; product matching baked in +- 🍽️ Meals: Plan, get, update, delete, mark consumed; validate participants and recipes +- πŸ›οΈ Shopping: Request meals and ingredients, view current list, purchase to a list, fetch past lists +- 🩺 Health: `GET /healthz` returns a tiny β€œok” model for probes +- πŸ“œ Errors: RFC7807 Problem Details everywhere, with tidy camelCase payloads + +## Project structure + +- `main.py` β€” FastAPI app factory, routers, exception handlers, health, frontend proxy/static +- `settings.py` β€” Environment-driven runtime settings (no external deps) +- `security.py` β€” Minimal JWT utilities (HS256) + helpers +- `db.py` β€” aiosqlite connect + `create()` bootstraps all domain tables +- `api/` β€” HTTP surface (versioned under `/api/v1`) + - `auth.py` β€” register, login, refresh, logout + - `households.py` β€” create/list, members, invitations, scoped routes + - `ingredients.py` β€” household-scoped NLP parsing + - `recipes.py` β€” household-scoped list/get/create/delete, public utilities + - `meals.py` β€” household-scoped CRUD + mark consumed + - `shopping.py` β€” household-scoped current list, purchase, request/unrequest + - `openapi.py` β€” OpenAPI augmentation (cookie auth, problem+json) + - `deps.py` β€” DB/session, auth, household scoping, error helpers +- Domain packages (models + repository + helpers): + - `users/`, `households/`, `ingredients/`, `recipes/` (incl. `scraping.py`), `meals/`, `products/` (Coles/Woolworths helpers), `shopping/` +- `scripts/export_openapi.py` β€” writes `openapi.json` from the live app +- `tests/` β€” API and domain tests with fixtures and sample files ## Quickstart First time setup: + ```bash make install ``` -Run the development server: +Run the dev server: + ```bash make dev ``` Run tests: + ```bash make test ``` -Run all quality checks (lint, typecheck, test, format check, OpenAPI export): +Run all checks (lint, typecheck, tests, format check, OpenAPI export): + ```bash make all-checks ``` -## Structure +## Environment -- `main.py`: FastAPI app with all HTTP endpoints. -- `db.py`: aiosqlite connection + schema bootstrap across subpackages (calls each feature's `repository.create`). -- Domain packages with models and persistence: - - `products/` (models.py, repository.py, scrapers for Woolworths/Coles) - - `ingredients/` (models.py, repository.py) - - `recipes/` (models.py, repository.py, scraping.py) - - `meals/` (models.py, repository.py, service.py) - - `persons/` (models.py, repository.py) - - `shopping/` (models.py, repository.py) -- `tests/`: unit and API tests with sample HTTP fixtures. +These are read from the environment (see `settings.py`): -## Getting started +- `DOOF_DB` β€” SQLite file path (default `./data/doof.sqlite`) +- `DOOF_PROD` β€” `true/false` controls frontend proxy vs. static serving (default `false`) +- `FRONTEND_DEV_URL` β€” dev server to reverse-proxy in non-prod (default `http://localhost:8080/`) +- `DOOF_JWT_ISSUER`, `DOOF_JWT_AUDIENCE` β€” JWT claims +- `DOOF_JWT_ACCESS_TTL`, `DOOF_JWT_REFRESH_TTL` β€” TTLs in seconds (default 900/2592000) +- `DOOF_JWT_ACCESS_SECRET_B64`, `DOOF_JWT_REFRESH_SECRET_B64` β€” base64 secrets (use in prod!) -### Manual setup (alternative to make install) +Dev convenience: if secrets aren’t provided, deterministic dev secrets are used. Don’t ship those. + +## API surface (v1) + +- Base: `/api/v1` +- Auth: `/auth/register`, `/auth/login`, `/auth/refresh`, `/auth/logout` +- Households: `/households`, `/users/me/households`, `/households/{slug}/members`, `/households/{slug}/whoami`, invitations create/accept +- Ingredients: `/households/{slug}/ingredients/parse` (single or batch parsing) +- Recipes: `/households/{slug}/recipes` (list/paged, create), `/{id}` (get/delete), `/parse-from-url` +- Meals: `/households/{slug}/meals` (create/update/delete/get/upcoming/mark-consumed) +- Shopping: `/households/{slug}/shopping/current`, `/{listId}`, request/unrequest meals and ingredients, purchase lists +- Health: `/healthz` + +Errors are consistent Problem Details (`application/problem+json`). Models serialize in camelCase. + +## Make targets + +- `make install` β€” Create venv and install dependencies +- `make dev` β€” Run dev server (`uvicorn main:app --reload`) +- `make test` β€” Run tests (pytest -q) +- `make format` β€” Format with ruff +- `make lint` β€” Lint with ruff +- `make typecheck` β€” Type check with mypy +- `make openapi` β€” Export OpenAPI to `openapi.json` +- `make all-checks` β€” Lint + typecheck + tests + format check + OpenAPI export +- `make clean` β€” Remove venv and caches + +## Development notes + +- Schema bootstrap: `db.create(conn)` calls each feature’s `repository.create` to make tables +- Frontend integration: + - Dev: requests for non-`/api/*` are reverse-proxied to `FRONTEND_DEV_URL` + - Prod: static files served from `./front-dist` +- Security: Argon2 password hashing; HS256 JWTs signed with your secrets; refresh token stored as HttpOnly cookie +- DX niceties: camelCase JSON by default, strict validation, helpful error messages + +## OpenAPI + +Generate the spec file used by the frontend and CI: -Create and activate virtual environment: ```bash -python3 -m venv .venv -source .venv/bin/activate # On Windows: .venv\Scripts\activate +make openapi ``` -Install packages: -```bash -pip install -r ./requirements.txt -pip install -r ./dev-requirements.txt -``` +This writes `openapi.json` at the repo root. -Install pre-commit hooks: -```bash -pip install pre-commit -pre-commit install -``` +--- -### Running the application - -Run API (dev): -```bash -make dev -# or: uvicorn main:app --reload -``` - -Run tests: -```bash -make test -# or: pytest -q -``` - -### Available Make targets - -- `make install` - Create venv and install all dependencies -- `make dev` - Run development server -- `make test` - Run tests -- `make format` - Format code with ruff -- `make lint` - Lint code with ruff -- `make typecheck` - Type check with mypy -- `make openapi` - Export OpenAPI schema -- `make all-checks` - Run all quality checks -- `make clean` - Remove venv and caches - -## Tooling - -This repo includes baseline configs in `pyproject.toml`: -- ruff (format and lint) -- mypy (type check) - -Pre-commit hooks are configured to run: -- ruff (lint + format) -- mypy -- trailing whitespace fixer -- end-of-file fixer - -Quality checks (run locally): -```bash -make format # Format code -make lint # Lint code -make typecheck # Type check -make all-checks # Run all checks -``` - -## Environment variables - -Copy `.env.example` to `.env` and customize as needed: - -- DOOF_DB: Path to sqlite database (default: `./data/doof.sqlite`) -- DOOF_PROD: Set to `true` in production (default: `false`) -- FRONTEND_DEV_URL: Frontend dev server URL for reverse proxy (default: `http://localhost:8080/`) -- DOOF_PORT: Port the server listens on when containerized; align Dockerfile `EXPOSE` accordingly. - -## OpenAPI schema - -- Generate the schema artifact used by the frontend and CI checks: -``` -python scripts/export_openapi.py -``` - -This writes `openapi.json` to the repo root. Versioned endpoints live under `/api/v1`, legacy under `/api` (deprecated with `Deprecation` header). - -## Schema lint/diff (manual) - -Optionally, lint and compare schemas locally using Node tools: -``` -npx -y @stoplight/spectral-cli lint openapi.json -npx -y openapi-diff --fail-on-changed --fail-on-incompatible path/to/baseline.json openapi.json -``` - -Keep a `baseline.json` on release branches to detect breaking changes. +Built with love and leftovers. Hungry for issues and PRs. πŸ§‘β€πŸ³ diff --git a/backend-spec.md b/backend-spec.md deleted file mode 100644 index 3114ee4..0000000 --- a/backend-spec.md +++ /dev/null @@ -1,31 +0,0 @@ -# Backend Specification: Final Polish - -## 1. Current State & Objective - -**The multi-tenancy migration is functionally complete and successful.** The backend has been refactored to a robust, household-scoped system with JWT-based authentication. The legacy `persons` model has been purged, and data isolation is enforced. - -The objective is to complete the final remaining tasks to officially close out the project. - -## 2. Final Tasks - -This checklist represents all remaining work. - -- [x] **1. Implement "Copy Invite Link" API**: - - **Objective**: Modify the invitation creation logic to support a "copy link" UX on the frontend, instead of sending an email. - - **File**: `api/households.py` - - **Action**: The existing `POST /api/v1/households/{householdSlug}/invitations` endpoint should be modified. Instead of returning a simple `201 Created`, it must create the invitation token and return a JSON object containing the full, shareable URL. - - **Example Response**: - ```json - { - "invite_link": "https://app.example.com/invitations/accept?token=a1b2c3d4e5f6..." - } - ``` - -- [x] **2. Final Codebase Sweep**: - - **Objective**: Perform a final search for and remove any dead code, comments, or variables related to the old system. - - **Action**: Search the entire codebase for the following keywords: `legacy`, `old`, `previous`, `workaround`, `fallback`, `person`. - - **Outcome**: Any remaining artifacts from the migration are pruned, leaving the codebase in a clean, maintainable state for future development. - -- [x] **3. Mark Project as Complete**: - - **Objective**: Once the above tasks are done, this document is complete. - - **Action**: Check this box and archive this specification. diff --git a/tighten-api-spec.md b/tighten-api-spec.md deleted file mode 100644 index aea3549..0000000 --- a/tighten-api-spec.md +++ /dev/null @@ -1,113 +0,0 @@ -# Tighten Public API Nullability - -Make the external API more consistent and predictable by eliminating unnecessary nulls (nullable fields) in models and responses. This plan lists concrete, low-risk changes, their rationale, and exact files/lines to modify. Each task is checkable and includes verification steps. - -Date: 2025-10-21 - -## Principles - -- Prefer non-nullable types where domain requires a value (DB constraints, logic always sets it). -- Keep optional only when a field truly may be absent by design (e.g., hiddenBy, prevCursor when first page). -- Preserve backward compatibility where feasible. When changing response shapes, update tests and OpenAPI examples. -- Pydantic already excludes None on serialization in some places; we still tighten model types to improve OpenAPI and client SDKs. - -## Quick wins (low risk) - -- [x] Page.total is non-nullable with default 0 - - Why: Pagination always returns a number. Current `Optional[int]` leads to `null` in schema and potential nulls in responses. - - Change: in `common.py`, change `total: Optional[int]` to `total: int = Field(default=0, description="Total count")`. - - Verify: - - [ ] mypy/pyright/ruff pass. - - [ ] Tests for persons/recipes list remain green. - - [ ] OpenAPI shows `total` as `integer` (no anyOf null). - -- [x] Recipe.created_by_id non-nullable - - Why: DB enforces NOT NULL and creation flow always sets it. - - Change: in `recipes/models.py` set `created_by_id: int` (remove Optional). Keep `created_by: Optional[Person]` (hydrated field). - - Knock-on: `api/recipes.load_full_recipe` can drop the `if r.created_by_id is not None` guard. - - Verify: - - [ ] All recipe-related tests green. - - [ ] OpenAPI for Recipe shows `createdById` required. - -- [x] Product.raw_data excluded from public schema - - Why: Internal/testing helper currently typed as `Optional[dict]` -> visible as nullable in OpenAPI. - - Change: in `products/models.py` use a `PrivateAttr` (with a `raw_data` property) so it stays out of the schema without creating Input/Output variants. - - Verify: - - [ ] Product schema in OpenAPI does not include `rawData`. - - [ ] Tests referencing raw_data still pass (field remains available in code, excluded from schema/response). - -## Shopping models and endpoints - -`ShoppingListItem` currently represents two cases (ingredient request vs meal request), so several linking fields are nullable. We can reduce nulls in the public API by introducing outward-facing variants while keeping the DB model as-is. - -- [ ] Optional: Introduce discriminated union for API returns (medium change) - - Rationale: Return `oneOf` in OpenAPI with variant-specific required fields; eliminates irrelevant nullable properties for each variant. - - Approach (sketch): - - Define `ListIngredientItem` and `RequestedMealItem` pydantic models with a `kind` discriminator. - - Update `api/shopping.py` response models (CurrentShoppingList and PurchasedShoppingList) to use `Union[ListIngredientItem, RequestedMealItem]` for item arrays. - - Conversion helpers in `shopping` module to map from `ShoppingListItem` DB model to the outward union. - - Verify: - - [ ] Update v2 tests to accept the new shape while preserving field meanings. - - [ ] OpenAPI shows `oneOf` for shopping list items. - -- [ ] Tighten invariants without breaking shape (keep for now) - - Keep model but document invariants (only one of ingredient_id/meal_id required; recipe_id optional when meal request). Repository already validates; consider pydantic validators later. - -## Persons and Recipes listings - -- [ ] Ensure total is populated for Person and Recipe lists - - Already implemented in `api/persons.py` and `api/recipes.py` using repository `count_*` helpers. After making `Page.total` non-nullable, nothing else required. - -## Authentication dependency - -- [x] Provide strict non-null person dependency for protected endpoints - - Why: Many endpoints assume an authenticated user; typing as non-null simplifies signatures and docs. - - Change: - - Consolidated on a single dependency `cookie_person` (strict): `Cookie(..., alias="user_id")` and raises 401 if missing/unknown. - - Removed `require_cookie_person` and switched usages to `cookie_person`. - - Verify: - - [x] Endpoint signatures updated. - - [x] Unauthorized behavior covered by handlers; overall tests still pass. - -## File-by-file checklist (edits) - -- [x] `common.py` - - [x] Page.total -> `int = Field(default=0, ...)` - -- [x] `recipes/models.py` - - [x] `created_by_id: int` - -- [x] `api/recipes.py` - - [x] In `load_full_recipe`, set `r.created_by = await persons.get_by_id(conn, r.created_by_id)` unconditionally. - -- [x] `products/models.py` - - [x] `raw_data` moved to `PrivateAttr` with property; kept out of schema. - -- [x] `api/deps.py` - - [x] Added `require_cookie_person(...) -> persons.Person` that raises 401. - - [x] Updated protected endpoints to depend on `require_cookie_person`. - -- [ ] (Optional) Shopping API union types - - [x] Add outward-facing union models and mapping helpers. - - [x] Update `api/shopping.py` response models to use union. - -## Tests and validation - -- [x] Run format/lint/typecheck - - make format && make lint && make typecheck -- [x] Run tests - - make test -- [x] Export OpenAPI and inspect schema - - make openapi (confirm: Recipe.createdById required; Product has single schema; Page.total non-nullable; cookie param required on protected ops.) - -## Rollout notes - -- API change in OpenAPI (non-null total; createdById required). Client SDKs generated from the spec may need re-gen. Runtime remains compatible because server fills these fields. -- Even without the optional union refactor, the quick wins remove several unnecessary nulls. - -## Acceptance criteria - -- OpenAPI no longer marks Page.total and Recipe.createdById as nullable. -- Product.rawData does not appear in the public schema. -- All tests pass; no runtime regressions. -- Optional: Shopping list items use `oneOf` variants.