# Doof Backend Refactor Strategy This document outlines a phased, low-risk plan to improve structure, readability, and Pythonic design. It also serves as a living progress log. Update checkboxes and the Progress Log as steps complete. ## Goals - Improve module boundaries and readability by splitting monolithic files and clarifying responsibilities. - Strengthen API design (consistent response models, error handling, security, health checks). - Reduce DB round-trips and add indexes where needed for performance. - Centralize configuration and lifecycle concerns. - Improve tests with fixtures and reduce duplication. ## Guiding principles - Small, incremental PR-sized steps; keep tests green. - Preserve public API paths for v1 unless explicitly stated. - Avoid new dependencies unless they clearly justify themselves. - Add types, docstrings, and examples where they clarify intent. --- ## Phase 0 — Scaffolding and safety rails - [x] Create this strategy document and track progress - [x] Add central settings module (env-based, no new deps) - [x] Add a tiny /healthz endpoint plan (defer code to Phase 1) - [x] Decide on router file layout and migration approach Acceptance criteria - A settings module exists and can be imported without side effects - Document tracks status and provides an actionable, sequenced plan --- ## Phase 1 — API structure and lifecycle - [ ] Extract routers by feature - [x] api/recipes.py - [x] api/meals.py - [x] api/persons.py - [x] api/shopping.py - [x] api/auth.py - [x] Wire routers in main with minimal app code - [x] Move dev reverse proxy setup into a lifespan handler and ensure httpx client is closed - [x] Add /healthz endpoint (simple JSON: {"status": "ok"}) - [x] Create api/deps module for get_db, cookie_person, and error_response - [x] Use settings.py (DOOF_DB) for DB path in main and deps Acceptance criteria - main.py primarily wires app, routers, settings, and lifespan - Reverse proxy cleaned up; no stray global clients - Health check available and excluded from auth/security Status: Complete Notes - Lifespan handler initializes a dev httpx.AsyncClient using settings.frontend_dev_url and stores it on app.state; it is closed on shutdown - Removed legacy inlined endpoints/helpers from main.py after extraction - Kept product creation endpoint in main pending Phase 4 move - Adjusted route signatures to use required Request and proper parameter ordering to satisfy FastAPI and Pydantic - Removed duplicated placeholder endpoints from api/shopping.py that were left from scaffolding - Added temporary back-compat shims in main.py (get_duplicates, validate_meal) forwarding to api.meals to keep tests passing; plan to remove in Phase 4 when tests are updated to import from feature modules --- ## Phase 2 — API design and OpenAPI - [x] Ensure response_model is set on all endpoints that return models - [x] Normalize inconsistent routes (remove trailing slash on POST /shopping/) — N/A in code; no stray trailing slash routes - [x] Replace MealIdWrapper body with path param for requesting meals: POST /shopping/current/meals/{mealId} - Decision: Kept existing v1 route POST /shopping/current/meals/me with body wrapper to avoid breaking tests; path-param variant deferred to v2 - [x] Add Location header on create endpoints (recipes, meals, persons, products) - Decision: Retained 200 response codes for v1 compatibility (tests expect 200); 201 can be adopted in v2 - [x] Define cookie-based security scheme in OpenAPI and apply to secured routes (by operationId) - [x] Keep reusable ProblemDetails responses; ensure 4xx schemas reference it consistently Acceptance criteria - openapi.json shows correct schemas, cookieAuth security for protected operations, and references to reusable ProblemDetails responses — Met - Clients can generate SDKs without manual fixes — Improved (consistent models and responses) --- ## Phase 3 — Data access and performance - [ ] Centralize transaction scoping per-request (middleware or dependency) and remove scattered conn.commit() in handlers - [ ] Add DB PRAGMAs on connect (WAL, foreign_keys=ON) - [ ] Batch-load related data to avoid N+1 (recipes/ingredients, meals/participants) - [ ] Add indexes for common filters/joins - [ ] ingredients.recipe_id - [ ] ingredients.meal_id - [ ] meal_participants.meal_id, role - [ ] meal_recipes.meal_id - [ ] recipes.date_hidden - [ ] persons.name (for LIKE queries) Acceptance criteria - Noticeable reduction in query count for list/detail endpoints (verified by logging/sqlite trace) - No functional regressions; tests still green --- ## Phase 4 — Modeling and validation - [ ] Move inline API request/response models out of main.py into feature modules - [ ] ProductUrl - [ ] CurrentShoppingList - [ ] PurchasedShoppingList - [ ] LoginBody - [ ] Extract validation (e.g., validate_meal) into a service layer for reuse - [ ] Add enums/constants for participant roles Acceptance criteria - Cleaner main.py; feature modules own their request/response contracts - Validation logic reusable outside HTTP layer --- ## Phase 5 — Testing and fixtures - [ ] Introduce pytest fixtures for common setup (DB, test data, auth) - [ ] Remove duplication in tests and centralize helpers - [ ] Add tests for new health endpoint and 201 Location headers Acceptance criteria - Test suite readability improved; repeated setup minimized - Coverage for new behaviors added --- ## Stretch items - [ ] Structured logging with request IDs - [ ] A background job or cron-friendly script for cleanup tasks - [ ] Optional migration to pydantic-settings if desired later --- ## Concrete file plan (proposed) - main.py (app wiring only) - settings.py (centralized configuration) - api/ - __init__.py - recipes.py, meals.py, persons.py, shopping.py, auth.py (routers) - domain/ - recipes/models.py, repo.py, service.py - meals/models.py, repo.py, service.py - ingredients/models.py, repo.py, service.py - persons/models.py, repo.py, service.py - shopping/models.py, repo.py, service.py Note: We can adopt this structure gradually without moving DB code immediately; start with routers, then iterate. --- ## Risks and mitigations - Router extraction could break imports - Mitigation: introduce api package and re-export symbols as needed during transition - OpenAPI changes can affect clients - Mitigation: keep v1 paths stable; document changes like 201 + Location - N+1 fixes may alter behavior subtly - Mitigation: add dedicated tests for list/detail payloads --- ## Progress log - 2025-10-18: Created strategy document and added settings.py (not yet wired) - 2025-10-18: Finalized router layout and health endpoint plan — Phase 0 complete - 2025-10-18: Created api package and scaffolded routers (recipes, meals, persons, shopping, auth) with placeholders - 2025-10-18: Extracted recipes routes to api/recipes.py and wired router; added /healthz - 2025-10-18: Extracted persons routes to api/persons.py and wired router - 2025-10-18: Extracted meals, shopping, and auth routes; created api/deps and switched DB path to settings - 2025-10-18: Moved dev reverse proxy to app lifespan; removed dead code from main.py; deduplicated models; tests all passing - 2025-10-18: Fixed FastAPI startup errors by normalizing Request usage/order; removed duplicate placeholder routes; added main.py shims for get_duplicates/validate_meal; full test suite green - 2025-10-18: Phase 2 complete — Added cookieAuth security to OpenAPI and annotated protected endpoints; normalized response_model across handlers; added Location headers on create endpoints while keeping 200 status for v1 compatibility; documented ProblemDetails responses in OpenAPI; regenerated openapi.json; full test suite still green --- ## Next actions - Phase 3: Plan DB PRAGMAs and indexes; add transaction scoping per request - Phase 3: Plan DB PRAGMAs and indexes; add transaction scoping per request - Phase 5: Add fixtures for DB/auth and tests for health + 201 Location - Phase 4: Move remaining request/response models and helpers (ProductUrl, CurrentShoppingList, PurchasedShoppingList, LoginBody, validate_meal/get_duplicates) fully into feature modules and update tests to import from there; then remove back-compat shims from main.py Follow-ups (v2 candidates) - Adopt 201 Created for create endpoints and adjust tests/clients - Switch POST /shopping/current/meals/me (body) to POST /shopping/current/meals/{mealId} (path param) ### Health endpoint plan (Phase 1 target) - Path: GET /healthz - Response: {"status": "ok"} - No DB access; fast and safe ### Router extraction migration notes - Step 1: Create api package and empty routers with existing path/operation_id/signatures (no logic change) - Step 2: Move handlers from main.py into respective files, fix imports - Step 3: In main.py, include_router for each; keep response_model and decorator metadata intact - Step 4: Run tests and adjust import paths