diff --git a/refactor-project-strategy.md b/refactor-project-strategy.md new file mode 100644 index 0000000..2f07a75 --- /dev/null +++ b/refactor-project-strategy.md @@ -0,0 +1,163 @@ +# Doof Backend Refactor Strategy + +This document outlines a phased, low-risk plan to improve structure, readability, and Pythonic design. It also serves as a living progress log. Update checkboxes and the Progress Log as steps complete. + +## Goals +- Improve module boundaries and readability by splitting monolithic files and clarifying responsibilities. +- Strengthen API design (consistent response models, error handling, security, health checks). +- Reduce DB round-trips and add indexes where needed for performance. +- Centralize configuration and lifecycle concerns. +- Improve tests with fixtures and reduce duplication. + +## Guiding principles +- Small, incremental PR-sized steps; keep tests green. +- Preserve public API paths for v1 unless explicitly stated. +- Avoid new dependencies unless they clearly justify themselves. +- Add types, docstrings, and examples where they clarify intent. + +--- + +## Phase 0 — Scaffolding and safety rails +- [x] Create this strategy document and track progress +- [x] Add central settings module (env-based, no new deps) + - [x] Add a tiny /healthz endpoint plan (defer code to Phase 1) + - [x] Decide on router file layout and migration approach + +Acceptance criteria +- A settings module exists and can be imported without side effects +- Document tracks status and provides an actionable, sequenced plan + +--- + +## Phase 1 — API structure and lifecycle +- [ ] Extract routers by feature + - [ ] api/recipes.py + - [ ] api/meals.py + - [ ] api/persons.py + - [ ] api/shopping.py + - [ ] api/auth.py +- [ ] Wire routers in main with minimal app code +- [ ] Move dev reverse proxy setup into a lifespan handler and ensure httpx client is closed +- [ ] Add /healthz endpoint (simple JSON: {"status": "ok"}) + +Acceptance criteria +- main.py primarily wires app, routers, settings, and lifespan +- Reverse proxy cleaned up; no stray global clients +- Health check available and excluded from auth/security + +--- + +## Phase 2 — API design and OpenAPI +- [ ] Ensure response_model is set on all endpoints that return models +- [ ] Normalize inconsistent routes (remove trailing slash on POST /shopping/) +- [ ] Replace MealIdWrapper body with path param for requesting meals: POST /shopping/current/meals/{mealId} +- [ ] Add 201 Created + Location on create endpoints (recipes, meals, persons) +- [ ] Define cookie-based security scheme in OpenAPI and apply to secured routes +- [ ] Keep reusable ProblemDetails responses; ensure 4xx schemas reference it consistently + +Acceptance criteria +- openapi.json shows correct schemas and security for affected routes +- Clients can generate SDKs without manual fixes + +--- + +## Phase 3 — Data access and performance +- [ ] Centralize transaction scoping per-request (middleware or dependency) and remove scattered conn.commit() in handlers +- [ ] Add DB PRAGMAs on connect (WAL, foreign_keys=ON) +- [ ] Batch-load related data to avoid N+1 (recipes/ingredients, meals/participants) +- [ ] Add indexes for common filters/joins + - [ ] ingredients.recipe_id + - [ ] ingredients.meal_id + - [ ] meal_participants.meal_id, role + - [ ] meal_recipes.meal_id + - [ ] recipes.date_hidden + - [ ] persons.name (for LIKE queries) + +Acceptance criteria +- Noticeable reduction in query count for list/detail endpoints (verified by logging/sqlite trace) +- No functional regressions; tests still green + +--- + +## Phase 4 — Modeling and validation +- [ ] Move inline API request/response models out of main.py into feature modules + - [ ] ProductUrl + - [ ] CurrentShoppingList + - [ ] PurchasedShoppingList + - [ ] LoginBody +- [ ] Extract validation (e.g., validate_meal) into a service layer for reuse +- [ ] Add enums/constants for participant roles + +Acceptance criteria +- Cleaner main.py; feature modules own their request/response contracts +- Validation logic reusable outside HTTP layer + +--- + +## Phase 5 — Testing and fixtures +- [ ] Introduce pytest fixtures for common setup (DB, test data, auth) +- [ ] Remove duplication in tests and centralize helpers +- [ ] Add tests for new health endpoint and 201 Location headers + +Acceptance criteria +- Test suite readability improved; repeated setup minimized +- Coverage for new behaviors added + +--- + +## Stretch items +- [ ] Structured logging with request IDs +- [ ] A background job or cron-friendly script for cleanup tasks +- [ ] Optional migration to pydantic-settings if desired later + +--- + +## Concrete file plan (proposed) +- main.py (app wiring only) +- settings.py (centralized configuration) +- api/ + - __init__.py + - recipes.py, meals.py, persons.py, shopping.py, auth.py (routers) +- domain/ + - recipes/models.py, repo.py, service.py + - meals/models.py, repo.py, service.py + - ingredients/models.py, repo.py, service.py + - persons/models.py, repo.py, service.py + - shopping/models.py, repo.py, service.py + +Note: We can adopt this structure gradually without moving DB code immediately; start with routers, then iterate. + +--- + +## Risks and mitigations +- Router extraction could break imports + - Mitigation: introduce api package and re-export symbols as needed during transition +- OpenAPI changes can affect clients + - Mitigation: keep v1 paths stable; document changes like 201 + Location +- N+1 fixes may alter behavior subtly + - Mitigation: add dedicated tests for list/detail payloads + +--- + +## Progress log +- 2025-10-18: Created strategy document and added settings.py (not yet wired) +- 2025-10-18: Finalized router layout and health endpoint plan — Phase 0 complete + +--- + +## Next actions +Begin Phase 1 work in small steps: + - Create api package and extract the first router (e.g., recipes) without logic changes + - Wire the router in main.py and run tests + - Prepare lifespan handler for dev reverse proxy, but keep behavior identical + +### Health endpoint plan (Phase 1 target) +- Path: GET /healthz +- Response: {"status": "ok"} +- No DB access; fast and safe + +### Router extraction migration notes +- Step 1: Create api package and empty routers with existing path/operation_id/signatures (no logic change) +- Step 2: Move handlers from main.py into respective files, fix imports +- Step 3: In main.py, include_router for each; keep response_model and decorator metadata intact +- Step 4: Run tests and adjust import paths \ No newline at end of file