munch-ease-backend/refactor-project-strategy.md

Doof Backend Refactor Strategy

This document outlines a phased, low-risk plan to improve structure, readability, and Pythonic design. It also serves as a living progress log. Update checkboxes and the Progress Log as steps complete.

Goals

• Improve module boundaries and readability by splitting monolithic files and clarifying responsibilities.
• Strengthen API design (consistent response models, error handling, security, health checks).
• Reduce DB round-trips and add indexes where needed for performance.
• Centralize configuration and lifecycle concerns.
• Improve tests with fixtures and reduce duplication.

Guiding principles

• Small, incremental PR-sized steps; keep tests green.
• Preserve public API paths for v1 unless explicitly stated.
• Avoid new dependencies unless they clearly justify themselves.
• Add types, docstrings, and examples where they clarify intent.

---

Phase 0 — Scaffolding and safety rails

• Create this strategy document and track progress
• Add central settings module (env-based, no new deps)
• Add a tiny /healthz endpoint plan (defer code to Phase 1)
• Decide on router file layout and migration approach

Acceptance criteria

• A settings module exists and can be imported without side effects
• Document tracks status and provides an actionable, sequenced plan

---

Phase 1 — API structure and lifecycle

• Extract routers by feature
  • api/recipes.py
  • api/meals.py
  • api/persons.py
  • api/shopping.py
  • api/auth.py
• Wire routers in main with minimal app code
• Move dev reverse proxy setup into a lifespan handler and ensure httpx client is closed
• Add /healthz endpoint (simple JSON: {"status": "ok"})

Acceptance criteria

• main.py primarily wires app, routers, settings, and lifespan
• Reverse proxy cleaned up; no stray global clients
• Health check available and excluded from auth/security

---

Phase 2 — API design and OpenAPI

• Ensure response_model is set on all endpoints that return models
• Normalize inconsistent routes (remove trailing slash on POST /shopping/)
• Replace MealIdWrapper body with path param for requesting meals: POST /shopping/current/meals/{mealId}
• Add 201 Created + Location on create endpoints (recipes, meals, persons)
• Define cookie-based security scheme in OpenAPI and apply to secured routes
• Keep reusable ProblemDetails responses; ensure 4xx schemas reference it consistently

Acceptance criteria

• openapi.json shows correct schemas and security for affected routes
• Clients can generate SDKs without manual fixes

---

Phase 3 — Data access and performance

• Centralize transaction scoping per-request (middleware or dependency) and remove scattered conn.commit() in handlers
• Add DB PRAGMAs on connect (WAL, foreign_keys=ON)
• Batch-load related data to avoid N+1 (recipes/ingredients, meals/participants)
• Add indexes for common filters/joins
  • ingredients.recipe_id
  • ingredients.meal_id
  • meal_participants.meal_id, role
  • meal_recipes.meal_id
  • recipes.date_hidden
  • persons.name (for LIKE queries)

Acceptance criteria

• Noticeable reduction in query count for list/detail endpoints (verified by logging/sqlite trace)
• No functional regressions; tests still green

---

Phase 4 — Modeling and validation

• Move inline API request/response models out of main.py into feature modules
  • ProductUrl
  • CurrentShoppingList
  • PurchasedShoppingList
  • LoginBody
• Extract validation (e.g., validate_meal) into a service layer for reuse
• Add enums/constants for participant roles

Acceptance criteria

• Cleaner main.py; feature modules own their request/response contracts
• Validation logic reusable outside HTTP layer

---

Phase 5 — Testing and fixtures

• Introduce pytest fixtures for common setup (DB, test data, auth)
• Remove duplication in tests and centralize helpers
• Add tests for new health endpoint and 201 Location headers

Acceptance criteria

• Test suite readability improved; repeated setup minimized
• Coverage for new behaviors added

---

Stretch items

• Structured logging with request IDs
• A background job or cron-friendly script for cleanup tasks
• Optional migration to pydantic-settings if desired later

---

Concrete file plan (proposed)

• main.py (app wiring only)
• settings.py (centralized configuration)
• api/
  • init.py
  • recipes.py, meals.py, persons.py, shopping.py, auth.py (routers)
• domain/
  • recipes/models.py, repo.py, service.py
  • meals/models.py, repo.py, service.py
  • ingredients/models.py, repo.py, service.py
  • persons/models.py, repo.py, service.py
  • shopping/models.py, repo.py, service.py

Note: We can adopt this structure gradually without moving DB code immediately; start with routers, then iterate.

---

Risks and mitigations

• Router extraction could break imports
  • Mitigation: introduce api package and re-export symbols as needed during transition
• OpenAPI changes can affect clients
  • Mitigation: keep v1 paths stable; document changes like 201 + Location
• N+1 fixes may alter behavior subtly
  • Mitigation: add dedicated tests for list/detail payloads

---

Progress log

• 2025-10-18: Created strategy document and added settings.py (not yet wired)
• 2025-10-18: Finalized router layout and health endpoint plan — Phase 0 complete
• 2025-10-18: Created api package and scaffolded routers (recipes, meals, persons, shopping, auth) with placeholders

---

Next actions

Begin Phase 1 work in small steps:

• Create api package and extract the first router (e.g., recipes) without logic changes
• Wire the router in main.py and run tests
• Prepare lifespan handler for dev reverse proxy, but keep behavior identical

Health endpoint plan (Phase 1 target)

• Path: GET /healthz
• Response: {"status": "ok"}
• No DB access; fast and safe

Router extraction migration notes

• Step 1: Create api package and empty routers with existing path/operation_id/signatures (no logic change)
• Step 2: Move handlers from main.py into respective files, fix imports
• Step 3: In main.py, include_router for each; keep response_model and decorator metadata intact
• Step 4: Run tests and adjust import paths