munch-ease-backend/refactor-project-strategy.md

# Doof Backend Refactor Strategy

This document outlines a phased, low-risk plan to improve structure, readability, and Pythonic design. It also serves as a living progress log. Update checkboxes and the Progress Log as steps complete.

## Goals
- Improve module boundaries and readability by splitting monolithic files and clarifying responsibilities.
- Strengthen API design (consistent response models, error handling, security, health checks).
- Reduce DB round-trips and add indexes where needed for performance.
- Centralize configuration and lifecycle concerns.
- Improve tests with fixtures and reduce duplication.

## Guiding principles
- Small, incremental PR-sized steps; keep tests green.
- Preserve public API paths for v1 unless explicitly stated.
- Avoid new dependencies unless they clearly justify themselves.
- Add types, docstrings, and examples where they clarify intent.

---

## Phase 0 — Scaffolding and safety rails
- [x] Create this strategy document and track progress
- [x] Add central settings module (env-based, no new deps)
 - [x] Add a tiny /healthz endpoint plan (defer code to Phase 1)
 - [x] Decide on router file layout and migration approach

Acceptance criteria
- A settings module exists and can be imported without side effects
- Document tracks status and provides an actionable, sequenced plan

---

## Phase 1 — API structure and lifecycle
- [ ] Extract routers by feature
  - [x] api/recipes.py
  - [ ] api/meals.py
  - [x] api/persons.py
  - [ ] api/shopping.py
  - [ ] api/auth.py
- [ ] Wire routers in main with minimal app code
- [ ] Move dev reverse proxy setup into a lifespan handler and ensure httpx client is closed
- [x] Add /healthz endpoint (simple JSON: {"status": "ok"})

Acceptance criteria
- main.py primarily wires app, routers, settings, and lifespan
- Reverse proxy cleaned up; no stray global clients
- Health check available and excluded from auth/security

---

## Phase 2 — API design and OpenAPI
- [ ] Ensure response_model is set on all endpoints that return models
- [ ] Normalize inconsistent routes (remove trailing slash on POST /shopping/)
- [ ] Replace MealIdWrapper body with path param for requesting meals: POST /shopping/current/meals/{mealId}
- [ ] Add 201 Created + Location on create endpoints (recipes, meals, persons)
- [ ] Define cookie-based security scheme in OpenAPI and apply to secured routes
- [ ] Keep reusable ProblemDetails responses; ensure 4xx schemas reference it consistently

Acceptance criteria
- openapi.json shows correct schemas and security for affected routes
- Clients can generate SDKs without manual fixes

---

## Phase 3 — Data access and performance
- [ ] Centralize transaction scoping per-request (middleware or dependency) and remove scattered conn.commit() in handlers
- [ ] Add DB PRAGMAs on connect (WAL, foreign_keys=ON)
- [ ] Batch-load related data to avoid N+1 (recipes/ingredients, meals/participants)
- [ ] Add indexes for common filters/joins
  - [ ] ingredients.recipe_id
  - [ ] ingredients.meal_id
  - [ ] meal_participants.meal_id, role
  - [ ] meal_recipes.meal_id
  - [ ] recipes.date_hidden
  - [ ] persons.name (for LIKE queries)

Acceptance criteria
- Noticeable reduction in query count for list/detail endpoints (verified by logging/sqlite trace)
- No functional regressions; tests still green

---

## Phase 4 — Modeling and validation
- [ ] Move inline API request/response models out of main.py into feature modules
  - [ ] ProductUrl
  - [ ] CurrentShoppingList
  - [ ] PurchasedShoppingList
  - [ ] LoginBody
- [ ] Extract validation (e.g., validate_meal) into a service layer for reuse
- [ ] Add enums/constants for participant roles

Acceptance criteria
- Cleaner main.py; feature modules own their request/response contracts
- Validation logic reusable outside HTTP layer

---

## Phase 5 — Testing and fixtures
- [ ] Introduce pytest fixtures for common setup (DB, test data, auth)
- [ ] Remove duplication in tests and centralize helpers
- [ ] Add tests for new health endpoint and 201 Location headers

Acceptance criteria
- Test suite readability improved; repeated setup minimized
- Coverage for new behaviors added

---

## Stretch items
- [ ] Structured logging with request IDs
- [ ] A background job or cron-friendly script for cleanup tasks
- [ ] Optional migration to pydantic-settings if desired later

---

## Concrete file plan (proposed)
- main.py (app wiring only)
- settings.py (centralized configuration)
- api/
  - __init__.py
  - recipes.py, meals.py, persons.py, shopping.py, auth.py (routers)
- domain/
  - recipes/models.py, repo.py, service.py
  - meals/models.py, repo.py, service.py
  - ingredients/models.py, repo.py, service.py
  - persons/models.py, repo.py, service.py
  - shopping/models.py, repo.py, service.py

Note: We can adopt this structure gradually without moving DB code immediately; start with routers, then iterate.

---

## Risks and mitigations
- Router extraction could break imports
  - Mitigation: introduce api package and re-export symbols as needed during transition
- OpenAPI changes can affect clients
  - Mitigation: keep v1 paths stable; document changes like 201 + Location
- N+1 fixes may alter behavior subtly
  - Mitigation: add dedicated tests for list/detail payloads

---

## Progress log
- 2025-10-18: Created strategy document and added settings.py (not yet wired)
- 2025-10-18: Finalized router layout and health endpoint plan — Phase 0 complete
- 2025-10-18: Created api package and scaffolded routers (recipes, meals, persons, shopping, auth) with placeholders
- 2025-10-18: Extracted recipes routes to api/recipes.py and wired router; added /healthz
- 2025-10-18: Extracted persons routes to api/persons.py and wired router

---

## Next actions
Begin Phase 1 work in small steps:
 - Create api package and extract the first router (e.g., recipes) without logic changes
 - Wire the router in main.py and run tests
 - Prepare lifespan handler for dev reverse proxy, but keep behavior identical

### Health endpoint plan (Phase 1 target)
- Path: GET /healthz
- Response: {"status": "ok"}
- No DB access; fast and safe

### Router extraction migration notes
- Step 1: Create api package and empty routers with existing path/operation_id/signatures (no logic change)
- Step 2: Move handlers from main.py into respective files, fix imports
- Step 3: In main.py, include_router for each; keep response_model and decorator metadata intact
- Step 4: Run tests and adjust import paths