jacob/munch-ease-backend
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

db.create() initializes all v2 tables, including users and households, and tenant tables already include household_id columns and indices by default. No separate migration script is required for fresh databases used by tests.

Browse source
This commit is contained in:
jableader 2025-11-02 00:19:45 +11:00
parent 854b924a28
commit ca735865fc
22 changed files with 93 additions and 193 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
api/ingredients.py
View file

@ -1,5 +1,4 @@
from __future__ import annotations
from typing import List
from fastapi import APIRouter, Depends, Query
import aiosqlite
@ -11,16 +10,29 @@ from api.deps import get_db, get_household_from_slug
router = APIRouter(prefix="/households/{householdSlug}/ingredients", tags=["ingredients"])
@router.get("/parse", response_model=list[ingredients_mod.Ingredient], summary="Parse an ingredient line from a string")
@router.get(
"/parse",
response_model=ingredients_mod.Ingredient | list[ingredients_mod.Ingredient],
summary="Parse an ingredient line or lines from a string",
)
async def parse_ingredient(
lines: List[str] = Query(..., description="Multiple ingredient lines to parse"),
line: str | None = Query(None, description="Single ingredient line to parse"),
lines: list[str] | None = Query(None, description="Multiple ingredient lines to parse"),
household=Depends(get_household_from_slug),
conn: aiosqlite.Connection = Depends(get_db),
):
# NLP parse + best-effort product match
parsed = [ingredients_mod.parse_ingredient_from_nlp(line) for line in lines]
# Batch mode takes precedence if provided
if lines is not None:
parsed = [ingredients_mod.parse_ingredient_from_nlp(item_line) for item_line in lines]
matched = await ingredients_mod.match_existing_products(conn, parsed)
return matched
# Single line mode
if line is None:
from fastapi import HTTPException
raise HTTPException(status_code=422, detail="Query parameter 'line' or 'lines' is required")
parsed_one = ingredients_mod.parse_ingredient_from_nlp(line)
matched_one = await ingredients_mod.match_existing_products(conn, [parsed_one])
return matched_one[0]

22
backend-spec.md
View file

@ -1,3 +1,13 @@
## 0.2 API surface (historical)
- Persons API removed.
- Recipes (v1 historical paths):
- Added `api/shopping.py` with:
- GET `/api/v1/households/{householdSlug}/shopping/current` (scoped aggregate).
- GET `/api/v1/households/{householdSlug}/shopping/{listId}` returning purchased list + lookups scoped to household; cross-household returns 404.
Implementation policy updates (2025-11-01):
- Routers must use repository helpers for all persistence; direct `conn.execute(...)` calls in routers are prohibited, except for controlled PRAGMA/transaction management in `api/deps.py`.
- Recipes create now accepts a lean body (`RecipeCreate`) without internal IDs and sets `createdById` from the JWT user; delete uses a repository helper to set `date_hidden` and `hidden_by_id` atomically.
## 0.5 Validated v2 household behaviors (tests snapshot)
- GET `/api/v1/households/{householdSlug}/shopping/{listId}` returning purchased list + lookups scoped to household; cross-household returns 404.
- POST `/api/v1/households/{householdSlug}/meals/{mealId}/consumed` marks a meal consumed within the household; validates timezone on provided `consumedDate`; clears outstanding meal requests only within that household.
@ -187,18 +197,12 @@ Route surface lockdown:
## 4. Actionable Implementation Steps
1. **[~] Database Schema and Migration**:
1. **[~] Database Schema**:
- ✅ **Schema Definition**: Added new packages and tables:
- `users` with tables `User`, `LocalCredentials`, `OAuthCredentials` (see `users/repository.py`).
- `households` with tables `Household`, `HouseholdMember`, `HouseholdInvitation` (see `households/repository.py`).
- `db.create()` now initializes these tables alongside existing v1 tables.
- ✅ **Migration Logic**: Implemented `scripts/migration_to_households.py` which:
- Connect to the database (reusing logic from `db.py`).
- Calls `create()` for new repos to ensure tables exist.
- Adds a `household_id` column to tenant tables: `Recipe`, `Ingredient`, `Meal`, `MealParticipant`, `MealRecipe`, `ShoppingList`, `ShoppingListItem` (idempotent).
- Creates indices `idx_<table>_household_id` for all above tables.
- Creates a default household `{ name: "My Household", slug: "default" }` and backfills `household_id` with its ID for existing rows.
- Ports `Person` rows to `User` (email derived as `<name>@example.com`) and creates `HouseholdMember` links (role `admin`).
- ✅ **Bootstrap**: `db.create()` initializes all v2 tables, including `users` and `households`, and tenant tables already include `household_id` columns and indices by default. No separate migration script is required for fresh databases used by tests.
### Feature parity checklist (OpenAPI diffs vs master)
@ -212,7 +216,7 @@ Outstanding (tracked):
- None identified blocking parity for shopping/recipes needed by the frontend as of 2025-11-01. Re-check if any v1 product scrape/create endpoint needs re-exposure; current frontend uses household flows and parsing utilities.
- **Bootstrap Update**: Modify `db.py` so that a fresh database bootstrap (`db.create_schema`) calls the `create()` functions for the new repositories and *not* the old `persons` repository.
- ✅ **Indices/Constraints**: Enforced unique `Household.slug`; added `idx_*_household_id` indices; foreign keys added with `ON DELETE CASCADE` where applicable in new tables.
- ✅ **Acceptance (initial)**: Added `tests/test_migration_households.py` covering: new tables exist, `household_id` columns exist, default household created, and data porting from `Person` to `User` and `HouseholdMember`. Full test suite passes.
- ✅ **Acceptance (initial)**: Household tables and `household_id` columns are covered by repository DDL and exercised by v2 tests. Full test suite passes.
- Pending follow-ups for this step:
- (Done) Add composite indices `(household_id, id)` where high-cardinality pagination will benefit (Recipe, Meal).

140
scripts/migration_to_households.py
View file

@ -1,140 +0,0 @@
from __future__ import annotations
import asyncio
from typing import Optional
import aiosqlite
import db
from settings import settings
async def column_exists(conn: aiosqlite.Connection, table: str, column: str) -> bool:
async with conn.execute(f"PRAGMA table_info({table});") as cursor:
async for row in cursor:
if row[1] == column:
return True
return False
async def table_exists(conn: aiosqlite.Connection, table: str) -> bool:
async with conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name=?;", (table,)
) as c:
row = await c.fetchone()
return row is not None
async def add_column_if_missing(conn: aiosqlite.Connection, table: str, column_def: str) -> None:
# column_def like "household_id INTEGER"
col_name = column_def.split()[0]
if not await column_exists(conn, table, col_name):
await conn.execute(f"ALTER TABLE {table} ADD COLUMN {column_def};")
async def ensure_default_household(conn: aiosqlite.Connection) -> int:
# Create a default household and return its id, idempotently
await conn.execute(
"""
INSERT INTO Household (name, slug)
VALUES ('My Household', 'default')
ON CONFLICT(slug) DO NOTHING
"""
)
async with conn.execute("SELECT id FROM Household WHERE slug = 'default' LIMIT 1;") as c:
row = await c.fetchone()
assert row is not None
return int(row[0])
async def backfill_table_household_id(
conn: aiosqlite.Connection, table: str, default_household_id: int
) -> None:
# If any NULL household_id rows exist, backfill to default
await conn.execute(
f"""
UPDATE {table}
SET household_id = ?
WHERE household_id IS NULL
""",
(default_household_id,),
)
async def run_migration(conn: Optional[aiosqlite.Connection] = None):
owned = False
if conn is None:
conn = await db.connect(settings.database_path)
owned = True
try:
# Ensure new domain tables exist
from users import repository as users_db
from households import repository as households_db
await users_db.create(conn)
await households_db.create(conn)
# Add household_id columns to tenant tables
for table in [
"Recipe",
"Ingredient",
"Meal",
"MealParticipant",
"MealRecipe",
"ShoppingList",
"ShoppingListItem",
]:
await add_column_if_missing(conn, table, "household_id INTEGER")
# Backfill default household
default_hid = await ensure_default_household(conn)
for table in [
"Recipe",
"Ingredient",
"Meal",
"MealParticipant",
"MealRecipe",
"ShoppingList",
"ShoppingListItem",
]:
await backfill_table_household_id(conn, table, default_hid)
# Create indices on household_id for efficient scoping
for table in [
"Recipe",
"Ingredient",
"Meal",
"MealParticipant",
"MealRecipe",
"ShoppingList",
"ShoppingListItem",
]:
await conn.execute(
f"CREATE INDEX IF NOT EXISTS idx_{table.lower()}_household_id ON {table}(household_id);"
)
# Port persons -> users (idempotent) if legacy table exists
if await table_exists(conn, "Person"):
async with conn.execute("SELECT id, name FROM Person;") as cur:
async for pid, name in cur:
email = f"{str(name).lower()}@example.com"
display_name = str(name)
await conn.execute(
"INSERT OR IGNORE INTO User (id, email, display_name) VALUES (?, ?, ?);",
(int(pid), email, display_name),
)
# Ensure all users are members of default household (idempotent)
await conn.execute(
"INSERT OR IGNORE INTO HouseholdMember (user_id, household_id, role) SELECT id, ?, 'admin' FROM User;",
(default_hid,),
)
await conn.commit()
finally:
if owned:
await conn.close()
if __name__ == "__main__":
asyncio.run(run_migration())

4
tests/test_health_and_location_headers.py
View file

@ -17,10 +17,6 @@ class TestHealthAndLocationHeaders(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
# Ensure v2 household schema is present
from scripts.migration_to_households import run_migration
await run_migration(self.conn)
await test_data.create_test_data(self.conn)
reload_test_data()

2
tests/test_household_members_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestHouseholdMembersV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_ingredients_parse_api_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestIngredientsParseApiV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

60
tests/test_ingredients_parse_multiple_v2.py Normal file
View file

@ -0,0 +1,60 @@
import unittest
from fastapi.testclient import TestClient
import main
from db import connect, create
class TestIngredientsParseMultipleV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
async def override_get_db():
try:
yield self.conn
finally:
pass
main.app.dependency_overrides[main.get_db] = override_get_db
self.client = TestClient(main.app)
# Register user and create household
r = self.client.post(
"/api/v1/auth/register",
json={"email": "ing2@test.com", "password": "pw", "displayName": "Ing2"},
)
assert r.status_code == 200, r.text
token = r.json()["accessToken"]
self.headers = {"Authorization": f"Bearer {token}"}
r2 = self.client.post("/api/v1/households", headers=self.headers, json={"name": "H2"})
assert r2.status_code == 200, r2.text
self.slug = r2.json()["slug"]
async def asyncTearDown(self):
await self.conn.close()
main.app.dependency_overrides.clear()
def test_parse_multiple_lines_with_lines_param(self):
lines = [
"14oz milk powder",
"2 cups flour",
"1 tsp salt",
"egg",
]
r = self.client.get(
f"/api/v1/households/{self.slug}/ingredients/parse",
params=[("lines", line) for line in lines],
headers=self.headers,
)
assert r.status_code == 200, r.text
items = r.json()
assert isinstance(items, list)
assert len(items) == 4
# spot check
oz, cups, tsp, egg = items
assert float(oz["quantity"]) == 14.0 and oz["unit"] == "Ounce"
assert cups["name"].lower() == "flour"
assert tsp["unit"] == "Teaspoon"
assert egg["name"].lower() == "egg"

2
tests/test_meals_consumed_v2.py
View file

@ -4,14 +4,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestMealsConsumedV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_meals_household_v2.py
View file

@ -4,14 +4,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestMealsHouseholdV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_meals_write_v2.py
View file

@ -4,14 +4,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestMealsWriteV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_recipes_household_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestRecipesHouseholdV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_recipes_parse_from_url_integration_v2.py
View file

@ -3,7 +3,6 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
SAMPLE_URL = "https://www.allrecipes.com/recipe/262696/cheese-omelette/"
@ -15,7 +14,6 @@ class TestRecipesParseFromUrlIntegrationV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_recipes_parse_from_url_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestRecipesParseFromUrlV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_current_structure_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingCurrentStructureV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_household_v2.py
View file

@ -4,14 +4,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingHouseholdV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_list_by_id_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingListByIdV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_purchase_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingPurchaseV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_request_ingredient_dedupe_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingRequestIngredientDedupeV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_request_ingredient_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingRequestIngredientV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_request_meal_v2.py
View file

@ -4,14 +4,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingRequestMealV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tests/test_shopping_unrequest_ingredient_v2.py
View file

@ -3,14 +3,12 @@ from fastapi.testclient import TestClient
import main
from db import connect, create
from scripts.migration_to_households import run_migration
class TestShoppingUnrequestIngredientV2(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
self.conn = await connect(":memory:")
await create(self.conn)
await run_migration(self.conn)
async def override_get_db():
try:

2
tighten-api-spec.md
View file

@ -47,7 +47,7 @@ Date: 2025-10-21
- Update `api/shopping.py` response models (CurrentShoppingList and PurchasedShoppingList) to use `Union[ListIngredientItem, RequestedMealItem]` for item arrays.
- Conversion helpers in `shopping` module to map from `ShoppingListItem` DB model to the outward union.
- Verify:
- [ ] Update tests in `tests/test_shopping_api.py` to accept the new shape while preserving field meanings.
- [ ] Update v2 tests to accept the new shape while preserving field meanings.
- [ ] OpenAPI shows `oneOf` for shopping list items.
- [ ] Tighten invariants without breaking shape (keep for now)