munch-ease-backend/recipes/__init__.py

111 lines
3.8 KiB
Python
Raw Normal View History

2025-10-18 03:26:42 +00:00
import re
from typing import Optional
2024-05-19 11:22:30 +00:00
2025-10-18 03:26:42 +00:00
from ingredients import match_existing_products, parse_ingredient_from_nlp
2025-11-01 12:36:46 +00:00
from api.dtos import MemberRef
Squashed commit of the following: commit fcd005b8624023547f28b7b28e59e6099bcfc7d4 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 20:24:07 2025 +1100 Openapi tightening commit f93bd8f641d561052c7bd075bae321b4ff3b676d Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 19:03:52 2025 +1100 Removed refactor strategy doc commit 0c5a61092f522be0c47cbbe86917c8a7e4e2d339 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 17:48:33 2025 +1100 mypy & ruff checks commit 23d66d6b18984127e17c73c3063f6120385935e9 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 16:49:35 2025 +1100 Final removal of db.py files commit f454aed1ca9783cc558cc203f29a7fe31b62a975 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:42:31 2025 +1100 Finalise restructure, remove db.py files commit 7187f6dd89489521538791c6bdebb426514beb99 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:34:54 2025 +1100 commit 6fea227ae20d32b8eb1e7a4885006a620fcc7bb1 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:32:53 2025 +1100 commit 27415e7e02d89195ad514cb017a9dbbf84d7a5e4 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:31:10 2025 +1100 commit b773428033d855f9ad82005602e049c1a2e3c585 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:28:58 2025 +1100 commit 116592c95278d995f4c516e87f2cea43cf5b7735 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:25:21 2025 +1100 commit 03ec565faea088971968ee2f9bb83e2de16b21f3 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:21:29 2025 +1100 Plan
2025-10-19 09:24:23 +00:00
from recipes.models import Recipe as Recipe
from recipes.repository import (
compute_prev_cursor as compute_prev_cursor,
count_all as count_all,
count_by_name as count_by_name,
2025-11-01 02:58:46 +00:00
count_all_scoped as count_all_scoped,
count_by_name_scoped as count_by_name_scoped,
2025-10-18 03:26:42 +00:00
find_recipe_by_id as find_recipe_by_id,
2025-11-01 02:58:46 +00:00
find_recipe_by_id_scoped as find_recipe_by_id_scoped,
2025-10-18 03:26:42 +00:00
find_recipes_by_name as find_recipes_by_name,
Squashed commit of the following: commit 4189d9f824f681b480f797b109e963762eb22e9c Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:41:57 2025 +1100 Openapi complete commit bebf8c30cba0b85a889198fe44879614065a0c34 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:35:39 2025 +1100 Removed unversioned api commit dd9cc2eae75d66fceebe14c918c3ed8498376ee6 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:07:03 2025 +1100 Spec updates commit b993c4530688f79ea983278283f984e9d8e83860 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:01:50 2025 +1100 docs(spec): update doof-back-spec with v1 RFC7807 422, reusable Problem* responses, and shopping/current aliasing; tests passing; openapi.json refreshed commit 30bac7e57367b14ce924667a7955845de949d779 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:00:13 2025 +1100 openapi polish commit eb7f7f224f7085fa5b3fadc7716db0ebb7f47eb0 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:54:52 2025 +1100 OpenAPI reusable responses: Added components.responses for `Problem400`, `Problem404`, and `Problem422`; v1 routes reference these consistently. - Units enum: Exposed advisory enum in schema for `Ingredient.unit` using existing units list (no runtime enforcement). commit 037037e17d684a89b264f2406377970a0de7ec99 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:43:01 2025 +1100 Add `total` counts to v1 page responses for recipes/persons; push persons name filter into SQL for v1 when `q` is provided. commit 07e7735076aae8cbd04bb10f9aa324c1a3d80ae4 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:40:17 2025 +1100 Add parameter descriptions for `cursor`, `limit`, and `q` on v1 list endpoints; include example `Page` envelopes in 200 responses. commit e5bf9396b0fe870501d1b4712cba2555dd2ef9b1 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:36:27 2025 +1100 DB pagination commit 782315cd2a0c18cc50e4deaf28e7ec6e4b58c6e2 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:29:38 2025 +1100 camelcase tests commit b207c33e2844c00fc9e531fb9cf8c07a3f5cd543 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:24:57 2025 +1100 OpenAPI enrichment, Error responses commit dc84681ab743008e5cd8bec7f3ccb0d05b557518 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:16:39 2025 +1100 v1 tests: Added basic tests to assert `Page` envelopes and RFC7807 responses for v1 endpoints without affecting legacy tests. commit 1524b7a98ffe04af11c2731c8125ac9348751cff Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:14:51 2025 +1100 Pagination commit 92e91d7acf15c09b14bc76f7b16a7a47e65129ec Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:03:50 2025 +1100 Use middleware for naming case changes commit c68f964f9b8e3d8f8ef020661e747e0990459c81 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:00:09 2025 +1100 Openapi gen
2025-10-18 05:44:36 +00:00
find_recipes_by_name_paged as find_recipes_by_name_paged,
2025-11-01 02:58:46 +00:00
find_recipes_by_name_paged_scoped as find_recipes_by_name_paged_scoped,
2025-10-18 03:26:42 +00:00
get_all as get_all,
Squashed commit of the following: commit 4189d9f824f681b480f797b109e963762eb22e9c Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:41:57 2025 +1100 Openapi complete commit bebf8c30cba0b85a889198fe44879614065a0c34 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:35:39 2025 +1100 Removed unversioned api commit dd9cc2eae75d66fceebe14c918c3ed8498376ee6 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:07:03 2025 +1100 Spec updates commit b993c4530688f79ea983278283f984e9d8e83860 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:01:50 2025 +1100 docs(spec): update doof-back-spec with v1 RFC7807 422, reusable Problem* responses, and shopping/current aliasing; tests passing; openapi.json refreshed commit 30bac7e57367b14ce924667a7955845de949d779 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 16:00:13 2025 +1100 openapi polish commit eb7f7f224f7085fa5b3fadc7716db0ebb7f47eb0 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:54:52 2025 +1100 OpenAPI reusable responses: Added components.responses for `Problem400`, `Problem404`, and `Problem422`; v1 routes reference these consistently. - Units enum: Exposed advisory enum in schema for `Ingredient.unit` using existing units list (no runtime enforcement). commit 037037e17d684a89b264f2406377970a0de7ec99 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:43:01 2025 +1100 Add `total` counts to v1 page responses for recipes/persons; push persons name filter into SQL for v1 when `q` is provided. commit 07e7735076aae8cbd04bb10f9aa324c1a3d80ae4 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:40:17 2025 +1100 Add parameter descriptions for `cursor`, `limit`, and `q` on v1 list endpoints; include example `Page` envelopes in 200 responses. commit e5bf9396b0fe870501d1b4712cba2555dd2ef9b1 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:36:27 2025 +1100 DB pagination commit 782315cd2a0c18cc50e4deaf28e7ec6e4b58c6e2 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:29:38 2025 +1100 camelcase tests commit b207c33e2844c00fc9e531fb9cf8c07a3f5cd543 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:24:57 2025 +1100 OpenAPI enrichment, Error responses commit dc84681ab743008e5cd8bec7f3ccb0d05b557518 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:16:39 2025 +1100 v1 tests: Added basic tests to assert `Page` envelopes and RFC7807 responses for v1 endpoints without affecting legacy tests. commit 1524b7a98ffe04af11c2731c8125ac9348751cff Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:14:51 2025 +1100 Pagination commit 92e91d7acf15c09b14bc76f7b16a7a47e65129ec Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:03:50 2025 +1100 Use middleware for naming case changes commit c68f964f9b8e3d8f8ef020661e747e0990459c81 Author: jableader <jacobdunk@gmail.com> Date: Sat Oct 18 15:00:09 2025 +1100 Openapi gen
2025-10-18 05:44:36 +00:00
get_all_paged as get_all_paged,
2025-11-01 02:58:46 +00:00
get_all_paged_scoped as get_all_paged_scoped,
2025-10-18 03:26:42 +00:00
insert_recipe as insert_recipe,
2025-11-01 02:58:46 +00:00
insert_recipe_scoped as insert_recipe_scoped,
2025-10-18 03:26:42 +00:00
load_recipe_ingredients as load_recipe_ingredients,
row_to_recipe as row_to_recipe,
)
2025-11-04 07:31:13 +00:00
from recipes.scraping import (
scrape_recipe_ldata as _scrape_recipe_ldata,
scrape_recipe_ldata_from_html as _scrape_recipe_ldata_from_html,
)
2024-01-13 03:21:38 +00:00
2025-11-04 07:31:13 +00:00
async def parse_recipe(conn, created_by, url: str, log=None, dump_dir: str | None = None) -> Optional[Recipe]:
"""Parse a recipe from a URL. Returns None if parsing fails.
Accepts an optional log callable taking a single string argument; when provided,
the scraper will emit diagnostic messages useful for manual testing.
"""
ldata = await _scrape_recipe_ldata(url, log=log, dump_dir=dump_dir)
2024-05-19 11:22:30 +00:00
2024-01-17 07:21:16 +00:00
if ldata:
2024-05-19 11:22:30 +00:00
return await _get_recipe_from_ldata(conn, url, ldata, created_by)
2024-01-13 03:21:38 +00:00
return None
2024-05-19 11:22:30 +00:00
2025-10-18 03:26:42 +00:00
2025-11-04 07:31:13 +00:00
async def parse_recipe_from_html(conn, created_by, base_url: str, html: str, log=None) -> Optional[Recipe]:
"""Parse a recipe from raw HTML (offline). Returns None if parsing fails.
This mirrors parse_recipe() but uses already-downloaded HTML via the offline
scraper entrypoint. Useful for tests/assertions against saved snapshots.
"""
ldata = _scrape_recipe_ldata_from_html(html, base_url, log=log)
if ldata:
return await _get_recipe_from_ldata(conn, base_url, ldata, created_by)
return None
2024-05-19 11:22:30 +00:00
def find_yield(recipe_ldata: dict) -> int:
2025-10-18 03:26:42 +00:00
if "recipeYield" in recipe_ldata:
yield_vals = recipe_ldata["recipeYield"]
2024-05-19 11:22:30 +00:00
if not isinstance(yield_vals, list):
yield_vals = [yield_vals]
for val in yield_vals:
try:
return int(val)
except ValueError:
pass
for val in yield_vals:
2025-10-18 03:26:42 +00:00
match = re.match(r"(\d+)", val)
2024-05-19 11:22:30 +00:00
if match:
return int(match.group(1))
return 4
2025-10-18 03:26:42 +00:00
async def _get_recipe_from_ldata(conn, url: str, ldata: dict, created_by) -> Recipe:
2025-10-18 03:26:42 +00:00
ingredients = [
parse_ingredient_from_nlp(ingredient) for ingredient in ldata["recipeIngredient"]
]
2024-05-21 12:12:09 +00:00
ingredients = await match_existing_products(conn, ingredients)
2025-10-18 03:26:42 +00:00
name = ldata["name"] if "name" in ldata else url
images = ldata["image"] if "image" in ldata else []
2024-05-19 11:22:30 +00:00
serves = find_yield(ldata)
2025-10-18 03:26:42 +00:00
2024-01-17 06:53:27 +00:00
if isinstance(images, list) and len(images) > 0 and isinstance(images[0], dict):
2025-10-18 03:26:42 +00:00
images = [image["url"] for image in images]
2024-01-13 23:41:15 +00:00
if isinstance(images, dict):
2025-10-18 03:26:42 +00:00
images = [images["url"]]
2024-01-13 23:41:15 +00:00
if isinstance(images, str):
images = [images]
2025-11-01 12:36:46 +00:00
# Ensure created_by is a MemberRef (not a full User) to satisfy model typing
mref = (
MemberRef(id=created_by.id, display_name=created_by.display_name)
if created_by is not None
else None
)
2024-01-13 03:21:38 +00:00
return Recipe(
Squashed commit of the following: commit fcd005b8624023547f28b7b28e59e6099bcfc7d4 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 20:24:07 2025 +1100 Openapi tightening commit f93bd8f641d561052c7bd075bae321b4ff3b676d Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 19:03:52 2025 +1100 Removed refactor strategy doc commit 0c5a61092f522be0c47cbbe86917c8a7e4e2d339 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 17:48:33 2025 +1100 mypy & ruff checks commit 23d66d6b18984127e17c73c3063f6120385935e9 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 16:49:35 2025 +1100 Final removal of db.py files commit f454aed1ca9783cc558cc203f29a7fe31b62a975 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:42:31 2025 +1100 Finalise restructure, remove db.py files commit 7187f6dd89489521538791c6bdebb426514beb99 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:34:54 2025 +1100 commit 6fea227ae20d32b8eb1e7a4885006a620fcc7bb1 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:32:53 2025 +1100 commit 27415e7e02d89195ad514cb017a9dbbf84d7a5e4 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:31:10 2025 +1100 commit b773428033d855f9ad82005602e049c1a2e3c585 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:28:58 2025 +1100 commit 116592c95278d995f4c516e87f2cea43cf5b7735 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:25:21 2025 +1100 commit 03ec565faea088971968ee2f9bb83e2de16b21f3 Author: jableader <jacobdunk@gmail.com> Date: Sun Oct 19 15:21:29 2025 +1100 Plan
2025-10-19 09:24:23 +00:00
id=-1,
2024-01-13 08:44:07 +00:00
name=name,
2024-01-13 03:21:38 +00:00
link=url,
2024-05-19 11:22:30 +00:00
serves=serves,
2024-01-13 08:44:07 +00:00
image_urls=images,
2024-05-19 11:22:30 +00:00
ingredients=ingredients,
2025-11-01 12:36:46 +00:00
created_by=mref,
created_by_id=created_by.id if created_by is not None else -1,
2025-10-18 03:26:42 +00:00
)