food/research.md

Research: How the Food app works today

Snapshot taken 2026-06-22, reading the code on master (HEAD 96d9c4b). Purpose: establish ground truth before making changes. Where the shipped docs (Meal Planning System.md) disagree with the code, the code is treated as authoritative and the discrepancy is called out.

---

1. What this is

A single-user Django app for Tom: pantry inventory + "meta-recipes" (cooking templates with swappable ingredient slots) + a smart shopping list + a cook log. Two front doors:

• REST API (/api/) — token auth, built for Caine (the Matrix assistant).
• HTMX web app (/app/) — server-rendered HTML fragments, no auth.

Plus Django admin (/admin/) as the full-CRUD fallback.

There is one Django app, kitchen, holding everything. ~1000 lines of Python total. No Celery, no JS build, no separate frontend — it's deliberately small.

2. Layout

food_project/        Django project config
  settings.py        single settings file, DEBUG=True (see §8)
  urls.py            routes /admin/, /api/, /api-auth/, /app/
kitchen/
  models.py          all 11 models
  admin.py           admin registrations + inlines
  serializers.py     DRF serializers (read + nested write)
  views.py           DRF viewsets + the "smart" API endpoints (JSON)
  urls.py            /api/ router + custom endpoints
  views_htmx.py      HTML-fragment views for the web app
  urls_htmx.py       /app/ routes
  templates/kitchen/ base.html + 4 pages + 5 partials
  static/kitchen/    htmx.min.js (vendored)
  management/commands/seed.py   initial data loader
  migrations/        0001_initial, 0002_add_rating_and_preferences
deploy/food.service  systemd unit (gunicorn, runs as user `openclaw`)
db.sqlite3.backup-…  committed snapshot = seed state only (see §9)
cookbooks/recipes_extracted.json   75 scraped Roasting Tin recipes (data only)

Note the README/design-doc say the project lives at /var/lib/food/ in production. This working copy is /home/tom/srcs/food. The DB (db.sqlite3), venv/, and staticfiles/ are gitignored.

3. Tech stack (from requirements.txt)

• Django 5.2.12 (LTS), SQLite, Django REST Framework 3.17.1
• HTMX 2.x (vendored JS), templates rendered server-side
• whitenoise (static files), gunicorn (prod)
• recipe_scrapers 15.11 + requests/lxml/extruct for URL recipe import

4. Data model (models.py)

The system models two kinds of meal: meta-recipes (templates) and fixed recipes (traditional ingredient lists). Everything else supports those plus inventory.

Model	Role	Key fields / rules
Tag	ingredient category	just a unique name (protein, carb, veg, …)
Ingredient	master ingredient	name (unique, stored lowercased), default_unit, tags (M2M), shelf_life_days, aliases (JSON list), preferences (free text — how Tom likes it cooked)
PantryItem	current stock	FK ingredient, quantity (Decimal), unit, location ∈ {fridge,freezer,cupboard}, stored_date (auto), expiry_date, is_staple, notes
MetaRecipe	template	name, method (markdown text), times, default_servings, gear_needed, tags (JSON)
Slot	swappable category in a template	FK meta_recipe, name, required, max_choices
SlotOption	an ingredient that can fill a slot	FK slot+ingredient, quantity_per_serving, unit, notes
MetaRecipeBase	always-needed ingredients	FK meta_recipe+ingredient, quantity_per_serving, unit (onion/garlic/oil etc.)
Recipe	fixed recipe	name, method, times, servings, source_url, source_book, tags
RecipeIngredient	line in a fixed recipe	FK recipe+ingredient, quantity, unit, optional
CookLog	what was cooked	date (auto), FK to either meta_recipe or recipe, slot_choices (JSON), servings, rating (1-5), notes
ShoppingListItem	shopping line	optional FK ingredient + fallback name, qty/unit, reason, checked

Model-level business rules (enforced in clean()/save())

• PantryItem.save() calls full_clean() on every save, so the rules below fire on API writes, admin writes, and HTMX writes alike.
  • Freezer item must not have an expiry date → raises ValidationError.
  • Fridge item with no expiry auto-fills from ingredient.shelf_life_days (if set); otherwise left null (no error).
• CookLog.clean(): must link to exactly one of meta_recipe / recipe (not zero, not both). But note CookLog does not override save(), so clean() only runs when something calls full_clean() — DRF does, the log_cook endpoint does not (it uses objects.create). So the "exactly one" rule is not actually enforced on the main cook-logging path.

5. The API (views.py, /api/)

Auth: TokenAuthentication + SessionAuthentication, all endpoints IsAuthenticated. Caine uses a token; browser/admin uses session.

Plain CRUD viewsets (DefaultRouter): tags, ingredients, pantry, meta-recipes, slots, slot-options, meta-recipe-bases, recipes, recipe-ingredients, cook-log, shopping-list. filterset_fields are declared on a couple of viewsets but django-filter is not installed (not in requirements) and not in INSTALLED_APPS, so those filters are effectively inert — query params like ?location=fridge are silently ignored. Same for search_fields without a SearchFilter backend configured.

Custom endpoints (the interesting logic):

• GET /api/what-can-i-cook/?servings=N — builds an in-memory pantry lookup (ingredient_id → list of stock rows with expiry flags), then for every meta-recipe checks each base ingredient and each slot, and for every fixed recipe checks each line. Classifies as ready ✅ / partial ⚠️ / missing ❌ and sorts ready-first. Staple base ingredients are assumed always available. Expiry warnings attached per ingredient.
• POST /api/log-cook/ — creates a CookLog; if deduct: true, subtracts used quantities from the pantry via _deduct_ingredient. Accepts meta_recipe_id or recipe_id, slot_choices, servings, notes. Does not accept rating even though the model has the field.
• POST /api/bulk-pantry-add/ — photo-intake path. Resolves each item by name/alias, auto-creates the ingredient if unknown, computes fridge expiry from shelf life, and merges into an existing same-location stock row if one exists (adds quantity, refreshes expiry). Marks known staples via a hardcoded name set.
• GET /api/generate-shopping-list/?recipes=…&servings=…&add=true — suggests: (1) staples at qty 0, (2) items expiring within 2 days, (3) per requested recipe, the first option of any required slot that can't currently be filled + missing base ingredients. Dedupes by name, sorts by section+priority, optionally writes ShoppingListItem rows.
• POST /api/create-meta-recipe/ (POST=create, PUT=update) — builds a whole meta-recipe (slots + options + bases) in one nested call. On PUT it deletes and rebuilds all slots and bases. Auto-creates ingredients by name, can tag them.
• POST /api/import-recipe/ — fetches a URL, parses with recipe_scrapers, optionally creates a Recipe. See the gotcha in §7 — it never creates RecipeIngredient rows.

_deduct_ingredient ordering note: the docstring/comment says "prefer fridge, then cupboard, then freezer", but the queryset only does .order_by("expiry_date"). In SQLite NULLs sort first, and freezer items have null expiry — so freezer stock is actually consumed first, the opposite of the stated intent.

6. The HTMX web app (views_htmx.py, /app/)

Four pages, server-rendered, each swapping HTML partials over HTMX:

• Pantry (/app/) — add item (auto-creates ingredient, merges into existing stock), delete, move fridge↔freezer (freezer clears expiry; fridge sets +shelf_life or +7d), inline-edit expiry. Items grouped fridge / freezer / cupboard with expired/expiring colour badges.
• Recipes (/app/recipes/) — "what can I cook", meta-recipes only (fixed recipes are not shown here, unlike the API endpoint). Servings is hardcoded to 2. This re-implements the matching logic from what_can_i_cook in a second place (see §7).
• Shopping (/app/shopping/) — generate (staples + expiring + recipe gaps, with a summary count), toggle checked, clear checked.
• Cook Log (/app/log/) — read-only history of the last 50 entries with star ratings. There is no way to create a cook log from the web app — only via API or admin.

All HTMX action views are @csrf_exempt (commit 6786502 — "localhost-only app"). CSRF token is still wired into base.html and sent, but the server ignores it.

Front-end gaps worth knowing before changing things: !! this has become more important since no more caine

• No UI to log a cook, set a rating, or pick slot choices for a real meal.
• No UI for fixed recipes at all (browse, import, or cook).
• No UI to edit ingredients, aliases, preferences, tags, or staples.
• No UI to manage/create meta-recipes (admin or API only).
• Servings is fixed at 2 across the recipes page.

7. Duplication & correctness traps (read before editing)

1. Two copies of the "what can I cook" matcher. views.what_can_i_cook (JSON, includes fixed recipes, configurable servings) and views_htmx.recipes_page (HTML, meta-only, servings=2) implement the same logic separately. They have already drifted. Any change to matching rules must be made in both, or the logic should be extracted into one shared helper first.

2. Recipe URL import creates a Recipe with zero RecipeIngredients. import_recipe_url stores the scraped ingredient list only as raw text in the response ("needs_review"); it never links ingredients to the model. Consequence: an imported fixed recipe has an empty ingredient list, so what_can_i_cook considers it always "ready" (nothing to be missing). Linking ingredients is a manual admin/API step that nothing automates yet.

3. Three different "expiring soon" windows. what_can_i_cook uses ≤2 days, PantryItem.expiring API action uses ≤3 days, shopping generation uses ≤2 days, and the HTMX pantry/recipe views use ≤2 days. No single constant — easy to make them inconsistent further.

4. log_cook bypasses CookLog.clean() (uses .create, no full_clean), so the "exactly one recipe link" invariant isn't enforced on that path, and rating can't be set through it.

5. Deduction order contradicts its comment (see §5).

6. is_staple / restock relies on quantity reaching exactly 0. Staple restock logic filters quantity=0. Items are decremented by deduction and merged on add, but nothing deletes a depleted row, so a staple at 0 stays as a 0-qty row — which is what restock/shopping keys off. Fine, but means stock rows accumulate rather than disappear.

7. Declared filterset_fields/search_fields do nothing (no django-filter / filter backend installed) — don't rely on them.

8. Security / production state

Matches the "TODO" list in the design doc — none of these are done in code:

• DEBUG = True and a literal django-insecure-… SECRET_KEY checked into settings.py. No env-var override.
• ALLOWED_HOSTS includes food.tomflux.xyz and food.jihakuz.xyz.
• HTMX app has no authentication and is CSRF-exempt. The design doc says it's exposed at food.tomflux.xyz; protection relies entirely on nginx blocking /api/ and /admin/, and on nobody pointing at /app/. Anyone who can reach /app/ can read and mutate the whole pantry.
• API is properly auth'd (token/session, IsAuthenticated).
• Secrets/hardening (DEBUG off, real SECRET_KEY, session-auth on /app/) are all still open.

9. Seed data vs. the live database (important)

seed.py creates: 8 tags, 29 ingredients, 22 pantry items, and only 2 meta-recipes (Stir Fry, Traybake) — no fixed recipes, no cook logs.

The committed db.sqlite3.backup-20260402-174550 matches the seed exactly: 2 meta-recipes, 0 fixed recipes, 0 cook logs, 0 shopping items, 16 slot options.

The design doc, however, describes 6 meta-recipes (the 4 extra from "The Quick Roasting Tin": Baked Pasta, One-Tin Curry, Roasted Fish, Traybake Meatballs), 30+ extra ingredients, and 75 imported recipes. None of that is in this repo — it was created on the production DB via the create-meta-recipe / import APIs and never re-seeded. cookbooks/ recipes_extracted.json holds the raw scraped data but nothing loads it automatically.

Implication: a fresh migrate && seed here gives you the minimal 2-recipe system, not what's described in the docs or running in prod. If you need parity, you'd pull the prod DB or replay the API calls.

10. Things to decide before changing (open questions)

These come straight out of the gaps above — likely candidates for "the changes" depending on intent:

• Auth on /app/ — the standing TODO. Add login_required (session auth) or keep relying on nginx?
• Cook logging from the web — currently API/admin only; no rating capture on the main path.
• Fixed recipes are half-built: importable but not ingredient-linked, not shown in the web UI, and trivially "always cookable".
• Unify the two matchers before touching matching behaviour.
• Production hardening (DEBUG, SECRET_KEY) untouched.
• Shopping list UX — the design doc explicitly notes "Tom not happy with current flow, needs more thought."

---

Method: read every Python module, all templates, settings, urls, migrations, the seed command, the deploy unit, and inspected the committed SQLite backup. No code was changed.