food/research.md

# Research: How the Food app works today

> Snapshot taken 2026-06-22, reading the code on `master` (HEAD `96d9c4b`).
> Purpose: establish ground truth before making changes. Where the shipped
> docs (`Meal Planning System.md`) disagree with the code, the **code** is
> treated as authoritative and the discrepancy is called out.

---

## 1. What this is

A single-user Django app for Tom: pantry inventory + "meta-recipes"
(cooking templates with swappable ingredient slots) + a smart shopping list +
a cook log. Two front doors:

- **REST API** (`/api/`) — token auth, built for Caine (the Matrix assistant).
- **HTMX web app** (`/app/`) — server-rendered HTML fragments, no auth.

Plus Django admin (`/admin/`) as the full-CRUD fallback.

There is **one Django app**, `kitchen`, holding everything. ~1000 lines of
Python total. No Celery, no JS build, no separate frontend — it's deliberately
small.

## 2. Layout

```
food_project/        Django project config
  settings.py        single settings file, DEBUG=True (see §8)
  urls.py            routes /admin/, /api/, /api-auth/, /app/
kitchen/
  models.py          all 11 models
  admin.py           admin registrations + inlines
  serializers.py     DRF serializers (read + nested write)
  views.py           DRF viewsets + the "smart" API endpoints (JSON)
  urls.py            /api/ router + custom endpoints
  views_htmx.py      HTML-fragment views for the web app
  urls_htmx.py       /app/ routes
  templates/kitchen/ base.html + 4 pages + 5 partials
  static/kitchen/    htmx.min.js (vendored)
  management/commands/seed.py   initial data loader
  migrations/        0001_initial, 0002_add_rating_and_preferences
deploy/food.service  systemd unit (gunicorn, runs as user `openclaw`)
db.sqlite3.backup-…  committed snapshot = seed state only (see §9)
cookbooks/recipes_extracted.json   75 scraped Roasting Tin recipes (data only)
```

Note the README/design-doc say the project lives at `/var/lib/food/` in
production. This working copy is `/home/tom/srcs/food`. The DB
(`db.sqlite3`), `venv/`, and `staticfiles/` are gitignored.

## 3. Tech stack (from requirements.txt)

- Django 5.2.12 (LTS), SQLite, Django REST Framework 3.17.1
- HTMX 2.x (vendored JS), templates rendered server-side
- whitenoise (static files), gunicorn (prod)
- recipe_scrapers 15.11 + requests/lxml/extruct for URL recipe import

## 4. Data model (`models.py`)

The system models two kinds of meal: **meta-recipes** (templates) and
**fixed recipes** (traditional ingredient lists). Everything else supports
those plus inventory.

| Model | Role | Key fields / rules |
|---|---|---|
| `Tag` | ingredient category | just a unique name (protein, carb, veg, …) |
| `Ingredient` | master ingredient | `name` (unique, stored lowercased), `default_unit`, `tags` (M2M), `shelf_life_days`, `aliases` (JSON list), `preferences` (free text — how Tom likes it cooked) |
| `PantryItem` | current stock | FK ingredient, `quantity` (Decimal), `unit`, `location` ∈ {fridge,freezer,cupboard}, `stored_date` (auto), `expiry_date`, `is_staple`, `notes` |
| `MetaRecipe` | template | name, method (markdown text), times, `default_servings`, `gear_needed`, `tags` (JSON) |
| `Slot` | swappable category in a template | FK meta_recipe, name, `required`, `max_choices` |
| `SlotOption` | an ingredient that can fill a slot | FK slot+ingredient, `quantity_per_serving`, `unit`, `notes` |
| `MetaRecipeBase` | always-needed ingredients | FK meta_recipe+ingredient, `quantity_per_serving`, `unit` (onion/garlic/oil etc.) |
| `Recipe` | fixed recipe | name, method, times, `servings`, `source_url`, `source_book`, `tags` |
| `RecipeIngredient` | line in a fixed recipe | FK recipe+ingredient, `quantity`, `unit`, `optional` |
| `CookLog` | what was cooked | date (auto), FK to **either** meta_recipe **or** recipe, `slot_choices` (JSON), `servings`, `rating` (1-5), `notes` |
| `ShoppingListItem` | shopping line | optional FK ingredient + fallback `name`, qty/unit, `reason`, `checked` |

### Model-level business rules (enforced in `clean()`/`save()`)

- **`PantryItem.save()` calls `full_clean()`** on every save, so the rules
  below fire on API writes, admin writes, and HTMX writes alike.
  - Freezer item **must not** have an expiry date → raises ValidationError.
  - Fridge item with no expiry auto-fills from `ingredient.shelf_life_days`
    (if set); otherwise left null (no error).
- **`CookLog.clean()`**: must link to exactly one of meta_recipe / recipe
  (not zero, not both). *But note `CookLog` does not override `save()`, so
  `clean()` only runs when something calls `full_clean()` — DRF does, the
  `log_cook` endpoint does **not** (it uses `objects.create`). So the
  "exactly one" rule is not actually enforced on the main cook-logging path.*

## 5. The API (`views.py`, `/api/`)

Auth: `TokenAuthentication` + `SessionAuthentication`, **all endpoints
`IsAuthenticated`**. Caine uses a token; browser/admin uses session.

**Plain CRUD viewsets** (DefaultRouter): tags, ingredients, pantry,
meta-recipes, slots, slot-options, meta-recipe-bases, recipes,
recipe-ingredients, cook-log, shopping-list. `filterset_fields` are declared
on a couple of viewsets but **django-filter is not installed** (not in
requirements) and not in `INSTALLED_APPS`, so those filters are effectively
inert — query params like `?location=fridge` are silently ignored. Same for
`search_fields` without a `SearchFilter` backend configured.

**Custom endpoints** (the interesting logic):

- `GET /api/what-can-i-cook/?servings=N` — builds an in-memory pantry lookup
  (ingredient_id → list of stock rows with expiry flags), then for every
  meta-recipe checks each base ingredient and each slot, and for every fixed
  recipe checks each line. Classifies as `ready` ✅ / `partial` ⚠️ /
  `missing` ❌ and sorts ready-first. Staple base ingredients are assumed
  always available. Expiry warnings attached per ingredient.
- `POST /api/log-cook/` — creates a `CookLog`; if `deduct: true`, subtracts
  used quantities from the pantry via `_deduct_ingredient`. Accepts
  meta_recipe_id **or** recipe_id, `slot_choices`, `servings`, `notes`.
  **Does not accept `rating`** even though the model has the field.
- `POST /api/bulk-pantry-add/` — photo-intake path. Resolves each item by
  name/alias, **auto-creates the ingredient if unknown**, computes fridge
  expiry from shelf life, and merges into an existing same-location stock row
  if one exists (adds quantity, refreshes expiry). Marks known staples via a
  hardcoded name set.
- `GET /api/generate-shopping-list/?recipes=…&servings=…&add=true` —
  suggests: (1) staples at qty 0, (2) items expiring within 2 days,
  (3) per requested recipe, the first option of any required slot that can't
  currently be filled + missing base ingredients. Dedupes by name, sorts by
  section+priority, optionally writes `ShoppingListItem` rows.
- `POST /api/create-meta-recipe/` (POST=create, PUT=update) — builds a whole
  meta-recipe (slots + options + bases) in one nested call. On PUT it
  **deletes and rebuilds** all slots and bases. Auto-creates ingredients by
  name, can tag them.
- `POST /api/import-recipe/` — fetches a URL, parses with `recipe_scrapers`,
  optionally creates a `Recipe`. **See the gotcha in §7 — it never creates
  `RecipeIngredient` rows.**

`_deduct_ingredient` ordering note: the docstring/comment says "prefer fridge,
then cupboard, then freezer", but the queryset only does
`.order_by("expiry_date")`. In SQLite NULLs sort first, and freezer items have
null expiry — so **freezer stock is actually consumed first**, the opposite of
the stated intent.

## 6. The HTMX web app (`views_htmx.py`, `/app/`)

Four pages, server-rendered, each swapping HTML partials over HTMX:

- **Pantry** (`/app/`) — add item (auto-creates ingredient, merges into
  existing stock), delete, move fridge↔freezer (freezer clears expiry; fridge
  sets +shelf_life or +7d), inline-edit expiry. Items grouped fridge /
  freezer / cupboard with expired/expiring colour badges.
- **Recipes** (`/app/recipes/`) — "what can I cook", **meta-recipes only**
  (fixed recipes are *not* shown here, unlike the API endpoint). Servings is
  **hardcoded to 2**. This re-implements the matching logic from
  `what_can_i_cook` in a second place (see §7).
- **Shopping** (`/app/shopping/`) — generate (staples + expiring + recipe
  gaps, with a summary count), toggle checked, clear checked.
- **Cook Log** (`/app/log/`) — read-only history of the last 50 entries with
  star ratings. **There is no way to create a cook log from the web app** —
  only via API or admin.

All HTMX action views are `@csrf_exempt` (commit `6786502` — "localhost-only
app"). CSRF token is still wired into `base.html` and sent, but the server
ignores it.

Front-end gaps worth knowing before changing things: !! this has become more important since no more caine
- No UI to log a cook, set a rating, or pick slot choices for a real meal. 
- No UI for fixed recipes at all (browse, import, or cook).
- No UI to edit ingredients, aliases, preferences, tags, or staples.
- No UI to manage/create meta-recipes (admin or API only).
- Servings is fixed at 2 across the recipes page.

## 7. Duplication & correctness traps (read before editing)

1. **Two copies of the "what can I cook" matcher.** `views.what_can_i_cook`
   (JSON, includes fixed recipes, configurable servings) and
   `views_htmx.recipes_page` (HTML, meta-only, servings=2) implement the same
   logic separately. They have already drifted. Any change to matching rules
   must be made in both, or the logic should be extracted into one shared
   helper first.

2. **Recipe URL import creates a `Recipe` with zero `RecipeIngredient`s.**
   `import_recipe_url` stores the scraped ingredient list only as raw text in
   the response ("needs_review"); it never links ingredients to the model.
   Consequence: an imported fixed recipe has an empty ingredient list, so
   `what_can_i_cook` considers it **always "ready"** (nothing to be missing).
   Linking ingredients is a manual admin/API step that nothing automates yet.

3. **Three different "expiring soon" windows.** `what_can_i_cook` uses ≤2
   days, `PantryItem.expiring` API action uses ≤3 days, shopping generation
   uses ≤2 days, and the HTMX pantry/recipe views use ≤2 days. No single
   constant — easy to make them inconsistent further.

4. **`log_cook` bypasses `CookLog.clean()`** (uses `.create`, no
   `full_clean`), so the "exactly one recipe link" invariant isn't enforced
   on that path, and `rating` can't be set through it.

5. **Deduction order contradicts its comment** (see §5).

6. **`is_staple` / restock relies on quantity reaching exactly 0.** Staple
   restock logic filters `quantity=0`. Items are decremented by deduction and
   merged on add, but nothing deletes a depleted row, so a staple at 0 stays
   as a 0-qty row — which is what restock/shopping keys off. Fine, but means
   stock rows accumulate rather than disappear.

7. **Declared `filterset_fields`/`search_fields` do nothing** (no
   django-filter / filter backend installed) — don't rely on them.

## 8. Security / production state

Matches the "TODO" list in the design doc — none of these are done in code:


- `DEBUG = True` and a literal `django-insecure-…` `SECRET_KEY` checked into
  `settings.py`. No env-var override.
- `ALLOWED_HOSTS` includes `food.tomflux.xyz` and `food.jihakuz.xyz`.
- **HTMX app has no authentication** and is CSRF-exempt. The design doc says
  it's exposed at `food.tomflux.xyz`; protection relies entirely on nginx
  blocking `/api/` and `/admin/`, and on nobody pointing at `/app/`. Anyone
  who can reach `/app/` can read and mutate the whole pantry.
- API is properly auth'd (token/session, IsAuthenticated).
- Secrets/hardening (DEBUG off, real SECRET_KEY, session-auth on `/app/`) are
  all still open.

## 9. Seed data vs. the live database (important)

`seed.py` creates: 8 tags, 29 ingredients, 22 pantry items, and **only 2
meta-recipes** (Stir Fry, Traybake) — no fixed recipes, no cook logs.

The committed `db.sqlite3.backup-20260402-174550` matches the seed exactly:
2 meta-recipes, 0 fixed recipes, 0 cook logs, 0 shopping items, 16 slot
options.

The design doc, however, describes **6 meta-recipes** (the 4 extra from "The
Quick Roasting Tin": Baked Pasta, One-Tin Curry, Roasted Fish, Traybake
Meatballs), 30+ extra ingredients, and 75 imported recipes. **None of that is
in this repo** — it was created on the production DB via the
`create-meta-recipe` / import APIs and never re-seeded. `cookbooks/
recipes_extracted.json` holds the raw scraped data but nothing loads it
automatically.

Implication: a fresh `migrate && seed` here gives you the minimal 2-recipe
system, *not* what's described in the docs or running in prod. If you need
parity, you'd pull the prod DB or replay the API calls.

## 10. Things to decide before changing (open questions)

These come straight out of the gaps above — likely candidates for "the
changes" depending on intent:

- **Auth on `/app/`** — the standing TODO. Add `login_required` (session
  auth) or keep relying on nginx?
- **Cook logging from the web** — currently API/admin only; no rating capture
  on the main path.
- **Fixed recipes** are half-built: importable but not ingredient-linked, not
  shown in the web UI, and trivially "always cookable".
- **Unify the two matchers** before touching matching behaviour.
- **Production hardening** (DEBUG, SECRET_KEY) untouched.
- **Shopping list UX** — the design doc explicitly notes "Tom not happy with
  current flow, needs more thought."

---

*Method: read every Python module, all templates, settings, urls, migrations,
the seed command, the deploy unit, and inspected the committed SQLite backup.
No code was changed.*