← Work

RAG that only cites what it retrieved — semantic search and a question-answering food log

Two retrieval features built on one rule: the model never produces a number. A pgvector hybrid search over the food catalog, and a three-step structured RAG over the user's log — verified against the database to the third decimal.

nutri.’s core priority is the accuracy of stored nutrition data, and every AI feature inherits that constraint. These two shipped together: a semantic layer for catalog search, and “Ask your log”, a natural-language question-answering flow over the food diary. Same discipline in both — the LLM decides what to look up and how to phrase it, and is never the source of a number.

Semantic search: keyword first, vectors when they earn it

Search was pure ilike: “citrus fruit” returned nothing even though the catalog had Orange, Grapefruit and Lemon. At 634 foods (490 user-created plus a 144-item shared USDA base), the user’s vocabulary stopped matching row names.

The design is deliberately conservative:

• A nullable embedding vector(768) column and a match_foods RPC — security invoker, so row-level security still decides what each user can match. Cosine distance, a max_dist cutoff for noise, and no HNSW index on purpose: at ~600 rows a sequential scan wins, and the threshold for adding the index (~10k rows) is written down instead of pre-built.
• Hybrid, not vector-first. Keyword match stays primary; only when it returns fewer than 8 hits does the query get embedded and merged in (deduplicated, keyword results first). If the embedding call fails for any reason — no key, network, quota — the function returns null and search behaves exactly as before. Degraded, never broken.
• One gotcha worth recording: gemini-embedding-001 only L2-normalizes its full 3072-dimension output. At 768 dimensions you must re-normalize yourself, or the “cosine” distance is quietly wrong.
• The embedded text formula (name + ' ' + brand) is frozen: the whole catalog was backfilled with it, and changing it would silently degrade every existing vector.

The backfill itself is part of the story: ~4 MB of vectors never passed through an AI agent’s context. Two temporary security definer RPCs (list pending / apply batch, guarded so existing values can’t be overwritten) plus a local resumable script pushed all 634 over HTTP — in batches of 20 with 65-second backoff, because the embedding free tier limits per minute. Roughly ten minutes end to end, and both RPCs were dropped afterwards.

Live checks that made it into the log: “fruta cítrica” → Naranja, Toronja, Limón (zero keyword hits); “pescado azul” → tuna, snapper, salmon — including a cross-lingual match on “Canned tuna in water”.

Ask your log: RAG where the retrieval step is SQL

The obvious build — embed the diary, retrieve by similarity — would have been worse on both precision and cost. Diary data is tabular and already exact in SQL views. So the pipeline is structured RAG in three steps:

1. Plan. A planner model turns the question into {date_from, date_to, nutrients[], need_detail} as structured output. The valid nutrient keys are injected into the prompt from the app’s single source of truth, and a pure, tested sanitizer clamps everything the model could get wrong: missing dates fall back to the last 30 days, future dates get cut to today, ranges cap at 92 days (surfaced in the UI, not silent), unknown nutrients are filtered out.
2. Retrieve. Daily totals for the range come from the same SQL views the dashboard reads; per-entry detail is fetched only when the plan asks for it; targets are resolved per-day with the app’s existing resolver. Nothing is recomputed, nothing is estimated.
3. Answer. The retrieved rows go in as a compact CSV-style context (capped at the 400 highest-calorie entries, with a notice when the cap bites), and the answering model is instructed to use exclusively figures present in that context, cite concrete days and foods, and describe without prescribing.

Verified to the third decimal

The claim “it only cites retrieved figures” was tested end to end: asked “which foods gave me the most sodium in the last two weeks?”, the answer cited “Whole-grain bread, 91 g → 409.5 mg” on July 2. The database view says 409.500. The pure pieces of the pipeline — vector normalization, result merging, plan sanitization, context formatting — are under unit test (21 tests of the suite’s 156), and the sheet carries a fixed footer: AI-generated answer — verify against the Dashboard. The real numbers are in the context, but prose is still a model’s; the mitigation is honesty, not pretending the risk is zero.

Limits, on the record

Foods created outside the app (MCP connector, REST) have no embedding until someone edits and saves them — hybrid search covers them by keyword meanwhile. The distance cutoff trades a little far-field noise for cross-lingual matches, and it’s a runtime parameter, not a migration. The question sheet keeps no conversational thread; each question stands alone. All three are documented decisions with written upgrade paths, not surprises.

The build lives in the open repo: github.com/vryahn/nutri — src/lib/ai.js, supabase/migrations/017_foods_embedding.sql. Context for the whole system: nutri. — running a production app inside an agentic stack.