Three system deep-dives. Each one follows the same structure: what broke, how we approached it, what we traded away, and what we'll measure.
LLM Symptom Understanding
7 MIN READ
#AI-Agents#LLMTools#Groq#Evals
Two-Pass Tool Orchestration for Symptom Triage
Splitting LLM symptom triage into two forced passes: a tool-only severity classification, then a deterministic facility lookup, then a grounded response, so the model can never invent a facility name or commit to a severity before it has enough information — then proving it holds with two independent evaluation tracks: an online deterministic check against simulated live traffic, and an offline LLM-as-judge pass over a purpose-built dataset.
Filtering facilities by severity eligibility before ranking by distance, using a plain Haversine calculation over an in-process cache, fast enough for the inline triage path, with the ranked candidate list already shaped for a future travel-time upgrade.
Two-Tier Facility State: In-Process Cache + Redis Wait Times
Splitting facility state into two tiers that match their actual freshness and failure requirements: an in-process ETag cache for the rarely-changing facility directory, and a Redis cache-aside chain with a Supabase fallback for wait times that change every scrape cycle.
Event-Driven Fan-Out: S3 to Lambda to EventBridge to dbt
Decoupling a multi-source ingestion pipeline into independent Lambda stages using native S3 and EventBridge event triggers instead of a Step Functions orchestrator or a polling coordinator — so a failure in one processor never blocks the others, and a completion signal from any of three independent sources fans into one shared transform-integrity-test-live-health checkpoint.
An LLM's output can't be verified by reading the code that calls it, so Pass 2's generator output is checked by two independent tracks, each catching a different class of unfaithfulness: a zero-cost deterministic substring check running on every live response, and an offline LLM-as-judge pass scoring subtler fabrications a string match can't see.
