Two-Pass Tool Orchestration for Symptom Triage
Architectural Overview
Splitting LLM symptom triage into two forced passes: a tool-only severity classification, then a deterministic facility lookup, then a grounded response, so the model can never invent a facility name or commit to a severity before it has enough information — then proving it holds with two independent evaluation tracks: an online deterministic check against simulated live traffic, and an offline LLM-as-judge pass over a purpose-built dataset.
MediCoord's chat interface routes patients to the nearest appropriate facility based on a short symptom conversation. The riskiest part of that pipeline isn't the routing. It's the classification. A large language model asked to output severity directly, in one shot, will produce a confident, well-formatted answer whether or not it actually has enough information. We split triage into two passes so the model's linguistic fluency and Python's determinism each do the job they're actually good at. Splitting the pipeline is a design decision; proving it holds is a separate discipline. An LLM's behavior can't be verified by reading the code that calls it, so evaluation isn't a QA afterthought bolted on at the end — it's built as two independent tracks that each catch a different failure mode: a zero-cost deterministic check that runs on every request against simulated live traffic, and an offline pass where a second model judges faithfulness against a dataset purpose-built to exercise this exact pipeline.
The Problem
Two failure modes show up when you let a single LLM call do both classification and response generation. First, models are trained to be helpful, which means they'll often generate a plausible-sounding facility name and address instead of admitting they don't have one, and there's no reliable way to catch that after the fact once it's already in a patient-facing sentence. Second, models are eager: given three words of symptom description, an unconstrained model will confidently commit to a severity level rather than asking a clarifying question first. Both failure modes are worse than a slow response: a wrong facility name sends a patient to the wrong door, and a premature classification is either a false alarm or a missed one.
A model asked to write "the nearest facility is X" in the same breath as reasoning about symptoms has no way to guarantee X is a real place. It is optimizing for a plausible sentence, not a grounded fact.
Nothing stops a one-shot model from committing to a severity level on the first message, before it has asked about duration, associated symptoms, or history.
The Architecture Strategy
triage_response is the only tool the model can call in Pass 1, and its schema forces a severity enum (routine, moderate, urgent, emergent) plus a short internal reasoning string and an information_sufficient boolean: no patient-facing text, no facility name, nothing the model could invent. A minimum-turns gate suppresses that tool call as a followup question unless the model has already seen enough of the conversation, unless severity comes back 'emergent' or the conversation has hit a hard turn ceiling, both of which bypass the gate, because refusing to route an emergency to force more questions is its own failure mode. Once severity is classified, Pass 2 is deterministic: a plain Python function looks up the nearest eligible facility from the in-process cache, no LLM involved, and returns a real facility record. That record is injected into a second grounding message before asking the model to write the two-to-four sentence patient-facing reply. The model never gets a chance to invent a name because by the time it's writing prose, the name is already a fact in its context, not a decision it's making.
TRIAGE_RESPONSE = ToolDefinition(
name="triage_response",
description=(
"Call this when you have sufficient information to classify the "
"patient's symptom severity. Do NOT include a patient-facing "
"response — the conversational response is generated separately "
"after the nearest facility is identified from the system's data. "
"Never invent or guess facility names."
),
parameters={
"severity": {
"type": "string",
"enum": ["routine", "moderate", "urgent", "emergent"],
},
"reasoning": {"type": "string"},
"information_sufficient": {"type": "boolean"},
},
required=["severity", "reasoning", "information_sufficient"],
)if facility:
facility_fact = (
f"The nearest appropriate facility is: {facility['name']} "
f"at {facility['address']}, approximately "
f"{facility['distanceKm']} km away. Use this exact facility "
f"name in your response — do not modify or replace it."
)
else:
facility_fact = (
"No location data is available. Do not mention any specific "
"facility. Advise the patient to call 211 or search online "
"for nearby care."
)Alternatives Considered
The simplest design is one LLM call that both classifies severity and writes the response in the same completion. We prototyped this first: it's faster (one round trip) and simpler to implement. It broke exactly the way you'd expect: the model would generate specific-sounding facility names that didn't exist in our data, and there was no clean place to intercept and correct a hallucinated fact inside an already-generated sentence.
We considered constraining the single-pass response with a JSON schema instead of splitting into two calls. That solves the format problem but not the grounding problem: a well-formed JSON object can still contain an invented facility name. Tool calling forced us to physically remove the facility field from what the model is allowed to write in pass one, which JSON mode alone doesn't do.
Lessons Learned
The Groq/Llama models we use will sometimes return both a text completion and a tool_calls array in the same response. Early logic branched on finish_reason and occasionally dropped a valid tool call. The fix was to treat a non-empty tool_calls list as authoritative regardless of finish_reason.
The minimum-turns gate was originally unconditional. That meant a first message like 'chest pain, can't breathe' would get a clarifying question instead of an immediate route. We added an explicit bypass: an emergent classification skips the gate no matter how early in the conversation it fires.
Two passes mean two LLM round trips instead of one, adding latency to every triage decision, acceptable for a chat interface, less so if this were a high-throughput batch job. The min-turns gate is also a blunt instrument: it counts user turns, not information content, so a chatty user who says a lot in one message still waits, and a terse user who needs more prompting can still get force-classified at the turn ceiling with information_sufficient: false. What's next: a knowledge-graph grounding step sourced from Canadian diagnostic data is in active development this week, aimed at the classification step itself. Today's two-pass design is the orchestration layer that step will plug into, not a replacement for it.
- Validated by two independent evaluation tracks — a zero-cost deterministic groundedness check and an offline LLM-as-judge faithfulness score — see Case Study: Two-Track LLM Evaluation for full methodology and measured results.
- See Case Study: Two-Track LLM Evaluation for full methodology.