A fine-tuned Llama 3.1 model for healthcare inquiries.

Domain fine-tuning that beat general-purpose models on accuracy, latency, and cost — while keeping data inside the customer's VPC.

+22%accuracy lift vs. base Llama 3.1 70B

At a glance

The challenge, approach & outcome.

Challenge
A healthcare information platform had data-residency requirements that ruled out hosted frontier models. Base Llama 3.1 was close but not accurate enough on the clinical inquiry task. They needed a self-hostable model that hit the eval bar.
Approach
Curated a 14,000-pair instruction dataset of approved clinical inquiry / response examples. LoRA fine-tuned Llama 3.1 70B with three evaluation rounds, distilled to 8B for inference cost. Served on vLLM inside the customer's AWS VPC with audit logs on every call.
Outcome
Fine-tuned model beat base Llama 3.1 70B by 22% on the eval set and matched GPT-4o accuracy at 61% lower inference cost. Fully self-hosted — no patient data ever leaves the customer's VPC.

Deliverables

Stack

Have a similar problem?

30-minute discovery call. No NDA gatekeeping. We'll tell you straight whether we're a fit.