Every time a new wave of AI hits healthcare, the first reaction is fear.

Fear that the model will hallucinate, give bad advice, or worse: miss something critical. With Voice AI, the fear multiplies. A text bot making a mistake is one thing. A voice on the other end of the phone line (especially when sounding like a human) raises the stakes.

And yet, we can’t ignore the other side of the equation: the healthcare system is already overburdened. Labor is maxed out, costs keep rising, and staffing every single follow-up call with a nurse or MA just doesn’t scale. If we want to keep patients engaged and taking preventative measures while keeping costs under control, some version of automation has to exist.

So the question becomes: who holds the risk in all this?

Two Archetypes to Learn From

Flowline Health isn’t the first patient engagement platform to exist, and we’re not the first people without a clinical license making clinical support calls. To figure out our lane, it helps to look at the companies that came before us.

1. Technology platforms: Think SeamlessMD and others that deliver patient care plans digitally. Their software is deterministic: if X happens, Y gets displayed. These companies protect themselves by disclaiming medical advice and getting their content reviewed and approved by one or more clinicians.

2. Call centers: Whether it’s a domestic operation or a group outside the US making clinical & non-clinical calls, the playbook looks different than software. Humans are trained not to give medical advice. If they slip and do it anyway, liability comes down to whether the call center had quality controls, training, and escalation protocols in place.

Where We (and other Voice AI companies) Fit

Voice AI doesn’t slot neatly into either bucket, but let’s be real: we’re closer to a call center. The non-deterministic nature of an AI agent looks more like a human caller than a piece of deterministic software. You can prompt it with rules all day, but there’s always some sliver of a chance it veers off (the same way a human might forget their training).

That means we hold the risk.

Our model, for safety and liability, needs to follow the call center pattern: scope limits, strict scripts, confirming all collected information, and airtight escalation protocols. If we ever release something that starts handing out bad medical advice or missing escalations, then we are the rightful owners of that liability, and more importantly, we must implement and maintain a quality framework that exceeds the standards of your average call center.

Our Quality Framework

If we think like a call center, the path becomes pretty clear.

Scope

Stick to calls that a medical assistant or call center rep could reasonably handle. Stay away from scenarios that require an RN’s training or judgment. Draw a hard line: if it requires clinical inference, it’s out of scope.

Scripts

Make questions lead to clear answers that don’t require a judgment call by the call center rep. A good example would be: “Are there any other clinical issues you’d like to discuss with a nurse?” This frames the question around whether the patient would like to speak to a nurse, as opposed to having them list off their symptoms and then requiring the representative to make a judgment call as to whether those symptoms constitute escalation.

Information Confirmation

AI Callers can mishear things in the same way that a person can. A patient may say something like “yeanoh” instead of a clear “yes” or “no”. For any data points that are collected over the course of a call, it is critical to read them back to the patient to ensure that they were heard correctly. This is a common practice in call centers.

Escalation Protocols

There’s no way around this: escalation rules have to be correct, both in definition and execution.

• Definition: co-designed with providers, anchored in Flowline’s own framework of best practices.

• Execution: Flowline’s framework includes multiple layers of validation. That means live transfers when an escalation is triggered, separate AI review of transcripts, and random human review of 5% of calls for hallucinations, accuracy, and transcription quality.

Closing

As it gets adopted, Voice AI in healthcare will make people nervous, and for good reason. But the cost pressure in the system means the alternative (never automating) isn’t realistic. The safe way forward is to take what already works in call centers, apply it to AI, and back it with transcripts, validation, and scope discipline.

If we get scope, scripts, and escalation right, then we can build something safe, defensible, and useful. If we can’t, we shouldn’t be building at all.