AI does not enter healthcare through the operating room. It enters through the paperwork.
I noticed this recently at my own dentist. The appointment reminder no longer came by text message but through a WhatsApp chatbot, with my name, the location and the time in it. And in the treatment room, two small microphones hung next to the chair, with the question whether I was okay with the treatment being recorded for an automatic summary. Not a question you calmly think through with your mouth wide open.
The most striking thing about AI in the treatment room is how invisible it is. The treatment itself went as it always does, craftsmanship of two skilled hands. You could easily miss the microphones, and that is exactly why this kind of technology settles in so quietly: nothing changes about what you feel, only about what happens to your data.

After the treatment I asked, curious what such a summary looks like and what happens to the recording, whether I could take a quick look at the screen. The automatic summary of forty-five minutes of treatment turned out to be three lines long, with errors in it, each of which my dentist fixed herself before the text went into the record. That looking along was simply possible says a lot about her confidence in the system. And it is my own data, after all; if anyone gets to see it, it might as well be me.
What I looked into at home
Curious, I dug into the software the practice uses. And credit where it is due: on paper, the vendor has things in order. ISO 27001 and NEN 7510 certified, hosting in Dutch data centers, a standard data processing agreement. The vendor is also clear about the recording itself: audio is only on during the appointment, is kept no longer than needed to generate the note, and every summary stays a draft until the dentist approves it. That is more than many young software companies can say.
But the most important question about a recording like this remained unanswered even after a long search: which AI model turns the audio into text, and does it run on their own servers, or does it go through an American provider such as OpenAI, Gemini or Anthropic? At least, I could not find it anywhere. And if I cannot find that out as an agentic engineer, how is an ordinary patient, without an AI background, supposed to weigh it at the moment the question is asked?

What do those microphones actually hear?
Ever since that visit, I have been thinking about what else is on that recording besides the clinical conversation. Because a treatment room does not sound like an office.

For a start, as a patient you cannot talk back. With four hands and a suction tube in your mouth, the best you can produce is "hng-hng", and the model just has to make something of that. Now imagine a heavier procedure, a root canal say, where the patient cries out now and then. That is on the recording too. Does the summary then read "treatment completed without complications"?
You could chuckle and move on, if there were not such a serious tail to it. There are now companies selling voice analysis as a medical instrument: stress, mood and energy measured from how you sound, and according to the industry itself, more accurate pain assessments through ambient listening are on the way. To a system like that, your cry in the chair is not background noise. It is data.
And then the drill itself. Research from Cornell showed that Whisper, the best-known transcription model (made by OpenAI), can invent entire sentences nobody said when it encounters silence and noise. A room full of drilling, suction and half-formed words is about the hardest dictation exercise imaginable for such a model.
And the microphone makes no distinction between "for the record" and "not for the record". The back-and-forth between dentist and assistant goes in just the same, including that one remark about the previous patient. So do the holiday plans your dentist shares with you in the meantime. And who knows what the model makes of the radio in the background. Everything that room produces in sound is, in principle, input. So the question "where does this go" is about far more than three lines of summary.
This turns out not to be an isolated incident
My experience did not stand alone. After some digging I saw a worldwide pattern.
First, the errors. In a study using simulated consultations, 70 percent of AI-generated notes contained errors. Often, relevant information was simply missing. An audit of a thousand hours of transcripts found critical errors in nearly one in five AI transcripts: wrong dosages, a dropped "not". And Ontario's auditor general examined twenty AI scribe systems used by doctors there: twelve wrote down medications that were never prescribed, seventeen missed important details about patients' mental health. My three lines with errors simply fit that picture. Not because the technology is worthless, it demonstrably gets better every month, but because it is often rolled out faster than it is carefully embedded. The difference rarely lies in the model and almost always in the implementation around it.

Then, the consent. In the United States, lawsuits are underway against several hospital systems for allegedly recording doctor-patient conversations with AI without valid consent. In one of those cases the claim goes a step further: the system allegedly noted in the medical record that the patient had given consent, while that question was never asked. An American patient described how, after a recorded appointment, she read through her records and found a statement she never made: the summary claimed she used AI dietary advice, while in the conversation she had explicitly said no.
And the Netherlands? Here the patient perspective is still hard to find, while the technology is arriving all the same. Fortunately, the frameworks already exist. The Dutch medical association KNMG is clear about it. An audio recording of a conversation with a patient counts under the GDPR as processing specially protected data. That requires explicit consent. So my dentist, with her question, was doing exactly what the law requires. That sounds obvious, but the American lawsuits above show it is not.
The trade journal ICT&health recently formulated the question that comes next: the point is not only whether consent was asked, but who has access to the recording, who can do something with it, and under which conditions. Exactly the question I could not find an answer to at home.
The recipe was already in the treatment room
It would be easy to end on a gloomy note here, but that is not what this story calls for. Because my dentist, out of sheer professionalism, did three things that together form exactly the recipe for responsible AI use.
She asked for consent. Not as a formality afterwards, but beforehand, in the room, to me. That is the legal foundation, and it is also simply the decent order of things.
She checked the result. The summary did not go into the record automatically. She read it, spotted the errors and fixed them. All the research above shows this is not excessive caution, but the step that makes the difference between a tool and a risk. The difference between using AI and gambling on AI is whether someone still reads the output.
The treatment stayed human work. AI sat in the paperwork, not in the treatment itself. That last part may still come one day, but for now I prefer to keep my molars with someone holding a diploma. And that is the right order anyway: let AI take over the predictable work first, the work that looks the same every week. The place where a mistake does the least damage is the place where you start.
Consent, verification, and a human where it matters most. That recipe is not unique to healthcare. It applies just as much to the quote an AI assistant drafts for you, the inbox that gets answered automatically or the customer calls that get summarized. Start with the work that has the smallest consequences, have a human read the output for as long as the technology still makes mistakes, and be honest with the people whose data you process about what happens to that data.
The question you may ask every vendor
What stays with me is not that the summary contained errors. Young technology makes mistakes, that comes with the territory, and there was a professional in between who caught them. And let there be no misunderstanding: a practice that dares to try this, and is open about it, deserves to be followed. My concern is not with the chair, but with the chain behind it. What stays with me is that the most important question remained unanswered: where does the data go?
Anyone buying software that works with sensitive data is entitled to ask that question out loud. Which model processes our data, where does it run, what is stored and for how long? A vendor with a clear answer has thought it through. And where that answer is still missing today, the question itself already has value: the more often customers ask it, the faster transparency becomes the standard.
Because the question "are you okay with AI listening in?" is one we will all hear more often in the coming years. At the dentist, on the phone, in your inbox. The best answer is not yes or no, but a question in return: where does it go? Technology with a clear answer to that question earns the trust it asks for. And if you build or buy yourself: make sure your answer is ready before anyone asks.
Curious what AI can do for your business?
Take the free AI Scan and find out in 1 minute.
Frequently asked questions
- Is a healthcare provider allowed to record a conversation or treatment with AI?
- Only with the patient's explicit consent. According to the Dutch medical association KNMG, an audio recording of a patient conversation counts under the GDPR as processing specially protected data. The provider must explain in advance what is recorded and why, and the patient may withdraw consent at any time.
- How often do AI-generated notes contain errors?
- More often than you would hope. In a study using simulated consultations, 70 percent of AI-generated notes contained errors, and an audit of a thousand hours of transcripts found critical errors such as wrong dosages in nearly one in five. Human review of every note is therefore not a luxury but a necessary step.
- What should you watch for when using AI on sensitive data?
- Three things: ask consent from the people whose data you process, have a human review the output for as long as the error rate demands it, and know exactly where your data goes. Ask every vendor which model processes the data, where it runs, what is stored and for how long. No clear answer is an answer too.
