All posts
Guides7 min read

How AI notetakers tell speakers apart (and why most guess)

Most tools borrow names from a calendar invite, which works on a video call and slips everywhere else. Only voice recognition knows a person again next week.

Siya RagadeGuides

AI notetakers tell people apart in two steps: first they split the recording by voice, and then they try to put a name on each voice. Most tools borrow those names from a calendar invite or a video call’s participant list, which works when everyone joins from their own laptop and slips almost everywhere else.

Picture a Tuesday check-in. Priya, the client, joins from a conference room with two colleagues, Marcus joins from home, and Lena from finance drops in without an invite. The notes arrive, and the key decision reads “Speaker 2 agreed to move the launch to March.” AI notetakers mix people up like this all the time, and it is easy to miss. This is the quiet problem with meeting notes: the words are right, but the people are wrong.

The short version

  • Three different jobs. Splitting audio by voice, borrowing names from a calendar, and recognizing a person’s voice itself.
  • Numbers, not names. Splitting by voice gives Speaker 1, Speaker 2 and so on, and the numbers start over in every recording.
  • Memory needs a voice. Only voice recognition knows a person again in a later conversation.

What does “speaker identification” actually mean?

“Speaker identification” is a catch-all for three different jobs, and most of the confusion comes from treating them as one. Each job needs different inputs, so each one fails in different places.

Diarization, or “who spoke when”

Speaker diarization, or splitting a recording into turns by voice, works out where one voice stops and the next begins without knowing anyone’s name. Google’s speech-to-text documentation explains that it labels each voice it detects with a number. These numbered tags are often called speaker labels. In Priya’s meeting, diarization produces Speaker 1 to Speaker 4, and the numbers start over in the next recording.

Calendar or meeting-app naming

The second job is matching those numbered voices to names from somewhere else, usually the calendar invite or the display names in a video call. In Priya’s meeting, this correctly names Marcus, since he has his own window on the call. Priya and her two colleagues share one room device, so the tool sees one person for three, and Lena is not on the invite at all.

Voice recognition

The third job is recognizing a person by their voice once they have been named. NVIDIA’s speech AI documentation describes diarization as working out who spoke when, and identification goes a step further by matching each voice to a real name. That same memory is what lets you turn old voice memos and interview files into named notes instead of a page of numbered speakers. In Priya’s meeting, a tool with voice recognition already knows Priya and Marcus from last week, and once someone names Lena, it knows her next time too.

How do the three approaches compare?

ApproachWhat it needsWhere it worksWhere it failsRemembers people across conversations?
DiarizationOnly the audioAny recording with clear turnsShort replies, crosstalk, noise; gives numbers, not namesNo
Calendar or meeting-app namingAn invite or participant listVideo calls where each person has their own deviceShared rooms, uninvited people, in-person meetingsOnly as far as the invite goes
Voice recognitionNaming each person onceOnline and in personSomeone never named; very noisy roomsYes

Real tools often mix these approaches.

Why are calendar names a guess?

Calendar names are a guess because an invite says who was asked to come, not who is talking. That is why calendar invites can’t tell you who’s talking once the room fills with people the invite never listed. This match holds up on a video call where each person has their own window, and it weakens as soon as the room stops looking like the invite:

  • The 2pm coffee. There is no invite and no video call, so there are no names to borrow.
  • Six people on one laptop. A conference room joins as one participant.
  • The colleague without an invite. There is no name to match her voice to.
  • The borrowed laptop. A latecomer’s words show up under the laptop owner’s name.

Dragon Notes starts with your calendar for online meetings too, since an invite is a helpful hint. A hint is a starting point, and a voice is what actually belongs to the person.

Why are in-person conversations the hardest case?

In-person conversations are the hardest case because the tool gets one stream of sound and no list of names. It is the same reason it is hard to transcribe interviews when people talk over each other, even with a good microphone. People interrupt, laugh and answer with a quick “yeah”, and AssemblyAI’s guide calls these short turns too brief to attribute with confidence.

This is where voice recognition earns its place, since a voice comes with the person into the lecture hall and the client lunch.

How does Dragon Notes recognize people?

Dragon Notes recognizes people by their voice. Name someone once, and Dragon Notes recognizes their voice from then on, online and in person, on Mac, Windows, iPhone, iPad, Android and Apple Watch.

In the first meeting, Lena shows up as an unnamed voice, and we tag her as Lena. The following week, she joins a hallway chat about the budget, and the note already shows her name. There is no invite and no meeting app in the room, only her voice.

Name the voice once. The next time Lena speaks, the note already knows her.

Because Dragon Notes knows who is talking, it can also bring back what each person said before, such as decisions, promises and what has changed. This is what “the AI notetaker that remembers the people in the room” means in practice. To see it at work with one phone on the table, read how we tested speaker recognition in a real room.

When does another approach fit better?

  • Every meeting is a video call. A calendar-based tool will usually name people well.
  • Only the words matter. Numbered turns from diarization may be enough.

How can you check how your notetaker names people?

Test any tool on the conversations it is likely to get wrong.

  • Record one in-person chat. See whether you get names, numbers or one name for everyone.
  • Invite someone not on the calendar. See whether their words go to someone else.
  • Name someone, then record them next week. See whether they are back to Speaker 2.

Frequently asked questions

What is the difference between speaker diarization and speaker recognition?

Speaker diarization splits a recording into turns by voice and labels them with numbers. Speaker recognition links a voice to a named person, so the tool knows them again later.

Why does my meeting transcript say “Speaker 1” instead of names?

The tool split the voices but had no name to match them to. That is why your transcript says “Speaker 2” where a name should be. This happens often with in-person recordings and anyone not on the invite.

Can AI transcription identify speakers by name?

Yes, but most AI transcription only gives speaker labels like Speaker 1 and Speaker 2. Names come from a calendar or meeting app, or from voice recognition after someone names each person once.

Do I have to name the same person in every meeting?

In Dragon Notes, no: name someone once, and Dragon Notes recognizes them in later conversations. Other tools differ, and Plaud’s help center, for example, says names added for other people apply only to that recording.

Try it on your next conversation

Most notetakers borrow names from an invite, which works on a video call and slips everywhere else. Dragon Notes recognizes voices, so a name given once follows that person from then on. Dragon Notes is free for the first week, then $1.99 a month. Try it on your next conversation.

Siya RagadeNov 11, 2026