How accurate are AI interview assistants?

Two very different products get called an AI interview assistant. The first is the live copilot: it listens to a real interview and puts suggested answers on your screen while you're being assessed. The second is the practice tool: it plays the interviewer before the real thing, asks you questions, and tells you what was wrong with your answers.

They fail in different places, so this post takes them one at a time. We build the second kind, which means we have a stake in the answer. That's also why the most useful part of this post is a test you can run on any assistant, ours included.

Accuracy is a stack, not a score

Every assistant, live or practice, has to get three things right in order. If an early layer is wrong, nothing after it can recover.

• Hearing. Turning speech into text. If the transcript says "item potent" where you said "idempotent", everything downstream is answering a question nobody asked.
• Knowing. Producing, or recognising, a technically correct answer. This is the layer the marketing talks about, and it's the one modern models are best at.
• Fitting. Whether that answer is right for this conversation, this candidate and this follow-up. This is where most of the real error lives, and it's the layer nobody advertises.

Layer one: hearing

Speech-to-text is genuinely good now on clear audio from a decent microphone. It gets worse, sometimes a lot worse, in exactly the conditions an interview produces: two people talking over each other, a laptop mic picking up the room, a strong accent, and a vocabulary full of words that don't exist in normal English.

Technical terms are the specific weak spot. Library names, acronyms, and words like useEffect, Kubernetes or idempotency are rare in general speech, so a transcriber that's excellent on a podcast can still mangle them. A live copilot that mishears the question answers a different one, confidently. A practice tool that mishears your answer may mark you down for a mistake you didn't make.

The practical check is simple: look at the transcript. A good practice tool shows you what it heard before it judges it, so you can tell the difference between "my answer was wrong" and "my answer was misheard". If you never get to see the transcript, you can't know which one happened.

Layer two: knowing

On a standalone question with a well-known answer, current models are accurate most of the time. Ask for the difference between a process and a thread, what a database index costs you on writes, or why a promise callback runs before a zero-millisecond timeout, and you'll usually get a correct, well-organised answer.

The errors cluster in predictable places, and they're worth knowing because a wrong answer delivered fluently is harder to catch than a hesitant one:

• Version-specific detail. Defaults that changed between major versions, APIs that were deprecated or renamed, framework behaviour that shifted in the last year. Models are trained on years of text written about older versions.
• Plausible APIs that don't exist. A method name that sounds exactly like what the library would call it, delivered in the same confident tone as everything else.
• Averaged advice. "It depends on your use case" followed by a balanced list of options. It's not wrong. It just isn't an answer, and an interviewer hears the difference immediately.
• Numbers. Latency figures, limits and throughput claims are often stated precisely and sourced from nowhere.

For a live copilot, that last point matters more than it looks. You're the one who says the number out loud, so you're the one who owns it when the follow-up asks where it came from.

Layer three: fitting

This is where accuracy stops being a property of the answer and becomes a property of the conversation. A technical screen doesn't grade answers in isolation. It builds the next question out of what you just said, which is covered in more detail in what actually happens in an AI technical screen.

A live copilot can generate a correct answer to the first question. It can't know which project you're about to use as an example, what your team actually decided, or which claim you made two turns ago that the interviewer is now circling back to. So its accuracy falls with every follow-up, and it falls fastest in exactly the part of the interview that decides the result. Do AI interview assistants actually work? walks through that failure and what it costs when someone notices.

Practice tools have their own version of the fitting problem, and it's more subtle, because it shows up as feedback that feels right.

How accurate is the feedback from a practice tool?

A practice assistant has to judge an answer, not just produce one, and judging is harder. Three failure modes come up again and again.

Grading generously. Models are trained to be agreeable, and an agreeable grader tells a candidate that a decent answer was strong. That feels good and teaches nothing. The cost shows up later, in a real round, when an interviewer asks the second "why" your practice tool never asked.

Rewarding fluency over substance. Long, well-structured answers that use the right vocabulary tend to score well whether or not they commit to anything. This is the most dangerous failure, because it trains exactly the habit real interviewers push hardest against.

Confidently wrong corrections. When a practice tool tells you your answer was wrong, it's using the same knowledge layer as everything else, with the same weak spots. On mainstream material its corrections are usually right. On a niche, version-specific or genuinely contested point, it can "correct" you into a worse answer. The defence is to treat a correction as a claim, not a verdict: if it surprises you, check the docs.

There's also one thing no practice tool can judge accurately: whether what you said about your own work is true. It can tell whether your account of a past project is specific and coherent. It can't know whether it happened. That's fine, because the real interviewer can't either, which is why the follow-up is the interview.

A ten-minute accuracy test for any assistant

You don't need benchmarks to find out whether a practice assistant is accurate enough to be worth your time. Give it four answers you already know the right verdict for, and see what it does.

1. Give a confidently wrong answer. Pick something you know well and state a plausible mistake with total conviction, for example that let isn't hoisted at all. An accurate tool catches it and tells you the mechanism. A weak one praises your confidence.
2. Give a fluent, empty answer. Name every relevant concept and commit to nothing, like the first answer in the example above. An accurate tool asks you to choose. A weak one scores it well.
3. Give a correct but unusual answer. Solve the problem in a legitimate way that isn't the textbook one. An accurate tool accepts it, maybe asks you to defend the trade-off. A weak one marks it wrong because it doesn't match what it expected.
4. Say "I don't know" and then reason out loud. An accurate tool gives credit for the reasoning and follows the thread. A weak one treats the opening words as a failed answer and moves on.
5. Watch the next question after each one. This is the real test. If the follow-up builds on what you actually said, the tool is listening. If it's the next item on a list regardless of your answer, it's a quiz with a microphone.

Is an AI interview assistant worth it?

It depends entirely on which kind you mean, and the two answers point in opposite directions.

A live copilot, used during a real interview, usually isn't. Its accuracy is highest on the first question and lowest on the follow-ups that decide the outcome. The pause while you read and the switch in how you speak are visible to anyone paying attention. And the downside, a note on your record at a company you may want to join later, lasts much longer than the upside, one round you might have passed anyway.

A practice assistant can be, if it passes the test above. The honest comparison isn't against a perfect human coach. It's against what most people actually do instead, which is reading lists of questions and never saying an answer out loud until the real interview. Against that baseline, being asked, answering in your own words and getting pushed on it is a large improvement, even from a tool that's occasionally wrong.

A human mock interviewer who knows your stack is still the gold standard. A good one catches things no tool will, and reads your body language and nerves. They're also expensive, hard to schedule, and rarely available at eleven at night the week before your screen. For most people the realistic plan is a practice tool for volume and repetition, plus one or two human mocks if you can get them.

Where a practice tool earns its cost is the gap between knowing something and explaining it under pressure. Eight answers, given twice shows how big that gap usually is. Reading won't close it. Being made to answer, again and again, is how it closes.