Your AI Voice Agent Is Only as Smart as Its Knowledge Base. Here Is How to Structure One.

The agent answered a question we never wrote down, and it was wrong. That is what a missing knowledge base sounds like on a live call.

A caller asked our agent about weekend pricing. We had never given it weekend pricing. So it guessed. The caller booked on a number that did not exist. We found out when they turned up annoyed.

That call cost about 40 cents to run. The cleanup cost an hour. The fix was not a smarter model. It was a better knowledge base.

We build voice agents for New Zealand and Australian businesses every week. The single biggest accuracy lever is how the AI voice agent knowledge base is structured. Get it right and the agent stops making things up.

The knowledge base is the agent's source of truth on every call.

What is an AI voice agent knowledge base?

An AI voice agent knowledge base is the structured set of facts the agent reads from before it answers a caller. It holds your prices, hours, services, locations and policies. The agent does not invent answers. It looks them up, then speaks them.

Think of it as the agent's source of truth. The model supplies fluent speech. The knowledge base supplies the facts. When a caller asks what you charge to unblock a drain, the answer comes from your file, not the model's imagination.

This matters because language models are confident even when wrong. Left alone, they fill gaps with plausible guesses. A good knowledge base removes the gaps.

If you want the bigger picture first, read our overview of how our AI voice agents handle real calls.

How do you structure it so the agent answers accurately?

Structure it in short, self-contained chunks, one fact per chunk, written the way a caller asks. Each chunk should answer one question fully without needing the chunk before it. The agent retrieves the closest match and reads it.

Long documents are the enemy. A 12-page PDF of terms is hard for the agent to search mid-call. Break it into 30 small entries. One for after-hours pricing. One for the cancellation window. One for the service area.

Here is the pattern we use in every build.

We also keep numbers exact. Not about 100 dollars. The real figure. A vague answer on a live call sounds like a dodge, and callers hear it.

Break long documents into small one-fact entries the agent can retrieve mid-call.

What should you put in, and what should you leave out?

Put in the 30 to 50 questions callers actually ask, plus your prices, hours, locations and booking rules. Leave out marketing fluff, internal process notes and anything you cannot say out loud to a stranger on the phone.

Start from real calls. Pull the last 50 enquiries and list every question. That list is your knowledge base. It is shorter than you expect and more useful than any brochure.

Leave out stale promotions. A finished special that still lives in the file will get quoted on a live call. Leave out staff names that change. Leave out anything covered by privacy rules that the agent has no reason to read aloud.

One rule we hold firm. If a fact is not in the knowledge base, the agent says it will find out and arrange a callback. It does not guess. That single behaviour prevents most of the damage.

How does a good knowledge base stop the agent hallucinating?

It stops hallucination by giving the agent a real answer to retrieve, so it never has to invent one. When the file is complete and well chunked, the agent reads facts instead of generating them. Gaps are where guesses come from.

We pair this with a hard instruction. When no chunk matches the question, the agent admits it does not know and offers a callback. No improvising. We cover the full method in our guide on how to stop an AI voice agent from hallucinating.

In testing, a complete knowledge base cut wrong answers to near zero across a 200-call sample. The few misses were questions we had never logged. We added them, and the gap closed.

How fast do updates take effect on live calls?

Updates take effect almost immediately. You change a price in the knowledge base, save it, and the next call reads the new figure. There is no retraining and no overnight rebuild. The agent reads the current file every time.

That speed matters in real businesses. A clinic that books out at 4pm can flip availability the moment it changes. A trade that lifts its callout fee on Monday has the agent quoting the new fee on Monday.

We go deeper on timing in our piece on how fast knowledge base updates reach live calls. The short version. Minutes, not days.

One owner, one weekly pass, and call records feeding the file keep it current.

How do you keep it current?

Keep it current with a short weekly review and a trigger for every price or policy change. Assign one owner. When anything changes in the business, it changes in the knowledge base the same day. Stale facts are the most common cause of wrong answers.

We also read the call records. Post-call analysis surfaces the questions the agent fumbled. Each fumble is a missing or outdated entry. Feed it back in and the file gets sharper every week.

One owner, one weekly pass, one rule that price changes update the file first. That is the whole maintenance routine. It takes about 20 minutes a week.

How do you test that it is right?

Test it by calling the agent with your 30 hardest questions and checking every answer against the file. Run the list before launch and after every major update. If one answer drifts, fix the chunk, not the model.

We script the test like a mystery shopper. Awkward phrasing. Edge cases. Questions with no answer, to confirm the agent offers a callback instead of guessing. A good script makes this easy, which is why scripts and knowledge bases are built together.

Read the transcripts afterwards. The agent's exact words tell you whether the file is clear. If two chunks compete, the answer wobbles, and you will hear it.

Where this fits in the bigger picture

The knowledge base is one layer of a working voice agent. The script controls flow. The knowledge base controls facts. The platform controls how it all runs on a live call. They only work together.

If you are mapping out a build, start with how an AI voice agent works on a call. Then write the script. Then build the knowledge base from real enquiries. In that order.

On data and trust

Your structured records, transcripts and the knowledge base itself sit on our servers in Sydney. Live call audio is processed offshore by our voice infrastructure partner under documented arrangements. We never claim all data stays in Australia, because that would not be true.

Every agent discloses it is AI at the start of the call. That is not optional. It keeps you onside with the NZ Privacy Act 2020 and the Office of the Privacy Commissioner. It also keeps you onside with the Australian Privacy Act 1988, the 13 Australian Privacy Principles and the OAIC.

A clean knowledge base helps here too. It contains the facts the agent should say and excludes the ones it should not. That is privacy by design, on a phone call.

Frequently Asked Questions

How big should an AI voice agent knowledge base be?

Most NZ and AU small businesses need 30 to 50 entries. That covers prices, hours, services, locations, booking rules and the common questions callers ask. Bigger is not better. A tight, accurate file beats a sprawling one, because the agent retrieves the right answer faster and drifts less.

Can the agent answer questions that are not in the knowledge base?

No, and that is the point. When no entry matches, the agent says it does not know and offers a callback. It does not guess. This single rule prevents wrong prices and fake bookings, which are the costliest mistakes a phone agent can make.

How long does it take to set up a knowledge base?

A focused first version takes a few hours once you have pulled your real enquiries. We build it from your last 50 calls, write each entry as a question and answer, then test against your hardest questions. Most of the work is gathering facts, not configuring software.

What happens when my prices change?

You update the knowledge base and the next call quotes the new price. No retraining, no delay. We recommend one owner who updates the file the same day any price or policy changes. Stale entries are the top cause of wrong answers on live calls.

Does a knowledge base work for both inbound and outbound calls?

Yes. Inbound agents use it to answer caller questions accurately. Outbound agents use it to answer the questions prospects raise mid-pitch. The same source of truth keeps both honest. A 200-dial campaign costs about 100 dollars NZD, so accuracy on every call protects that spend.

How do I know the knowledge base is actually right?

Test it like a mystery shopper before launch and after updates. Call with your 30 hardest questions, including ones with no answer, and check every reply against the file. Then read the transcripts. The agent's exact words show you which chunks are clear and which compete.