How to Train an AI Voice Agent on Your Business Knowledge Base

An AI voice agent is only as reliable as the information and boundaries around it. “Training” normally means preparing approved source material, retrieval rules, prompts, tests and escalation—not teaching the model every fact about the business in one upload.

The goal is a voice system that can answer a defined question, show where its answer came from internally, admit uncertainty and hand off cleanly. The ICO's AI and data-protection guidance and GOV.UK consumer-law guidance for AI agents belong in the knowledge-base brief.

Start with a source inventory

List the documents and systems the agent may use:

Do not index a document simply because it exists. If nobody owns it, the agent has no reliable way to know whether it is current.

Step 1: define the call jobs

Write five to ten jobs in the form:

When this caller context occurs, the agent may take this bounded action using these sources; otherwise it must ask, transfer or stop.

Examples include collecting details for a callback, answering a published service question or offering a documented booking route. “Handle all customer calls” is not a testable job.

Step 2: write answer cards

Each answer card should contain:

• the question it answers;
• the approved answer in plain language;
• source and last-reviewed date;
• terms the caller may use for the same idea;
• exclusions and uncertainty triggers;
• the next action or human route.

Keep answers short enough to speak. A long policy document can remain a source, but the voice layer needs a concise response and a way to offer more detail or transfer.

Step 3: document uncertainty

Give the agent a refusal and repair pattern:

1. say what is known;
2. say what is missing or uncertain;
3. ask one clarifying question;
4. transfer when the boundary is reached.

Never fill a missing price, appointment, entitlement or policy with a plausible guess. A safe “I need to check that with the team” is better than a confident wrong answer.

Step 4: set escalation rules

Transfer when the caller:

• asks for a person;
• disputes a charge or decision;
• sounds distressed or vulnerable;
• requests regulated or sensitive advice;
• cannot be identified safely;
• repeats a misunderstood question;
• asks for an action outside the agent's authority.

Pass the intent, relevant answers already given, collected details and reason for transfer. The human should not need to restart the call from zero.

Step 5: control personal data

Minimise what the agent collects. Decide whether audio, transcript, summary and CRM fields are all necessary. Document provider access, retention, deletion, correction and review routes. Do not put secrets or unnecessary personal data into a prompt or a shared knowledge file.

For marketing calls or follow-up messages, keep the permission and suppression state separate from the service transcript. A caller asking for help has not automatically asked for marketing.

Step 6: test before a live call

Create a test set with:

• ordinary phrasing and regional accents;
• incomplete questions and interruptions;
• two conflicting source documents;
• an out-of-hours request;
• a price or availability question with no current data;
• an upset or vulnerable caller;
• an unsafe request for a refund or account change;
• a request for a human;
• an integration failure and a delayed hand-off.

Score factual correctness, source match, clarity, disclosure, escalation and the final record. Keep the failed examples; they are the next release's training material.

Step 7: maintain the knowledge base

Use an owner and review date for every answer card. A lightweight rhythm works:

• after an incident: add the exact failed question and correction;
• weekly: inspect unanswered and transferred calls;
• monthly: review changed products, hours and policies;
• before a prompt or model change: rerun the critical test set.

Do not claim that a system “learns” simply because a transcript is stored. A controlled update process is safer than silent adaptation.

The release checklist

• source inventory has owners and dates;
• answers are short, current and attributable internally;
• uncertainty and escalation are explicit;
• permissions and retention are documented;
• write actions are limited and tested;
• the critical test set passes;
• a human can pause the agent;
• support staff know what the hand-off contains;
• monitoring separates answer, transfer and complaint outcomes.

FAQ

How many FAQs do I need?

Enough to cover the first defined call jobs and their failure routes. A smaller, maintained set is safer than a large, stale dump.

Does the agent need technical training data?

Not always. Start with approved source material, retrieval and a test set. Fine-tuning or more complex methods may be appropriate later, but they do not replace ownership and evaluation.

Can the agent learn from every call automatically?

Do not enable silent learning by default. Review transcripts, remove unnecessary personal data and approve each knowledge change.

What happens when the answer is wrong?

Give the caller a human route, correct the source or rule, record the failure and rerun the relevant test. The hand-off should protect the customer while the system is improved.

For the wider call architecture, read how AI voice agents handle customer calls. If you need help turning a messy knowledge base into a testable first slice, Get unstuck.