Skip to main content

Improving answers

When your chatbot is not answering well, resist the urge to start rewriting the system prompt. Nine times out of ten the problem is the knowledge base, not the instructions.

Work through these in order.

"I don't have information about that" — but it is on my site​

This is the most common complaint, and it has a short list of causes.

1. Check the page was actually indexed​

Open the data source and read the crawl summary. Look at skipped and failed. If the page you expected is in either list, the chatbot never saw it.

  • Failed — the page timed out or errored. Try refreshing the source.
  • Skipped — the page had too little text, or was a duplicate or non-HTML file.

2. Check the content is really text​

Look at the page in a browser and try to select the text of the relevant part. If you cannot select it, it is an image, and the chatbot cannot read it.

Prices in a graphic, opening hours in a banner image, a specification table rendered as a screenshot — all invisible. Add the same information as pasted text.

3. Check the chunk count​

A page count in the dozens with a chunk count in single figures means very little usable text was extracted. See sparse content.

4. Consider how the information is phrased​

The chatbot matches on meaning, but meaning has to be present. If a visitor asks "how long does delivery take?" and your site only says "orders ship promptly", there is no fact to retrieve.

The fix: write the explicit answer as a pasted text source. This is the single most effective repair available to you.

The answer is vague or partially right​

Usually a retrieval dilution problem: too much loosely related content competing, so the best passage does not make it through.

  • Split large documents. A 300-page manual as one file produces hundreds of chunks that all look mildly relevant. Split it by topic.
  • Delete thin content. Tag archives, near-empty pages and boilerplate crowd the index without adding facts.
  • Add a focused source. A short, direct pasted-text answer will usually outrank a paragraph buried on page 40 of a PDF.

The bot answers, but in the wrong voice​

Now — and only now — look at the chatbot's configuration.

Do not remove the knowledge base placeholder

If you edit the system prompt, keep {{context_chunks}} in it. That placeholder is where your retrieved content gets inserted. Remove it and the chatbot has nothing to answer from — it will say it does not know, every single time.

The bot is too eager to say it does not know​

That behaviour is deliberate — it is what stops the chatbot inventing policies you never wrote. It is not a bug to be tuned away.

If it triggers too often, the answer is more content, not looser rules. Go through the live inbox, collect the questions it declined, and write those answers as pasted text.

Answers reference something that no longer exists​

Old content is still indexed. Delete the data source that carries it — refreshing alone will not always remove content you deleted from the site.

→ Keeping content fresh

Answers are slow or shallow on complex questions​

Consider switching model. gpt-4o-mini is the default and is fast and inexpensive; gpt-4o reasons better on multi-part questions at higher cost.

→ Choosing a model

Use real conversations as your to-do list​

The live inbox shows what people actually asked, in their own words. It is a far better guide to what is missing than any amount of guessing.

A good monthly habit:

  1. Read the last month of conversations
  2. Note every question the bot could not answer
  3. Write those answers as a single pasted-text source
  4. Retrain

Two or three rounds of that and most chatbots get noticeably better.

Thumbs up and thumbs down​

Visitors can rate individual replies in the widget. The satisfaction view aggregates those ratings, which points you at specific bad answers rather than a general feeling that something is off.