Training overview
"Training" in SiteMesh means giving your chatbot content to answer from. Every piece of content you add is a data source, and a chatbot can have as many as you like.
The four source types
| Type | Best for | Guide |
|---|---|---|
| Website URL | Your public pages: services, pricing, about, blog, FAQ | Websites |
| File upload | Brochures, manuals, price lists, policy documents | Files |
| Pasted text | Anything not published anywhere — internal policies, seasonal notes | Text and YouTube |
| YouTube | Product demos, tutorials, recorded webinars | Text and YouTube |
Mix them freely. A typical setup is one recursive website crawl plus two or three uploaded PDFs.
Training statuses
Each data source moves through a lifecycle you can watch on the Training page:
| Status | Meaning |
|---|---|
| Pending | Queued, not started yet |
| Processing | Being fetched, read and indexed |
| Done | Indexed successfully and available to the chatbot |
| Error | Something went wrong — an error message explains what |
Your chatbot has its own status alongside these:
| Status | Meaning |
|---|---|
| Idle | No content yet — it cannot answer anything |
| Training | At least one source is still processing |
| Ready | Indexed and answering |
| Error | Training failed; check the individual sources |
A chatbot in training still answers using whatever has already finished. You do not have to wait for a large crawl to complete before testing.
Pages and chunks
After training you will see two numbers.
Pages is how many documents were read — web pages crawled, or one per uploaded file.
Chunks is how many searchable pieces those pages were broken into. Each chunk is roughly a thousand characters, with deliberate overlap between neighbours so no sentence gets orphaned by a split.
Chunks are what the chatbot actually searches, so the chunk count is the better measure of how much your bot knows. A 40-page site producing 600 chunks is healthy; 40 pages producing 45 chunks means most of those pages were nearly empty.
How much content is enough?
More is not automatically better. A focused knowledge base beats a sprawling one, because every irrelevant chunk is one more thing competing to be retrieved.
Good practice:
- Include the pages that answer real customer questions — services, pricing, policies, contact details, FAQ, how-to content
- Skip thin pages with no substance: tag archives, pagination, image galleries, login pages
- Split very large documents into focused ones rather than uploading a single 400-page manual
Removing content
Deleting a data source removes its indexed content too — the chatbot stops being able to answer from it immediately. There is no separate "unlearn" step.
This is the right move when you have taken a product off sale or a policy has been withdrawn. Leaving stale content indexed is how a chatbot ends up quoting a discount you no longer offer.
Keeping it current
Content is indexed when you crawl it, not read live from your site. When your site changes, the index needs to catch up — either manually or on a schedule.
When answers are not good enough
There is a diagnostic playbook for this — start with the crawl summary rather than rewriting your system prompt.