Setting up RAG over your knowledge base

Flaxxa AI's managed RAG removes the need to operate your own vector DB. Documents go in; cited responses come out.

Creating a collection

AI → Knowledge → New collection. Pick a name (e.g. "support-kb", "product-docs"); the collection is empty.

Per-tenant scoping is automatic if you're running multi-tenant. The Knowledge model is tenant-scoped via the kernel identity layer; cross-tenant retrieval is impossible by construction.

Uploading documents

Supported formats: PDF (incl. scanned with OCR), DOCX, Markdown, HTML, plain text, Notion exports. Drag-and-drop in the UI or POST to the API:

POST /v1/ai/knowledge/{collection_id}/documents
Content-Type: multipart/form-data
{ file: ... }

The pipeline is async. You get a job id back; a webhook fires when indexing finishes. Typical: 30 seconds for a 50-page PDF, 2-3 minutes for a 500-page document.

Chunking strategy

Documents are chunked heading-aware — the chunker respects document structure (headings, sections, lists, tables) instead of cutting arbitrarily at 500 tokens. Overlap is configurable per collection (default: 50 tokens of overlap between adjacent chunks).

For most knowledge bases, the defaults produce better retrieval than custom chunking. Override only when you have a measurable retrieval-quality problem.

Querying a collection

Two patterns:

Standalone query (just retrieval, no chat)

POST /v1/ai/knowledge/{id}/query
{
  "query": "How does cart-abandonment recovery work?",
  "top_k": 5,
  "rerank": true
}

Returns the top-k chunks with source path + page number + retrieval score.

RAG-augmented chat (retrieval + generation in one)

POST /v1/ai/chat
{
  "model": "auto",
  "messages": [{"role": "user", "content": "How does cart-abandonment recovery work?"}],
  "knowledge": "kb_id"
}

The gateway runs retrieval, injects retrieved chunks as context, generates the response. The response carries an array of cited sources you can render to the user.

Optional cross-encoder re-ranker

Setting rerank: true on the query (or per route in the policy) runs a cross-encoder over the top-N retrieved chunks. Costs more per query but lifts retrieval precision sharply (typically +10-15% recall@5 on the cohorts we've measured).

Refresh and invalidation

When you update a document, the gateway re-embeds the changed chunks and invalidates dependent /chat caches automatically. No manual cache-bust step.

Eval harness

Bring 20-50 gold-standard QA pairs ("question we expect, answer we expect"). The eval harness runs them on every collection update and tracks recall@k as a regression-tested metric. Find regressions before they ship.