WMP Copilot multi-tenant RAG on AWS Bedrock

One knowledge base, 100+ tenants

Under Bedrock's constraints, a knowledge base per tenant would have meant duplicating everything 100+ times. I kept one and isolated tenants with metadata-based filtering.

The tenant identity in that filter is validated on the server. A request cannot name its own tenant.

AI workloads kept apart from the core

The core systems are Java and the AI services are Python. I built them as separate microservices so that AI workloads are decoupled from the core. Each side is easier to maintain, and each can be deployed without waiting on the other.

Measuring retrieval and generation separately

I built an evaluation framework around a golden test set, sorted into 7 question categories. Retrieval is scored with Precision@K, Recall@K and MRR. Generation is scored on faithfulness and hallucination rate.

Scoring them apart showed where the problem was: retrieval averaged 3.06 out of 5, generation 3.77. The per-category breakdown drove two changes, a parser migration and dynamic top-k routing.

Cost and latency

The AI services run in 128 MB pods, which keeps cold starts under 2 seconds. Tuning the chunking strategy, retrieval depth and model selection brought inference cost down to $0.019 a query.

Knowing what it did

Every query, retrieval result, token count and guardrail trigger is logged in a structured form. That is what makes per-tenant cost attribution and anomaly detection possible.

Related writing

• Building a RAG System: The Decisions That Actually Mattered