How should enterprises evaluate a RAG system before production?

A production RAG system should prove six things: its corpus is approved and traceable; retrieval returns relevant and permitted evidence; answers are faithful to retrieved evidence; the system handles uncertainty and absent evidence safely; corpus and query attacks are tested; and owners can monitor, investigate and reverse material changes.

NIST calls for documented testing, metrics and tools, formal reporting of results, documented risks and ongoing monitoring.

Its Generative AI Profile also calls for reviewing data suitability, documenting fact-checking techniques where needed and conducting regular adversarial testing. These are useful foundations for a RAG release gate.

Write a one-page evaluation charter before collecting test cases.

Define the user groups, business decisions the system may support, permitted source repositories, content classes that must never be retrieved, required citations or source displays, escalation path and actions the system must not take.

The charter should state whether the system is advisory only, whether a human must validate outputs, and what constitutes a harmful answer.

This makes quality thresholds meaningful because a policy summarizer, clinical support workflow and internal IT assistant do not share the same consequence profile.

Build the test set from real task patterns, domain-expert questions, historical support or search logs where appropriate, and deliberately designed edge cases.

OpenAI recommends task-specific evaluations that reflect real-world distributions, with human judgment used to calibrate automated scoring.

NIST similarly expects test sets, metrics and evaluation tooling to be documented.

Each case should retain enough evidence for another reviewer to reproduce the result. Store the question, requesting user role, expected retrieval permissions, approved source passages, expected answer attributes, expected abstention where applicable, risk tag and reviewer decision.

Retrieval must be evaluated independently because generation quality cannot repair missing, irrelevant or unauthorized evidence.

Measure whether the retrieved set contains the approved evidence, whether irrelevant content crowds it out, whether citations resolve to the right passage, and whether retrieval honors the requesting user's access rights.

AWS distinguishes retrieve-only measures from retrieve-and-generate measures and documents metrics for response correctness, completeness, faithfulness, citation precision and citation coverage.

The exact metric names and thresholds may vary by platform, but the evaluation design should preserve the separation between evidence selection and answer generation.

Score answers against explicit criteria, not general impressions.

For each test case, assess correctness, completeness, relevance, faithfulness to retrieved material, citation coverage, clarity and compliance with the required response format.

Where automated or model-based graders are used, validate their agreement with trained human reviewers before scaling them.

A RAG system should be allowed to decline. A no-answer response that explains the limit and routes the user correctly is preferable to a confident answer that fills gaps with unsupported content. Track abstention quality separately so the team does not optimize only for answer rate.

RAG expands the attack surface across ingestion, embedding, vector storage, retrieval, generation and downstream integrations.

OWASP identifies document poisoning, embedding and vector weaknesses, query probing, context-window attacks and indirect prompt injection as material RAG risks.

Access control at the source alone is not enough if ingestion, retrieval filters or displayed citations break the permission model.

Use controlled adversarial cases to test the expected control behavior.

Tests should include hostile instructions embedded in documents, invisible text, conflicting sources, documents crafted to dominate retrieval, user attempts to probe restricted content, oversized context and malformed citations.

Record the result, impact, owner and remediation decision.

Use a held-out case set with typical, edge and adversarial questions. Have domain reviewers label material claims and calibrate automated scoring. For non-RAG tasks, use trusted reference facts or executable checks rather than assuming citations are available.