RAG Metrics

Determines if the response is factually based on the provided context. The groundedness score reflects the proportion of claims in the response answer that can be inferred from the given context over the total number of claims made. A higher score indicates that most claims are backed by the context. A lower score may indicate factual inaccuracies or unfounded claims. This score evaluates the factual alignment of the response to provided context, not whether the statements are true in a more general sense.

$\text{Groundedness} = \frac{|\text{Number of claims in the AI answer that can be inferred from given context}|}{|\text{Total number of claims in the AI answer}|}$

Measures how on-point the retrieved context is. The score represents the fraction of relevant sentences over total sentences retrieved. A high score means most retrieved content directly answers the question. A low score indicates excess or irrelevant information. Note that this score does not address the quality of the answer, it only measures the relevance of the context retrieved given the question.

$\text{Context Relevance} = \frac{|\text{S}|}{|\text{T}|}$

• $|\text{S}|$ is number of sentences relevant to the question.
• $|\text{T}|$ total number of sentences retrieved.