General Purpose Metrics

The recall score measures how well a response covers the requested input. It is calculated by the proportion of required content included. A high score indicates a comprehensive response, while a low score indicates an incomplete response. This score reflects the thoroughness of the information provided.

$\text{Recall} = \frac{|\text{Number of claims in the AI answer that address the user query}|}{|\text{Total number of distinct, relevant claims that could be made to directly answer the user query}|}$

Evaluates the specificity and directness of the information provided in the response. The precision score is based on the ratio of relevant claims to the total content in the response. A high score signifies that the content is specifically relevant to the input with minimal extraneous information. A low score indicates the presence of superfluous or less relevant details. This score focuses on the accuracy and relevance of the details within the answer, not the volume of information.

$\text{Precision} = \frac{|\text{Number of claims in the AI answer that directly address the user query}|}{|\text{Total number of claims in the AI answer}|}$

Determines the logical flow, consistency, and rationality of the response. A high score means that the content is highly logical and coherent, with a clear, consistent, and rational progression of ideas from beginning to end, free of any significant contradictions or illogical statements. A low score signifies that the content demonstrates significant logical inconsistencies or contradictions, lacking a coherent flow of ideas.