Precision (also known as positive predictive value) is a crucial evaluation metric in information retrieval, Text Classification, and machine learning. It measures the quality or accuracy of a system’s results by calculating the proportion of retrieved or predicted items that are actually Relevant (correct). In essence, it answers the question: Out of all the items the system said were relevant, how many were truly relevant?
Context: Relation to LLMs and Search
Precision is one of the two core metrics (along with Recall) used to evaluate the performance of the Retrieval and Ranking Algorithm stages in a Retrieval-Augmented Generation (RAG) system, making it vital for Generative Engine Optimization (GEO).
- Quality of Context: In a RAG pipeline, the system must retrieve a fixed number of document chunks to fit into the LLM’s Context Window. High precision means that most of the chunks fed to the LLM are, in fact, relevant to the user’s query. Low precision means the context is filled with irrelevant “noise,” which can distract the LLM and lead to inaccurate or corrupted Generative Snippets.
- Mitigating Noise: The goal of the Reranking step is to maximize the precision of the top results by filtering out irrelevant documents that may have passed the initial, broader Vector Search filter.
- Factual Accuracy: Since the LLM is instructed to answer only based on the provided context, the precision of that context is a direct determinant of the factual accuracy of the final answer. Low precision increases the risk of the LLM prioritizing irrelevant facts or struggling to synthesize a coherent answer.
The Mechanics: The Formula
Precision is defined mathematically as:
$$\text{Precision} = \frac{\text{Number of Relevant Documents Retrieved}}{\text{Total Number of Documents Retrieved}}$$
This can also be expressed in terms of True Positives (TP) and False Positives (FP):
$$\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}}$$
- True Positive (TP): A relevant document that was correctly retrieved.
- False Positive (FP): An irrelevant document that was incorrectly retrieved (a false alarm or noise).
Precision vs. Recall
Precision and Recall are often conflicting goals. Optimizing for one typically comes at the expense of the other.
| Feature | Precision | Recall |
| Focus | Accuracy (Minimizing noise/False Positives). | Completeness (Minimizing misses/False Negatives). |
| Question | Out of the answers the system found, how many were correct? | Out of all the answers that exist, how many did the system find? |
| Strategy | Narrow the search scope (e.g., use Reranking). | Broaden the search scope (e.g., use Query Expansion). |
A system with perfect precision (1.0) guarantees that every item it retrieved was correct, but it may have missed many other relevant items (low recall).
Related Terms
- Recall: The sister metric to precision, measuring the completeness of the retrieved set.
- Reranking: The process designed specifically to improve the precision of the top-ranked retrieved items.
- F1-Score: A combined metric that uses the harmonic mean of precision and recall to provide a balanced measure of performance.