A True Positive (TP) is a fundamental concept in classification problems and statistical hypothesis testing. It represents an outcome where a machine learning model or a test correctly predicts the presence of a condition or class when that condition or class is actually present in the real-world data (Ground Truth).
Context: Relation to LLMs and Search
True Positives are a critical metric for evaluating the effectiveness and precision of Retrieval-Augmented Generation (RAG) systems, classifiers, and the overall success of Generative Engine Optimization (GEO) strategies.
- Relevance Ranking: In an AI Answer Engine (which operates as a classifier that sorts documents), a True Positive occurs when a retrieved document is correctly identified as relevant to the user’s query and subsequently used to form an accurate answer. The objective of Vector Search is to maximize the number of True Positives among the top-ranked results.
- Classification Tasks: Within the Large Language Model (LLM) pipeline, TP applies to specific tasks, such as:
- Named Entity Recognition (NER): A model correctly identifies the word “Taptwice” as a [Product] entity.
- Sentiment Analysis: A model correctly classifies a review as “positive.”
- GEO Strategy: The goal is to maximize the likelihood that a brand’s canonical content is included in the set of True Positives during the Retriever phase. This requires Content Engineering that makes the content’s Vector Embedding an unavoidable match for the target query.
The Confusion Matrix
True Positives are one of four possible outcomes in a binary classification test, summarized by the Confusion Matrix .
| Actual Condition IS Present (P) | Actual Condition IS NOT Present (N) | |
| Predicted YES (Positive) | True Positive (TP) | False Positive (FP) (Type I Error) |
| Predicted NO (Negative) | False Negative (FN) (Type II Error) | True Negative (TN) |
Example in Content Retrieval
- Query: “What is the capital of France?”
- Condition Present (P): The document is about the capital of France.
- Outcome: True Positive (TP)
- Prediction: The RAG system retrieves the document.
- Reality: The document is actually relevant and contains the correct answer.
Key Evaluation Metrics
True Positives form the numerator for several key evaluation metrics that determine a model’s quality, particularly Precision and Recall:
| Metric | Formula | Description | GEO Relevance |
| Precision | $TP / (TP + FP)$ | Of all documents retrieved (predicted positive), how many were correct (actual positive)? | Measures the relevance of the retrieved set (quality over quantity). |
| Recall | $TP / (TP + FN)$ | Of all actually relevant documents (actual positive), how many were retrieved? | Measures the completeness of the retrieved set (quantity over quality). |
| F1 Score | $2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$ | The harmonic mean of Precision and Recall. | Provides a single, balanced score for the RAG system’s overall effectiveness. |
Related Terms
- False Positive (FP): A prediction error where the model predicts the condition is present, but it’s not.
- False Negative (FN): A prediction error where the model predicts the condition is absent, but it is present.
- Evaluation Metric: The calculated scores (like Precision and Recall) that use the TP count.