Intersection Over Union (IoU) is a metric used to evaluate the similarity and spatial overlap between a predicted region (a bounding box) and a ground-truth region (the true bounding box). It is a key performance measure in computer vision tasks like object detection and image segmentation.
IoU is calculated as the area of the intersection of the two regions divided by the area of their union. A higher IoU score indicates a better prediction that more closely matches the true location and size of the object.
Context: Relation to LLMs and Search Relevance
While IoU is primarily a computer vision metric, the underlying concept of measuring the overlap of two sets or regions has conceptual parallels in Natural Language Processing (NLP), particularly in evaluating search and Generative Engine Optimization (GEO) performance.
- Conceptual Analogy in Search:
- Predicted Region: The set of documents retrieved by a Neural Search (Vector Search) system in response to a query.
- Ground Truth Region: The set of truly Relevant documents for that query (the human-annotated Labels).
- IoU Metric: Metrics like Jaccard Similarity (which is mathematically identical to IoU for sets) are used to measure the overlap between the set of retrieved documents and the set of relevant documents. High overlap means the Large Language Model (LLM)‘s retrieval component is working effectively.
- Evaluation in Question Answering (QA): IoU’s principle is directly applied in evaluating the performance of Extractive Question Answering (QA) models (often based on LLM encoders like BERT). In this task, the model must predict the start and end Tokens of the answer span within a document.
- Predicted Region: The predicted answer span (a sequence of tokens).
- Ground Truth Region: The actual answer span (the gold-standard sequence).
- Metric: Metrics like F1-Score are more common for text sequences, but the goal is the same as IoU: maximize the overlap (the common tokens) between the predicted and true sequences.
The IoU Formula
IoU is calculated using the following mathematical formula:
$$\text{IoU} = \frac{\text{Area of Intersection}}{\text{Area of Union}}$$
Interpretation of Scores:
- IoU = 1.0: Perfect overlap. The predicted box is identical to the ground-truth box.
- IoU = 0.5: The intersection area is equal to the area of the two boxes that do not overlap.
- IoU = 0.0: No overlap between the predicted and ground-truth boxes.
A typical threshold for a correct detection is often set at an IoU of 0.5 or 0.75.
Related Terms
- Jaccard Similarity: The set-based metric that is mathematically equivalent to IoU.
- F1-Score: A common metric in NLP used to evaluate the overlap of predicted and true sequences in QA tasks.
- Object Detection: The primary computer vision task where IoU is used.
- Bounding Box: The rectangular region whose overlap is measured by IoU.