# retrieval and matching

`vertebrae` can compare frozen embedding backbones under an explicit exact
query--gallery ranking protocol. This is complementary to OverlapIndex:
OverlapIndex measures global labeled representation separation, while retrieval
measures whether declared relevant candidates rank near the top for each query.

Use `RetrievalDataset` to provide independent query and gallery inputs plus either a
NumPy/scipy grade matrix or sparse `(query_id, gallery_id, grade)` records. Non-matrix
iterables, including nested Python lists, are parsed as relevance records; use
`RetrievalDataset.from_relevance_matrix(...)` for a nested-list matrix. Sparse matrices
are normalized from their nonzeros without densification. Relevance grades must be
finite and non-negative; grades above zero are relevant for binary metrics. Missing
pairs have grade zero, and every query must retain at least one eligible positive after
exclusions. Equal query/gallery IDs are valid matches unless explicitly excluded.

`RetrievalBenchmark` supports precomputed embeddings, ordinary same-modality
extractors, and explicit branch encoders through `CallableRetrievalExtractor` or the
retrieval-capable OpenCLIP, SigLIP, and compatible Hugging Face multimodal adapters.
The default is exact cosine ranking with NDCG@10 as the primary score. Reports include
NDCG, precision, recall, hit rate, MRR, mAP, and positive/negative similarity margins.
Cosine retrieval requires every query and gallery embedding to have a nonzero L2 norm;
scoring fails with the affected endpoint and row identities when this requirement is
violated. Zero vectors remain valid for dot-product and squared-L2 retrieval.

Pass `MemoryConfig` to `RetrievalBenchmark` to bound local endpoint materialization.
Streaming-safe extractors are admitted progressively; when the resident budget is
exceeded, embeddings spill to temporary local staging if `allow_disk_spill=True`, or
the run fails before accumulating the full endpoint when spill is disabled.

Optional paired compression is fitted on gallery embeddings and applied to query
embeddings with the same transform. Requests whose `n_components` is greater than or
equal to the endpoint width are explicit no-ops: both endpoints retain their values,
sparsity, and dtype, and result metadata records `applied=False` with a warning. The
same rule applies in local and artifact-backed retrieval workflows.

The protocol is deliberately not an ANN, reranking, learned retrieval, or recommender
benchmark. It evaluates the fixed embedding geometry and declared candidate set only.

Bidirectional artifact scoring matches local evaluation: score artifacts contain a
`forward` result, an optional `reverse` result, and their averaged `primary_score`.
Reconstruction requires identical retrieval configuration and protocol fingerprints.