vertebrae.scoring.metrics

Generic embedding metric interfaces and adapters.

Classes

MetricResult	A normalized, rankable result returned by an embedding metric.
EmbeddingMetric	Protocol for metrics that evaluate a full batch of embeddings.
CallableMetric	Adapt a callable into an EmbeddingMetric.
OverlapMetric	Built-in metric adapter for OverlapIndex-family scoring.
LabelRetrievalMetric	Leave-one-out exact retrieval derived from single-label targets.

Functions

as_embedding_metric(metric)	Normalize a metric object or callable into the embedding metric protocol.
load_metric_callable(path)	Load an importable metric callable from module:attribute syntax.

Module Contents

class vertebrae.scoring.metrics.MetricResult

A normalized, rankable result returned by an embedding metric.

to_dict()

Serialize this result to JSON-compatible data.

Return type:

Dict[str, Any]

class vertebrae.scoring.metrics.EmbeddingMetric

Bases: Protocol

Protocol for metrics that evaluate a full batch of embeddings.

score(embeddings, labels, *, target_metadata=None, groups=None, seed=None)

Return one aggregate result for an embedding matrix.

Parameters:

• embeddings (Any)

• labels (Any)

• target_metadata (Optional[Dict[str, Any]])

• groups (Optional[Any])

• seed (Optional[int])

Return type:

MetricResult

recipe()

Return a JSON-compatible recipe when available.

Return type:

Dict[str, Any]

class vertebrae.scoring.metrics.CallableMetric

Adapt a callable into an EmbeddingMetric.

The callable receives embeddings and labels plus any supported keyword arguments from target_metadata, groups, and seed. It may return a MetricResult, a mapping containing score, or a numeric score.

recipe()

Return enough metadata to reconstruct importable metrics on workers.

Return type:

Dict[str, Any]

classmethod from_import_path(path, *, name=None, config=None, higher_is_better=True, cache_identity=None)

Build a portable callable metric from module:attribute syntax.

Parameters:

• path (str)

• name (Optional[str])

• config (Optional[Dict[str, Any]])

• higher_is_better (bool)

• cache_identity (Optional[str])

Return type:

CallableMetric

class vertebrae.scoring.metrics.OverlapMetric

Built-in metric adapter for OverlapIndex-family scoring.

with_config(config)

Return a resolved metric without mutating a caller-owned metric object.

Parameters:

config (Any)

Return type:

OverlapMetric

score(embeddings, labels, *, target_metadata=None, groups=None, seed=None)

Run the internal OverlapIndex adapter through the generic metric API.

Parameters:

• embeddings (Any)

• labels (Any)

• target_metadata (Optional[Dict[str, Any]])

• groups (Optional[Any])

• seed (Optional[int])

Return type:

MetricResult

class vertebrae.scoring.metrics.LabelRetrievalMetric

Leave-one-out exact retrieval derived from single-label targets.

vertebrae.scoring.metrics.as_embedding_metric(metric)

Normalize a metric object or callable into the embedding metric protocol.

Parameters:

metric (Any)

Return type:

EmbeddingMetric

vertebrae.scoring.metrics.load_metric_callable(path)

Load an importable metric callable from module:attribute syntax.

Parameters:

path (str)

Return type:

Callable[Ellipsis, Any]