vertebrae.execution.zero_shot

Artifact-backed primitives for frozen zero-shot evaluation.

Functions

zero_shot_embedding_artifact_key(dataset, extractor, ...)	Build a stable endpoint key while allowing sample artifacts to be prompt-reused.
zero_shot_protocol_artifact_key(dataset)	Build the canonical serialized prompt-protocol artifact key.
zero_shot_compression_artifact_key(...)	Build a paired compression prefix for zero-shot endpoints.
zero_shot_scoring_artifact_key(sample_embedding_key, ...)	Build a stable zero-shot scoring artifact key for resolved evaluation settings.
materialize_zero_shot_protocol(dataset, store[, key])	Persist labels and fixed prompt declarations independently of model outputs.
materialize_zero_shot_embedding_shard(job, store)	Materialize one deterministic zero-shot endpoint shard.
merge_zero_shot_embedding_shards(job, store)	Merge one endpoint after validating its side, branch, and provenance.
compress_zero_shot_embedding_artifacts(job, store)	Fit compression on sample embeddings and transform prompt embeddings.
score_zero_shot_artifact(job, store)	Score persisted frozen endpoints and contextual sample overlap.
score_zero_shot_artifacts(jobs, store, execution)	Submit independent zero-shot scoring jobs through local, Ray, or Dask execution.
plan_zero_shot_embedding_shard_jobs(dataset, ...[, ...])	Plan deterministic endpoint jobs for one zero-shot side.
zero_shot_benchmark_result_from_artifacts(score_keys, ...)	Reconstruct a rankable zero-shot report from persisted score artifacts.

Module Contents

vertebrae.execution.zero_shot.zero_shot_embedding_artifact_key(dataset, extractor, side, branch)

Build a stable endpoint key while allowing sample artifacts to be prompt-reused.

Parameters:

• dataset (Any)

• extractor (Any)

• side (str)

• branch (str)

Return type:

str

vertebrae.execution.zero_shot.zero_shot_protocol_artifact_key(dataset)

Build the canonical serialized prompt-protocol artifact key.

Parameters:

dataset (Any)

Return type:

str

vertebrae.execution.zero_shot.zero_shot_compression_artifact_key(sample_embedding_key, prompt_embedding_key, config)

Build a paired compression prefix for zero-shot endpoints.

Parameters:

• sample_embedding_key (str)

• prompt_embedding_key (str)

• config (Any)

Return type:

str

vertebrae.execution.zero_shot.zero_shot_scoring_artifact_key(sample_embedding_key, prompt_embedding_key, protocol_key, zero_shot_config=None, scoring_config=None)

Build a stable zero-shot scoring artifact key for resolved evaluation settings.

Parameters:

• sample_embedding_key (str)

• prompt_embedding_key (str)

• protocol_key (str)

• zero_shot_config (Optional[vertebrae.config.ZeroShotConfig])

• scoring_config (Optional[vertebrae.config.OverlapScoringConfig])

Return type:

str

vertebrae.execution.zero_shot.materialize_zero_shot_protocol(dataset, store, key=None)

Persist labels and fixed prompt declarations independently of model outputs.

Parameters:

• dataset (Any)

• store (vertebrae.cache.ArtifactStore)

• key (Optional[str])

Return type:

dict

vertebrae.execution.zero_shot.materialize_zero_shot_embedding_shard(job, store)

Materialize one deterministic zero-shot endpoint shard.

Parameters:

• job (vertebrae.execution.jobs.ZeroShotEmbeddingShardJob)

• store (vertebrae.cache.ArtifactStore)

Return type:

dict

vertebrae.execution.zero_shot.merge_zero_shot_embedding_shards(job, store)

Merge one endpoint after validating its side, branch, and provenance.

Parameters:

• job (vertebrae.execution.jobs.EmbeddingMergeJob)

• store (vertebrae.cache.ArtifactStore)

Return type:

dict

vertebrae.execution.zero_shot.compress_zero_shot_embedding_artifacts(job, store)

Fit compression on sample embeddings and transform prompt embeddings.

Parameters:

• job (vertebrae.execution.jobs.ZeroShotCompressionJob)

• store (vertebrae.cache.ArtifactStore)

Return type:

dict

vertebrae.execution.zero_shot.score_zero_shot_artifact(job, store)

Score persisted frozen endpoints and contextual sample overlap.

Parameters:

• job (vertebrae.execution.jobs.ZeroShotScoringJob)

• store (vertebrae.cache.ArtifactStore)

Return type:

dict

vertebrae.execution.zero_shot.score_zero_shot_artifacts(jobs, store, execution)

Submit independent zero-shot scoring jobs through local, Ray, or Dask execution.

Parameters:

• jobs (Iterable[vertebrae.execution.jobs.ZeroShotScoringJob])

• store (vertebrae.cache.ArtifactStore)

• execution (Any)

Return type:

list[dict]

vertebrae.execution.zero_shot.plan_zero_shot_embedding_shard_jobs(dataset, extractor, total_shards, *, side, branch, batch_size=128, resource_profiling_config=None, output_key=None, run_prefix=None)

Plan deterministic endpoint jobs for one zero-shot side.

Parameters:

• dataset (Any)

• extractor (Any)

• total_shards (int)

• side (str)

• branch (str)

• batch_size (int)

• resource_profiling_config (Optional[vertebrae.config.ResourceProfilingConfig])

• output_key (Optional[str])

• run_prefix (Optional[str])

Return type:

list[vertebrae.execution.jobs.ZeroShotEmbeddingShardJob]

vertebrae.execution.zero_shot.zero_shot_benchmark_result_from_artifacts(score_keys, store, output_key=None)

Reconstruct a rankable zero-shot report from persisted score artifacts.

Alignment and overlap results remain separate fields; the ranking always follows the configured zero-shot primary metric recorded in each score artifact.

Parameters:

• score_keys (Iterable[str])

• store (vertebrae.cache.ArtifactStore)

• output_key (Optional[str])

Return type:

Any