vertebrae.execution.jobs

Distributed-ready job and shard helpers.

Classes

`ShardSpec`	Deterministic non-overlapping sample shard specification.
`SampleBatch`	A deterministic batch of sample references.
`EmbeddingJob`	Description of a future distributed embedding job.
`ResourceSpec`	Resource request for local or distributed work.
`EmbeddingShardJob`	Executable embedding shard job.
`EmbeddingMergeJob`	Executable embedding merge job.
`CompressionJob`	Description of a compression job over persisted embeddings.
`ScoringJob`	Description of a scoring job over persisted embeddings.
`RetrievalScoringJob`	Score persisted query/gallery embeddings against a relevance artifact.
`RetrievalEmbeddingShardJob`	Materialize one query or gallery endpoint shard for a retrieval dataset.
`RetrievalCompressionJob`	Fit compression on a gallery artifact and transform its paired query artifact.
`ZeroShotEmbeddingShardJob`	Materialize one sample or prompt endpoint shard for zero-shot evaluation.
`ZeroShotCompressionJob`	Fit sample-side compression and transform paired zero-shot prompts.
`ZeroShotScoringJob`	Score paired zero-shot endpoint artifacts against a prompt protocol.
`SeparatixJob`	Description of a Separatix diagnostic job over persisted embeddings.
`StabilityJob`	Run stability analysis over persisted embeddings and targets.

Module Contents

class vertebrae.execution.jobs.ShardSpec[source]

Deterministic non-overlapping sample shard specification.

Parameters:

total_shards – Total number of shards in the embedding job.
shard_index – Zero-based index for this shard.

owns(sample_index)[source]

Return whether this shard owns a sample index.

Parameters:: sample_index (int) – Zero-based sample index.
Returns:: Whether the sample belongs to this shard.
Return type:: bool

indices(n_samples)[source]

Return all sample indices owned by this shard.

Parameters:: n_samples (int) – Total number of samples in the dataset.
Returns:: One-dimensional array of non-overlapping sample indices.
Return type:: numpy.ndarray

property is_complete: bool[source]

Return whether this spec covers the complete dataset.

Return type:: bool

class vertebrae.execution.jobs.SampleBatch[source]

A deterministic batch of sample references.

indices[source]: Original dataset row indices for this batch.

X[source]: Batch inputs sliced from the dataset.

class vertebrae.execution.jobs.EmbeddingJob[source]

Description of a future distributed embedding job.

dataset_id[source]: Dataset identity key.

extractor_id[source]: Extractor identifier or fingerprint.

recipe_hash[source]: Hash of the extractor recipe.

shard[source]: Deterministic shard assignment.

output_uri[source]: Destination embedding artifact URI or path.

class vertebrae.execution.jobs.ResourceSpec[source]

Resource request for local or distributed work.

cpus[source]: Number of CPU cores requested.

memory_bytes[source]: Optional memory budget for the job.

gpus[source]: Number of GPUs requested.

gpu_memory_bytes[source]: Optional GPU memory budget.

walltime_seconds[source]: Optional walltime limit for schedulers such as SLURM.

queue[source]: Optional queue, partition, or scheduling lane.

class vertebrae.execution.jobs.EmbeddingShardJob[source]

Executable embedding shard job.

dataset[source]: Dataset object available to the worker.

extractor[source]: Feature extractor available to the worker.

shard[source]: Deterministic shard assignment.

output_key[source]: Artifact-store key for the shard output.

batch_size[source]: Number of samples per transform batch.

resources[source]: Resource request for the shard.

class vertebrae.execution.jobs.EmbeddingMergeJob[source]

Executable embedding merge job.

shard_keys[source]: Artifact-store keys for shard outputs.

output_key[source]: Artifact-store key for the merged embedding artifact.

n_samples[source]: Expected full dataset sample count.

resources[source]: Resource request for the merge.

class vertebrae.execution.jobs.CompressionJob[source]: Description of a compression job over persisted embeddings.

class vertebrae.execution.jobs.ScoringJob[source]

Description of a scoring job over persisted embeddings.

embedding_key[source]: Artifact-store key for embeddings.

labels_key[source]: Artifact-store key or URI for labels.

groups_key[source]: Optional artifact-store key for aligned independence groups.

output_key[source]: Artifact-store key for scoring results.

scoring_config[source]: Optional OverlapIndex scoring configuration.

metrics[source]: Optional custom embedding metrics. OverlapIndex is always included.

primary_metric[source]: Metric name selected for aggregate score collection.

seed[source]: Optional scoring seed, commonly used for stability repeats.

resources[source]: Resource request for scoring.

class vertebrae.execution.jobs.RetrievalScoringJob[source]: Score persisted query/gallery embeddings against a relevance artifact.

class vertebrae.execution.jobs.RetrievalEmbeddingShardJob[source]: Materialize one query or gallery endpoint shard for a retrieval dataset.

class vertebrae.execution.jobs.RetrievalCompressionJob[source]: Fit compression on a gallery artifact and transform its paired query artifact.

class vertebrae.execution.jobs.ZeroShotEmbeddingShardJob[source]: Materialize one sample or prompt endpoint shard for zero-shot evaluation.

class vertebrae.execution.jobs.ZeroShotCompressionJob[source]: Fit sample-side compression and transform paired zero-shot prompts.

class vertebrae.execution.jobs.ZeroShotScoringJob[source]: Score paired zero-shot endpoint artifacts against a prompt protocol.

class vertebrae.execution.jobs.SeparatixJob[source]: Description of a Separatix diagnostic job over persisted embeddings.

class vertebrae.execution.jobs.StabilityJob[source]: Run stability analysis over persisted embeddings and targets.