vertebrae.cache.local_store
Local filesystem artifact store.
Classes
Store dense/sparse arrays and JSON metadata under a local directory. |
Module Contents
- class vertebrae.cache.local_store.LocalArtifactStore(root='.vertebrae_cache')[source]
Store dense/sparse arrays and JSON metadata under a local directory.
- Parameters:
root (str) – Root cache directory.
- exists(key)[source]
Return whether an embedding artifact exists for key.
- Parameters:
key (str) – Artifact key.
- Returns:
Whether a dense .npy or sparse .npz embedding file exists.
- Return type:
bool
- put_array(key, arr)[source]
Store a dense or sparse embedding matrix.
- Parameters:
key (str) – Artifact key.
arr (Any) – Dense array-like object or scipy sparse matrix.
- Returns:
Filesystem path to the saved artifact.
- Return type:
str
- put_array_batches(key, batches, n_samples, require_complete=True)[source]
Store embeddings from deterministic batches.
- Parameters:
key (str) – Artifact key.
batches (Iterable[Tuple[numpy.ndarray, Any]]) – Iterable of (indices, embeddings) batch pairs.
n_samples (int) – Total number of rows in the full embedding artifact.
require_complete (bool) – Whether every row must be written exactly once.
- Returns:
Filesystem path to the saved artifact.
- Raises:
ValueError – If batches contain duplicate indices, invalid shapes, or incomplete coverage when require_complete is true.
- Return type:
str
- get_array(key)[source]
Load a dense or sparse embedding matrix.
- Parameters:
key (str) – Artifact key.
- Returns:
Dense NumPy array or scipy sparse matrix.
- Return type:
Any
- stat_array(key)[source]
Return the persisted array file size without loading it.
- Parameters:
key (str)
- Return type:
- put_artifact(key, arr, metadata, *, metadata_finalizer=None)[source]
Commit an array and JSON metadata with one last-written manifest.
- Parameters:
key (str)
arr (Any)
metadata (dict)
metadata_finalizer (Optional[Callable[[dict, vertebrae.cache.artifact_store.ArrayArtifactManifest, vertebrae.cache.artifact_store.ArtifactStat], dict]])
- Return type:
str
- put_artifact_batches(key, batches, n_samples, metadata, require_complete=True, *, metadata_finalizer=None)[source]
Commit batched arrays and JSON metadata with one final manifest switch.
- Parameters:
key (str)
batches (Iterable[Tuple[numpy.ndarray, Any]])
n_samples (int)
metadata (dict)
require_complete (bool)
metadata_finalizer (Optional[Callable[[dict, vertebrae.cache.artifact_store.ArrayArtifactManifest, vertebrae.cache.artifact_store.ArtifactStat], dict]])
- Return type:
str
- get_artifact(key)[source]
Load an array and metadata guarded by the same shared read lock.
- Parameters:
key (str)
- Return type:
tuple[Any, dict]
- put_labels_artifact(key, labels, metadata, *, label_names=None, target_type='auto', target_names=None)[source]
Commit labels and their decoding metadata as one generation.
- Parameters:
key (str)
labels (Any)
metadata (dict)
label_names (Optional[Iterable[Any]])
target_type (str)
target_names (Optional[Iterable[str]])
- Return type:
str
- get_labels_artifact(key)[source]
Load labels and decoding metadata under one shared generation lock.
- Parameters:
key (str)
- Return type:
tuple[numpy.ndarray, dict]
- put_labels(key, labels, *, label_names=None, target_type='auto', target_names=None)[source]
Store labels as a JSON artifact.
- Parameters:
key (str) – Artifact key.
labels (Any) – One-dimensional labels.
label_names (Optional[Iterable[Any]])
target_type (str)
target_names (Optional[Iterable[str]])
- Returns:
Filesystem path to the saved labels file.
- Return type:
str
- get_labels(key)[source]
Load labels from a JSON artifact.
- Parameters:
key (str) – Artifact key.
- Returns:
One-dimensional label array.
- Return type:
numpy.ndarray
- put_json(key, obj)[source]
Store JSON metadata for an artifact key.
- Parameters:
key (str) – Artifact key.
obj (dict) – JSON-serializable metadata.
- Returns:
Filesystem path to the saved metadata file.
- Return type:
str