vertebrae.extractors.huggingface_vision

Optional Hugging Face vision embedding extractor.

Classes

HFVisionExtractor

Hugging Face vision backbone extractor with explicit pooling.

Module Contents

class vertebrae.extractors.huggingface_vision.HFVisionExtractor(name, model_id, processor_id=None, pooling='cls', hidden_layer=None, outputs=None, batch_size=16, image_mode='auto', alpha_mode='drop', device=None, revision=None, trust_remote_code=False, processor_kwargs=None, model_kwargs=None, spatial_outputs=None, structured_outputs=None, checkpoint_paths=None, cache_identity=None)[source]

Hugging Face vision backbone extractor with explicit pooling.

Parameters:

  • name (str) – User-facing extractor name.

  • model_id (str) – Hugging Face model identifier or local path.

  • processor_id (Optional[str]) – Optional Hugging Face processor identifier or local path. Defaults to model_id.

  • pooling (str) – Pooling mode: “cls”, “mean”, or “pooler”.

  • hidden_layer (Optional[int]) – Optional hidden-state layer index to pool from. Defaults to the model’s final output.

  • batch_size (int) – Number of images encoded per batch.

  • image_mode (str) – Image representation to pass to the processor: “auto”, “rgb”, “grayscale”, or “preserve”.

  • alpha_mode (str) – How alpha channels are handled when converting: “drop”, “white_background”, or “black_background”.

  • device (Optional[str]) – Optional device string.

  • revision (Optional[str]) – Optional model revision.

  • trust_remote_code (bool) – Whether to allow remote model code.

  • processor_kwargs (Optional[Dict[str, Any]]) – Extra keyword arguments for AutoImageProcessor.

  • model_kwargs (Optional[Dict[str, Any]]) – Extra keyword arguments for AutoModel.

  • outputs (Optional[List[Dict[str, Any]]])

  • spatial_outputs (Optional[List[Dict[str, Any]]])

  • structured_outputs (Optional[List[Dict[str, Any]]])

  • checkpoint_paths (Optional[List[str]])

  • cache_identity (Optional[str])

fit(X, y=None)[source]

No-op fit for frozen Hugging Face vision models.

Parameters:

  • X (Any) – Image inputs.

  • y (Any) – Optional labels.

Returns:

This extractor.

Return type:

HFVisionExtractor

transform(X)[source]

Encode image inputs into dense embeddings.

Parameters:

X (Any) – PIL images, NumPy image arrays, image paths, or a sequence of them.

Returns:

Dense float32 embedding matrix.

Raises:

  • ImportError – If optional Hugging Face vision dependencies are missing.

  • ValueError – If pooling is invalid for the model output.

Return type:

numpy.ndarray

fit_transform(X, y=None)[source]

Encode image inputs into dense embeddings.

Parameters:

  • X (Any) – Image inputs.

  • y (Any) – Optional labels.

Returns:

Dense float32 embedding matrix.

Return type:

numpy.ndarray

recipe()[source]

Return a serializable Hugging Face vision recipe.

Returns:

JSON-compatible recipe dictionary.

Return type:

Dict[str, Any]

get_resource_profile_adapter()[source]

Return Torch profiling hooks without forcing model loading.

Return type:

Any