vertebrae.extractors.openclip

Optional OpenCLIP-style vision-language extractor.

Classes

OpenCLIPExtractor

Wrap an OpenCLIP-style model with image/text branch outputs.

Module Contents

class vertebrae.extractors.openclip.OpenCLIPExtractor(name, model_name, pretrained=None, input_modalities=None, outputs=None, batch_size=16, image_mode='auto', alpha_mode='drop', device=None, model_kwargs=None, checkpoint_paths=None, cache_identity=None)[source]

Wrap an OpenCLIP-style model with image/text branch outputs.

Parameters:

  • name (str)

  • model_name (str)

  • pretrained (Optional[str])

  • input_modalities (Optional[Dict[str, str]])

  • outputs (Optional[Sequence[Dict[str, Any]]])

  • batch_size (int)

  • image_mode (str)

  • alpha_mode (str)

  • device (Optional[str])

  • model_kwargs (Optional[Dict[str, Any]])

  • checkpoint_paths (Optional[Sequence[str]])

  • cache_identity (Optional[str])

encode_retrieval(X, *, branch, modality)[source]

Encode one independent image or text endpoint for exact retrieval.

Parameters:

  • X (Any)

  • branch (str)

  • modality (str)

Return type:

numpy.ndarray