MedSigLIP

Medical Image-Text Encoder for Zero-Shot Classification

A 400M parameter dual-tower encoder for medical image classification, retrieval, and zero-shot inference tasks.

🔍

400M

400M Parameters

448×448

448×448 Images

64-token Text

About MedSigLIP

MedSigLIP MedSigLIP is a lightweight 400M parameter dual-tower encoder (vision + text) that supports 448×448 images and up to 64-token text inputs. Released on July 9, 2025, as part of Google's Health AI Developer Foundations project.

The model is trained on diverse medical imaging data including chest X-rays, dermatology, ophthalmology, pathology slides, and CT/MRI scans along with their corresponding reports. Natural images are also included to maintain generalization capabilities.

MedSigLIP is specifically designed for data-efficient classification, zero-shot classification, and semantic image retrieval tasks. For text generation tasks, Google recommends using MedGemma instead.

📅

Release Information

Health AI Developer Foundations

July

2025

2025-07-09 UTC

Model Architecture & Specifications

Built on SigLIP foundation with medical specialization

Parameters

~400M

Dual-tower architecture with vision and text encoders

Image Input

448×448

High-resolution medical image processing

Text Input

64 tokens

Medical text and report understanding

Training Data

Multi-modal

Medical images + reports + natural images

Training Data Coverage

✓Chest X-rays and radiology reports

✓Dermatology images and descriptions

✓Ophthalmology scans and findings

✓Pathology slides and annotations

✓CT/MRI scans and interpretations

✓Natural images for generalization

Recommended Use Cases

Optimized for classification and retrieval tasks

Primary Applications

📊

Data-Efficient Classification

Train classifiers with minimal labeled medical data using pre-trained representations

🎯

Zero-Shot Classification

Classify medical images without task-specific training using text descriptions

🔍

Semantic Image Retrieval

Find relevant medical images using natural language queries

Not Recommended For

For text generation tasks, use MedGemma instead

⚠️Medical report generation
⚠️Conversational medical AI
⚠️Clinical decision support requiring text output

Have a photo of a visible symptom?

The models described on this page are research tools for clinical imaging. If what you have is an everyday photo — a rash, a mole, a nail, a wound — AI Doctor Scan reads it and returns a free urgency level and a plain-language summary in about 30 seconds. It does not name a diagnosis, and it needs no account.

Try AI Doctor Scan — free

Performance & Benchmarks

Competitive performance across medical imaging tasks

Performance metrics vary by specific medical domain and task configuration. Refer to the official model card for detailed benchmarks.

Implementation Guide

Get started with MedSigLIP in your projects

Access the Model

Download MedSigLIP from the official repository or use via API

# Example: Loading MedSigLIP
from transformers import AutoModel, AutoProcessor

model = AutoModel.from_pretrained("google/medsiglip")
processor = AutoProcessor.from_pretrained("google/medsiglip")

Prepare Your Data

Format medical images (448×448) and text descriptions (≤64 tokens)

💡Resize images to 448×448 pixels

💡Keep text descriptions under 64 tokens

💡Use clear, medical terminology in text

Implement Your Use Case

Choose from classification, zero-shot inference, or retrieval applications

🚀Fine-tune for specific classification tasks

🚀Use embeddings for similarity search

🚀Implement zero-shot classification pipelines

Resources & Documentation

Official links and community resources

Official Resources

MedSigLIP Documentation

Complete documentation and API reference

Model Card

Detailed model specifications and performance metrics

GitHub Repository

Code examples, notebooks, and implementation guides

Health AI Developer Foundations

Complete HAI-DEF project overview

Community & Analysis

BiopharmaTrend Analysis

Industry analysis of MedGemma and MedSigLIP release

AI Magazine Coverage

In-depth coverage of Google's healthcare AI initiatives

MedSigLIP vs MedGemma

Choose the right model for your use case

MedSigLIP

Lightweight Encoder

✓Smaller model size (~400M parameters)
✓Faster inference for classification tasks
✓Excellent for retrieval and similarity search
✓Zero-shot classification capabilities
✓Lower computational requirements

Classification, retrieval, and embedding tasks

MedGemma

Generative Models

✓Text generation capabilities
✓Conversational medical AI
✓Report generation and summarization
✓Complex reasoning tasks
✓Multiple model sizes (4B, 27B)

Text generation, conversation, and complex reasoning

Compliance & Limitations

Important considerations for medical AI deployment

Medical Disclaimer

MedSigLIP is a research model and is not intended for direct clinical decision-making. All medical AI applications require proper validation, regulatory compliance, and human oversight.

Validation Required

Thorough testing and validation needed before clinical deployment

Regulatory Compliance

Ensure compliance with local healthcare regulations and standards

Human Oversight

Medical professionals must review and validate all AI-generated outputs

Data Privacy

Follow HIPAA and other privacy regulations when handling medical data