<\/span><\/h3>\n\n\npython<\/p>\n\n\n\n
import torch\nfrom transformers import AutoProcessor, AutoModel\nfrom PIL import Image\n\nmodel_id = \"google\/medsiglip-448\"\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = AutoModel.from_pretrained(model_id).to(device).eval()\nprocessor = AutoProcessor.from_pretrained(model_id)\n\nimage = Image.open(\"example_cxr.png\").convert(\"RGB\")\nlabels = [\n \"Normal chest X-ray without acute findings.\",\n \"Large right-sided pneumothorax with mediastinal shift.\",\n \"Bilateral pleural effusions with cardiomegaly.\"\n]\n\ninputs = processor(text=labels, images=image, return_tensors=\"pt\", padding=True)\ninputs = {k: v.to(device) for k, v in inputs.items()}\n\nwith torch.no_grad():\n outputs = model(**inputs)\n image_embeds = outputs.image_embeds\n text_embeds = outputs.text_embeds\n\nimage_embeds = image_embeds \/ image_embeds.norm(dim=-1, keepdim=True)\ntext_embeds = text_embeds \/ text_embeds.norm(dim=-1, keepdim=True)\n\nlogits = (image_embeds @ text_embeds.T).squeeze(0)\nprobs = logits.softmax(dim=-1)\n\npred_idx = int(probs.argmax())\nprint(labels[pred_idx], float(probs[pred_idx]))<\/code><\/pre>\n\n\n\nThis is not production\u2011grade, but it’s enough to benchmark zero-shot performance on de\u2011identified validation sets.<\/p>\n\n\n
<\/span>Analysis Examples: Radiology (X-Ray), Pathology, and Dermatology<\/span><\/h3>\n\n\nIn my own experiments:<\/p>\n\n\n\n
\n- Radiology<\/strong> (X\u2011ray):<\/strong> MedSigLIP performs well in distinguishing normal vs. clearly abnormal CXRs and picking up large effusions or obvious pneumothorax. It struggles more with subtle interstitial disease and early edema without tailored prompts.<\/li>\n\n\n\n
- Pathology (<\/strong>WSI<\/strong>tiles<\/strong>):<\/strong> after tiling whole\u2011slide images into 224\u2013448 px crops, zero-shot prompts like “high\u2011grade carcinoma” vs. “benign inflammatory changes” give a useful screening signal. I still rely on task\u2011specific fine\u2011tuning for any serious clinical use.<\/li>\n\n\n\n
- Dermatology:<\/strong> the model can separate inflammatory vs. neoplastic lesions reasonably well from clinical photos, but I’ve seen misclassifications on darker skin tones, an important fairness red flag that must be quantified before deployment.<\/li>\n<\/ul>\n\n\n\n
In all three domains, I treat MedSigLIP as a prior: something to guide sampling, triage, or downstream model training, never as the final diagnostic decision-maker.<\/p>\n\n\n
<\/span>Integration & Production Patterns<\/span><\/h2>\n\n\nOnce I’m happy with offline metrics (AUROC, calibration, subgroup performance), I focus on how MedSigLIP fits into a full pipeline.<\/p>\n\n\n