Characterization of Foundation Models for Longitudinal Similarity Measurement in Medical Video Data

Neubig L, Larsen D, Ikuma T, Kunduk M, Kist A (2026)

Publication Type: Conference contribution

Publication year: 2026

Journal

Informatik aktuell Gesellschaft für Informatik (GI)

Publisher: Springer Science and Business Media Deutschland GmbH

Pages Range: 252-258

Conference Proceedings Title: Informatik aktuell

Event location: Lübeck DE

ISBN: 9783658510992

DOI: 10.1007/978-3-658-51100-5_51

Abstract

Foundation models provide general-purpose image representations that promise to capture structural and semantic information. However, their suitability for measuring similarity across image sequences has not been thoroughly examined. Conventional metrics such as the structural similarity index measure (SSIM) are commonly used to assess frame-to-frame consistency but are sensitive to motion, deformation, and intensity changes, which limits their usefulness for dynamic imaging. In this study, we compare embeddings from a variety of pretrained models, including DINOv2, ResNet50, CLIP, SAM, and LPIPS, to evaluate their ability to represent temporal and structural similarity in videos and their ability to assess the quality of image registration. We analyzed sensitivity to global and local motion in two medical imaging datasets. We focused on videofluoroscopic swallowing studies (VFSS) with global and local motion and the BAGLS dataset of vocal fold vibrations with mainly local motion. Our results indicate differences in how models maintain consistent similarity under motion, and suggest that some embedding-based approaches provide a more stable representation than SSIM without additional fine-tuning.

Authors with CRIS profile

Luisa Neubig Juniorprofessur für Artificial Intelligence in Communication Disorders Andreas Kist Juniorprofessur für Artificial Intelligence in Communication Disorders

Involved external institutions

East Carolina University US United States (USA) (US) Louisiana State University Health Sciences Center US United States (USA) (US) Louisiana State University US United States (USA) (US)

How to cite

APA:

Neubig, L., Larsen, D., Ikuma, T., Kunduk, M., & Kist, A. (2026). Characterization of Foundation Models for Longitudinal Similarity Measurement in Medical Video Data. In Heinz Handels, Katharina Breininger, Thomas Deserno, Andreas Maier, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff (Eds.), Informatik aktuell (pp. 252-258). Lübeck, DE: Springer Science and Business Media Deutschland GmbH.

MLA:

Neubig, Luisa, et al. "Characterization of Foundation Models for Longitudinal Similarity Measurement in Medical Video Data." Proceedings of the Bildverarbeitung für die Medizin Workshop, BVM 2026, Lübeck Ed. Heinz Handels, Katharina Breininger, Thomas Deserno, Andreas Maier, Klaus Maier-Hein, Christoph Palm, Thomas Tolxdorff, Springer Science and Business Media Deutschland GmbH, 2026. 252-258.

BibTeX: Download