Federated Learning Framework for Privacy-Safe Breast Imaging AI

Paper: Evaluating Federated Vision-Language and Vision-Only Models for Privacy-Aware Breast Cancer Imaging

Authors: Malithi Mithsara Wanniarachchi Kankanamge, Ning Yang, Sree Bash Debnath, Md Zobaer Islam, Abdur R. Shahid

Executive Summary

Recent research explores the use of artificial intelligence (AI) in breast imaging, addressing challenges like limited labeled data and privacy concerns that impede centralized training. The study introduces a federated learning framework that prioritizes privacy, integrating pretrained vision-language models (VLMs) with client-level differential privacy. This approach allows for the analysis of data from multiple sources while keeping sensitive information secure.

The research evaluates various models, including general-purpose CLIP models, a specialized biomedical model called BiomedCLIP, and a baseline model trained on ImageNet. These models were tested on two publicly available breast imaging datasets. The evaluation criteria included patient-level performance metrics, specifically the ROC-AUC, which measures the model's ability to distinguish between normal and sick cases, along with other factors like model communication costs and sensitivity to thresholds.

Under the federated learning setup, all five models exhibited strong performance in distinguishing between normal and sick cases, achieving ROC-AUC scores between 0.940 and 0.990. However, the study also highlights potential confounding factors, noting that simply using the year of acquisition for images achieved a comparable ROC-AUC score of 0.974. This raises questions about whether the AI models are genuinely recognizing underlying pathology or if they are influenced by other variables.

The ability to differentiate between benign and malignant cases proved more challenging, with ROC-AUC scores ranging from 0.504 to 0.637. Importantly, all confidence intervals included the value of 0.50, indicating uncertainty in the models' effectiveness for this classification task.

The research also examines the impact of different components of the models. For instance, removing the text pathway from the CLIP model maintained classification accuracy while significantly reducing communication costs by half. Conversely, the implementation of differential privacy resulted in a notable decrease in ROC-AUC scores by 0.375. This decline is attributed to the approach of applying perturbations to the entire model state rather than limiting changes to only the trainable parameters.

These findings emphasize the necessity of assessing confounding factors, ensuring predictive reliability, and evaluating the efficiency of privacy implementations when developing federated AI systems for breast imaging. The insights gained from this research could inform future developments in AI applications for medical imaging, particularly in enhancing privacy while maintaining accuracy.

Academic Abstract

Artificial intelligence (AI) shows promise for breast imaging, but limited labeled data, institutional data fragmentation, and privacy concerns hinder centralized training. We investigate a privacy-aware federated learning framework combining pretrained vision-language models (VLMs) with client-level differential privacy. We compare general-purpose CLIP models, biomedical BiomedCLIP, and an ImageNet-pretrained ViT-B/16 baseline on two public breast imaging datasets, with and without differential privacy. We also evaluate a CLIP ViT-B/16 ablation that removes the text pathway. Evaluation includes patient-level ROC-AUC with bootstrap 95% confidence intervals, baseline comparisons, threshold sensitivity, inter-model agreement, and communication cost. Under federated learning, all five models achieve high normal-versus-sick discrimination (ROC-AUC: 0.940-0.990). However, acquisition year alone achieves 0.974 on the same test partition, raising concerns about confounding rather than genuine pathology recognition. Benign-versus-malignant discrimination remains uncertain (ROC-AUC: 0.504-0.637), with all confidence intervals including 0.50. Removing the text pathway preserves classification accuracy while halving communication cost. Differential privacy reduces ROC-AUC by 0.375, attributable to perturbing the full model state rather than only trainable parameters. These findings highlight the importance of evaluating confounding, predictive reliability, privacy implementation, and deployment efficiency when developing federated AI systems for breast imaging.