Patient-Centric Statistical Multi-Modal Fusion for Medical Diagnosis: Integrating DICOM, Radiomics, and Patient Attributes

Abstract

Deep learning has led to substantial progress in medical image analysis, particularly for disease classification. However, integrating patient-specific attributes, including age, body mass index, and lifestyle factors, with radiomics and raw imaging data remains a key challenge for personalized diagnosis. We propose a novel multi-modal framework, Statistically Coherent Network (SCN), which jointly models imaging, radiomic, and patient metadata through a structured multi-space latent representation. SCN encourages distributional coherence across subpopulations using a statistics-based loss combined with triplet loss, aligning feature distributions among clinically similar cohorts. Across multiple clinically diverse tasks, SCN consistently outperforms conventional single-space and multi-modal baselines.