Dynamic Multimodal Augmentation Network (DMAN) for Alzheimer’s Diagnosis

Abstract

Alzheimer’s disease (AD) is still one of the major public health concerns due to its progressive severity and its effects on the economy and population. Precise and early diagnosis of Alzheimer’s disease has not improved significantly over the years, mainly because of the multifactorial nature of the disease and the use of a single imaging modality such as MRI or a cognitive parser. Prior approaches do not effectively incorporate multiple diagnostic features thereby reducing the modality’s sensitivity and specificity. To fill these gaps this study introduces the DMAN, a novel framework that integrates MRI imaging, genetic data analysis, and AR-based cognitive tests for a dynamic, individualized diagnosis of Alzheimer’s. DMAN employs state-of-the-art machine learning methods including MRI examination using 3D Convolutional Neural Networks (3D-CNN), transformation of data acquired through augmented reality (AR) and Transformer models, and meta-learning to apply existing diagnostic architectures to the target population. Single-head and multi-head attention allow for the dynamic spotlighting of each given modality by an individual patient’s profile while preserving interpretability. The proposed framework incorporates a disease progression model across space and time to estimate the future course of Alzheimer’s, which can then be helpful to clinicians at early stages. Moreover, DMAN strengthens the concept of real-time explainability by using the contrastive explanation strategy, thereby improving the trust of the clients. The system has been constructed for operation in the cloud environment, providing extensibility and availability, combined with a mobile application for remote administration and testing using Augmented Reality. Early evaluations show that for DMAN high diagnostic accuracy is feasible, prognosis can be predicted better, and interpretability of results does not pose difficulties. This study offers a novel solution to address key challenges in Alzheimer’s diagnostic system including the use of modalities, adaptability, and explainability, thereby contributing to the advancement of early diagnosis and management of Alzheimer’s, which has a high impact on patient outcome, and better usage of healthcare resources.