If you have any problems related to the accessibility of any content (or if you want to request that a specific publication be accessible), please contact us at firstname.lastname@example.org.
Generative Adversarial Networks for Synthesizing Medical Images of Multiple Modalities
AuthorKamran, Sharif Amit
AltmetricsView Usage Statistics
Deep learning architectures have revolutionized the field of medical image analysis and consistently achieve state-of-the-art accuracy by learning from high volumes of data. Despite these advances, differential diagnoses could still potentially require a multitude of complicated procedures with potentially adverse effects. For example, fluorescein angiography (FA) for detecting retinal neurovascular abnormalities requires the injection of an exogenous dye to capture retinal vascular structures. This procedure could cause potentially life-threatening allergic reactions to the dye. Although there is only one non-invasive procedure for generating retinal vasculature is optical coherence tomography-angiography (OCTA), this technology is very expensive and only suitable in limited cases – e.g. small macula regions. In addition, limitations in medical imaging technologies result in low-quality and noisy data. As another example, Calcium imaging is used to monitor Ca2+ transient activities in pacemaker population inthe gut, termed: interstitial cells of Cajal (ICC). Ca2+ spatio-temporal maps (STMaps) are utilized to effectively quantity Ca2+ signal events in these cells. These maps contain high levels of sensory noise in these and manual generation and processing of STMaps is intractable. These challenges introduce the necessity for novel approaches to process medical image for developing non-invasive screening protocols with potential for automation. This thesis investigates novel approaches to fundamentally overcome these problems. The main challenge is extracting a representative feature manifold from which higher level of information can be obtained. Accurately establishing a shared feature manifold will potentially improve outcome in the absence of certain measurements, the availability of potentially conflicting data, and under high signal-to-noise ratios (SNR). Generative networks are investigated in this thesis to establish a theoretical platform for extracting shared features from different sensory domains. The proposed architecture incorporates multiple attention-based skip connections in generators and comprises novel residual blocks for both generators and discriminators. Additionally, it employs reconstruction, feature-matching, and perceptual loss along with adversarial training to fundamentally learn shared features across domains. This work is the first in the literature to employ the proposed pipeline for multiple medical imaging modalities while investigating the use of decoding to further enhance semantic segmentation. To showcase the proposed architecture, anatomically accurate fluorescein angiography images are produced from fundus images. In addition, the framework shows significant promise in accurately segmenting extremely noisy Ca2+ STMaps for calcium imaging.