Automatic breast lesion detection and classification is an important task in computer-aided diagnosis, in which breast ultrasound (BUS) imaging is a common and frequently used screening tool. Recently, a number of deep learning-based methods have been proposed for joint localization and classification of breast lesion using BUS images. In these methods, features extracted by a shared network trunk are appended by two independent network branches to achieve classification and localization. Improper information sharing might cause conflicts in feature optimization in the two branches and leads to performance degradation. Also, these methods generally require large amounts of pixel-level annotated data for model training. To overcome these limitations, we proposed a novel joint localization and classification model based on the attention mechanism and disentangled semi-supervised learning strategy.

Methodology

The proposed framework is shown in Figure 1 (A). The model used in this study is composed of a classification network and an auxiliary lesion-aware network. By use of the attention mechanism, the auxiliary lesion-aware network can optimize multi-scale intermediate feature maps and extract rich semantic information to improve classification and localization performance. The disentangled semi-supervised learning strategy only requires incomplete training datasets for model training. The proposed modularized framework allows flexible network replacement to be generalized for various applications.

Results

Experimental results on two different breast ultrasound image datasets, a public dataset and another dataset collected at Mayo Clinic, demonstrate the effectiveness of the proposed method. The impacts of various network factors on model performance are also investigated to gain deep insights into the designed framework. The classification performances obtained by different feature extraction network architectures are shown in the Figure 1 (B), while Figure 1 (C) and (D) shows the improved localization performance and interoperability of extract feature maps.

Related publications

1. Zong Fan, Ping Gong, Shanshan Tang, Christine U. Lee, Xiaohui Zhang, Shigao Chen, Pengfei Song, Hua Li*. Joint localization and classification of breast masses on ultrasound images using a novel auxiliary attention-based framework. In Proceedings.