A Study on the Classification of Cancers with Lung Cancer Pathological Images Using Deep Neural Networks and Self-Attention Structures

Abstract

In this paper, we propose a ResNet-based lung cancer pathology image classification model using deep neural networks and self-attention modules. With a shortcut structure that adds input as an output, which is ResNet's concept, we not only solve the vanishing gradient problem but also perform well even when layers are piled densely. Based on this idea, the pre-activation structure in which the output enters the input as it is was used by moving the position of batch normalization and activation function in front of the weight layer was used. ResNet's bottleneck structure is made up of layers with 1x1, 3x3, and 1x1 convolution layers, which are utilized for depth wise convolution and 1x1 convolution, respectively, to conduct convolution operations in the channel direction. In addition, channel attention and spatial attention were used as self-attention modules that focus on certain features after the bottleneck structure. Finally, batch normalization and activation functions were used, and after using the Funnel Activation function considering two-dimensional space as the activation function, the model is constructed with a fully connected layer with average pooling and activation function as sigmoid. The accuracy, precision, recall, and f1-score of our method are 82.83%, 83%, 84.14%, and 83.56 respectively. We show through experiments that our method is better than existing ResNet-based models.