TY - JOUR
T1 - TSFD-Net
T2 - Tissue specific feature distillation network for nuclei segmentation and classification
AU - Ilyas, Talha
AU - Mannan, Zubaer Ibna
AU - Khan, Abbas
AU - Azam, Sami
AU - Kim, Hyongsuk
AU - De Boer, Friso
N1 - Funding Information:
This work was supported in part by the OASIC of Charles Darwin University , Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education ( NRF-2019R1A6A1A09031717 and NRF-2019R1A2C1011297 ), and the US Air Force Office of Scientific Research under Grant number FA9550-18-1-0016 .
Publisher Copyright:
© 2022 The Authors
PY - 2022/7
Y1 - 2022/7
N2 - Nuclei segmentation and classification of hematoxylin and eosin-stained histology images is a challenging task due to a variety of issues, such as color inconsistency that results from the non-uniform manual staining operations, clustering of nuclei, and blurry and overlapping nuclei boundaries. Existing approaches involve segmenting nuclei by drawing their polygon representations or by measuring the distances between nuclei centroids. In contrast, we leverage the fact that morphological features (appearance, shape, and texture) of nuclei in a tissue vary greatly depending upon the tissue type. We exploit this information by extracting tissue specific (TS) features from raw histopathology images using the proposed tissue specific feature distillation (TSFD) backbone. The bi-directional feature pyramid network (BiFPN) within TSFD-Net generates a robust hierarchical feature pyramid utilizing TS features where the interlinked decoders jointly optimize and fuse these features to generate final predictions. We also propose a novel combinational loss function for joint optimization and faster convergence of our proposed network. Extensive ablation studies are performed to validate the effectiveness of each component of TSFD-Net. The proposed network outperforms state-of-the-art networks such as StarDist, Micro-Net, Mask-RCNN, Hover-Net, and CPP-Net on the PanNuke dataset, which contains 19 different tissue types and 5 clinically important tumor classes, achieving 50.4% and 63.77% mean and binary panoptic quality, respectively. The code is available at: https://github.com/Mr-TalhaIlyas/TSFD.
AB - Nuclei segmentation and classification of hematoxylin and eosin-stained histology images is a challenging task due to a variety of issues, such as color inconsistency that results from the non-uniform manual staining operations, clustering of nuclei, and blurry and overlapping nuclei boundaries. Existing approaches involve segmenting nuclei by drawing their polygon representations or by measuring the distances between nuclei centroids. In contrast, we leverage the fact that morphological features (appearance, shape, and texture) of nuclei in a tissue vary greatly depending upon the tissue type. We exploit this information by extracting tissue specific (TS) features from raw histopathology images using the proposed tissue specific feature distillation (TSFD) backbone. The bi-directional feature pyramid network (BiFPN) within TSFD-Net generates a robust hierarchical feature pyramid utilizing TS features where the interlinked decoders jointly optimize and fuse these features to generate final predictions. We also propose a novel combinational loss function for joint optimization and faster convergence of our proposed network. Extensive ablation studies are performed to validate the effectiveness of each component of TSFD-Net. The proposed network outperforms state-of-the-art networks such as StarDist, Micro-Net, Mask-RCNN, Hover-Net, and CPP-Net on the PanNuke dataset, which contains 19 different tissue types and 5 clinically important tumor classes, achieving 50.4% and 63.77% mean and binary panoptic quality, respectively. The code is available at: https://github.com/Mr-TalhaIlyas/TSFD.
KW - Bidirectional feature pyramid
KW - Computational pathology
KW - Deep learning
KW - Medical imaging
KW - Nuclei classification
KW - Nuclei segmentation
UR - http://www.scopus.com/inward/record.url?scp=85127294220&partnerID=8YFLogxK
U2 - 10.1016/j.neunet.2022.02.020
DO - 10.1016/j.neunet.2022.02.020
M3 - Article
C2 - 35367734
AN - SCOPUS:85127294220
VL - 151
SP - 1
EP - 15
JO - Neural Networks
JF - Neural Networks
SN - 0893-6080
ER -