From Neonatal to Adult Brain MR Image Segmentation in a Few Seconds Using 3D-Like Fully Convolutional Network and Transfer Learning

From LRDE

Revision as of 22:57, 6 February 2017 by Xu (talk | contribs)

Abstract

Brain magnetic resonance imaging (MRI) is widely used to assess brain development in neonates and to diagnose a wide range of neurological diseases in adults. Such studies are usually based on quantitative analysis of different brain tissues, so it is essential to be able to classify accurately these tissues. In this paper, we propose a fast automatic method that segments 3D brain MR images into different tissues using fully convolutional network (FCN) and transfer learning. As compared to existing deep learning-based approaches that rely either on 2D patches or on fully 3D FCN, our method is way much faster: it only takes a few seconds, and only a single modality (T1 or T2) is required. In order to take the 3D information into account, the set of all 3 successive 2D slices are stacked to form a set of 2D color images, which serve as input for the FCN pre-trained on ImageNet for natural image classification. To the best of our knowledge, this is the first method that attempts to apply transfer learning for segmenting both neonatal and adult brain 3D MR images, and we show that the proposed method achieves state-of-the-art results on two public datasets.

Method and datasets

Method

Architecture of the proposed network. We fine tune it and combine linearly fine to coarse feature maps of the pre-trained VGG network. The coarsest feature maps are discarded for the adult images.

Xu.17.icip-pepeline.png

Datasets

  • Dataset of the MICCAI challenge of Neonatal Brain Segmentation 2012 (NeoBrainS12)
    • Axial images acquired at 40 weeks: 2 training images + 5 test images
    • Coronal images acquired at 30 weeks: 2 training images + 5 test images
    • Coronal images acquired at 40 weeks: 5 test images
  • Dataset of the MICCAI challenge of MR Brain Image Segmentation (MRBrainS13)
    • Axial images acquired at 70 years: 5 training images + 15 test images

Materials

Trained models

The trained models and corresponding files for training for the proposed method on NeoBrainS12 and MRBrainS13 datasets are available in the following:

  • Training on Axial images at 40 weeks in NeoBrainS12 dataset are available in this archive
  • Training on coronal images at 30 weeks in NeoBrainS12 dataset are available in this archive
  • Training on previous training sets for coronal images at 40 weeks in NeoBrainS12 dataset are available in this archive
  • Training on MRBrainS13 dataset is available in this archive

Segmentation results

The pre-computed segmentation results of the proposed method on NeoBrainS12 and MRBrainS13 datasets are available in the following:

  • Results on Axial images at 40 weeks in NeoBrainS12 dataset are available in this archive
  • Results on coronal images at 30 weeks in NeoBrainS12 dataset are available in this archive
  • Results on coronal images at 40 weeks in NeoBrainS12 dataset are available in this archive
  • Results on MRBrainS13 dataset are available in this archive

Illustrations

Experiments

Leave-One-Subject-Out (LOSO) cross-validation on N images + normal training/test experiments. Note that only one training image is used for LOSO 2. Xu.17.icip-experiments.jpg

LOSO experiments

Quantitative results of LOSO experiments in terms of Dice coefficient as compared to the state-of-the-art results. The last one is from P. Moeskops et al. on the 15 test images in MRBrainS13 dataset.

Xu.17.icip-losoresults.jpg

  • Some results on axial images at 40 weeks in NeoBrainS12 dataset
  • Some results on coronal images at 30 weeks in NeoBrainS12 dataset
  • Some results on axial images of aging adults at 70 ages in MRBrainS13 dataset

Neonatal brain MR image segmentation

  • Results on axial images at 40 weeks in NeoBrainS12 dataset. More details can be found Here

Xu.17.icip-axial40results.jpg

Some result images


  • Results on coronal images at 30 weeks in NeoBrainS12 dataset. More details can be found Here

Xu.17.icip-coronal30results.jpg

Some result images (some small errors inside the red circle)


  • Results on coronal images at 40 weeks in NeoBrainS12 dataset. More details can be found Here

Xu.17.icip-coronal40results.jpg

Some result images (some small errors inside red circles)

Adult brain MR image segmentation

  • Results on aging adult images at 70 years in MRBrainS13 dataset. Only top 10 methods among 38 submitted ones are shown. More results and details can be found Here

Xu.17.icip-adult70results.jpg

Some result images

Bibtex (lrde.bib)

@Article{	  xu.17.icip,
  author	= {Yongchao Xu and Thierry G\'eraud and Isabelle Bloch},
  title		= {From Neonatal to Adult Brain MR Image Segmentation in a
  Few Seconds Using 3D-Like Fully Convolutional Network and Transfer
  Learning},
  journal	= {Submitted for publication},
  year		= 2017,
  project	= {Image},
  abstract	= {Brain magnetic resonance imaging (MRI) is widely used to assess
                  brain development in neonates and to diagnose a wide range of
                  neurological diseases in adults. Such studies are usually based on
                  quantitative analysis of different brain tissues, so it is essential
                  to be able to classify accurately these tissues.  In this paper, we
                  propose a fast automatic method that segments 3D brain MR images
                  into different tissues using fully convolutional network (FCN) and
                  transfer learning.  As compared to existing deep learning-based
                  approaches that rely either on 2D patches or on fully 3D FCN, our
                  method is way much faster: it only takes a few seconds, and only a
                  single modality (T1 or T2) is required.  In order to take the 3D
                  information into account, the set of all 3 successive 2D slices are
                  stacked to form a set of 2D color images, which serve as input for
                  the FCN pre-trained on ImageNet for natural image classification.
                  To the best of our knowledge, this is the first method that attempts
                  to apply transfer learning for segmenting both neonatal and adult
                  brain 3D MR images, and we show that the proposed method achieves
                  state-of-the-art results on two public datasets.},
  note		= {Submitted},
  optlrdepaper	= {https://www.lrde.epita.fr/dload/papers/xu.2017.icip.pdf}
		  
}