Publications/xu.hierarchymsll

Authors

Yongchao Xu, Thierry Géraud, Laurent Najman

Journal

Submitted

Type

article

Keywords

Image

Date

2015-09-16

Abstract

Hierarchies, such as the tree of shapes, are popular representations for image simplification and segmentation thanks to their multiscale structures. Selecting meaningful level lines (boundaries of shapes) yields to simplify image while preserving intact salient structures. Many image simplification and segmentation methods are driven by the optimization of an energy functional, for instance the celebrated Mumford-Shah functional. In this paper, we propose an efficient approach to hierarchical image simplification and segmentation based on the minimization of some energy functional. This method conforms to the current trend that consists in producing hierarchical results rather than a unique partition. Contrary to classical approaches which compute optimal hierarchical segmentations from an input hierarchy of segmentations, we rely on the tree of shapes, a unique and well-defined representation equivalent to the image. Simply put, we compute for each level line of the image an attribute function that characterizes its persistence under the energy minimization. Then we stack the level lines from meaningless ones to salient ones through a saliency map based on extinction values defined on the tree-based shape space. Qualitative illustrations and quantitative evaluation on Weizmann segmentation evaluation database demonstrate the state-of-the-art performance of our method.

Materials

Mumford-Shah Simplification on the Color Tree of Shapes

You can download the x86_64 binary to compute the Mumford-Shah simplification running on the color tree of shapes Here.

Usage: ./mumford_shah_on_ctos input[rgb] α₀ α₁ λ output[rgb]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)
λ	Mumford-shah regularisation weight (e.g. 5000)

• 

  Input RGB Image

• 

  Output Simplified Image

• 

  Input RGB Image

• 

  Output Simplified Image

Saliency Map Computation Relying on Mumford-Shah-Salient Level Line Selection

You can download the x86_64 binary to compute the saliency map representing hierarchical image simplification and segmentation Here. This application outputs the saliency map as a float image. The simplification and segmentation result can be obtained by thresholding this float image. Note that the image is twice as big as the original one and has a border for topogical and algorithmic purposes. Thus, any pixel with coordinates (x,y) in the original image is now at coordinates (2*(x+1), 2*(y+1)) in the saliency map.

Usage: ./saliency_map_mumford_ctos input[rgb] α₀ α₁ output[float]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)

In the previous binary, we use the absolute difference between values of neighboring pixels to compute the average of gradient's magnitude, which is used to sort the shapes. An alternative is to use a gradient image given by a sophisticated contour detection method (e.g., the Structured Edge) to compute the average of gradient's magnitude. You can download this x86_64 binary Here.

Usage: ./saliency_map_mumford_ctos_grad input[rgb] α₀ α₁ gradSE.pgm output[float]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)
gradSE 	Gradient image given by a contour detection method (Gpb or SE)

• 

  Input RGB Image

• 

  Saliency map

Illustrations

Saliency map computation on BSDS500 database

The first test was performed on the BSDS500 database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

• From top to bottom: input images; saliency maps; slight/moderate/strong simplification.
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Saliency map computation on Weizmann database

The second test was performed on the Weizmann database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

• From top to bottom: input images; saliency maps; slight/moderate/strong simplification.
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Bibtex (lrde.bib)

@Article{	  xu.hierarchymsll,
  author	= {Yongchao Xu and Thierry G\'eraud and Laurent Najman<nowiki>},
  title		= {Hierarchical image simplification and segmentation based on Mumford-Shah-salient level line selection},
  journal	= {Submitted for publication},
  year		= 2015,
  project	= {Image},
  abstract	= {Hierarchies are popular representations for image simplification and
                  segmentation thanks to their multiscale structures. An example is the
                  tree of shapes. Selecting meaningful level lines (boundaries of
                  shapes) yields to simplify image while preserving intact salient
                  structures. Many image simplification and segmentation methods are
                  driven by the optimization of an energy functional, for instance the
                  Mumford-Shah functional. In this paper, we propose an efficient
                  approach of hierarchical image simplification and segmentation based
                  on the minimization of some energy functional.  This method conforms
                  to the current trends that are to find hierarchical results rather
                  than a unique partition. Contrary to classical approaches which
                  compute optimal hierarchical segmentations from an input hierarchy of
                  segmentations, we rely on the tree of shapes, a unique and equivalent
                  image representation. Whereas, the construction of the input hierarchy
                  for the classical approaches is an interesting problem in
                  itself. Simply put, we compute an attribute function for each level
                  line that characterizes its resistance under the energy
                  minimization. Then we stack the level lines from meaningless ones to
                  salient ones through a saliency map using shape-space filtering
                  framework. Qualitative illustrations and quantitative evaluation on
                  Weizmann segmentation evaluation database demonstrate the good
                  performance of our method.},
  note		= {Submitted},
  optlrdepaper	= {https://www.lrde.epita.fr/dload/papers/xu.hierarchymsll.pdf}
		  
}