Publications/xu.15.prl

Authors

Yongchao Xu, Thierry Géraud, Laurent Najman

Journal

Pattern Recognition Letters

Type

article

Keywords

Image

Date

2015-09-16

Abstract

Hierarchies are popular representations for image simplification and segmentation thanks to their multiscale structures. An example is the tree of shapes. Selecting meaningful level lines (boundaries of shapes) yields to simplify image while preserving intact salient structures. Many image simplification and segmentation methods are driven by the optimization of an energy functional, for instance the Mumford-Shah functional. In this paper, we propose an efficient approach of hierarchical image simplification and segmentation based on the minimization of some energy functional. This method conforms to the current trends that are to find hierarchical results rather than a unique partition. Contrary to classical approaches which compute optimal hierarchical segmentations from an input hierarchy of segmentations, we rely on the tree of shapes, a unique and equivalent image representation. Whereas, the construction of the input hierarchy for the classical approaches is an interesting problem in itself. Simply put, we compute an attribute function for each level line that characterizes its resistance under the energy minimization. Then we stack the level lines from meaningless ones to salient ones through a saliency map using shape-space filtering framework. Qualitative illustrations and quantitative evaluation on Weizmann segmentation evaluation database demonstrate the good performance of our method.

Materials

Mumford-Shah Simplification on the Color Tree of Shapes

You can download the x86_64 binary to compute the Mumford-Shah simplification running on the color tree of shapes Here.

Usage: ./mumford_shah_on_ctos input[rgb] α₀ α₁ λ output[rgb]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)
λ	Mumford-shah regularisation weight (e.g. 5000)

• 

  Input RGB Image

• 

  Output Simplified Image

• 

  Input RGB Image

• 

  Output Simplified Image

Saliency Map Computation Relying on Mumford-Shah-Salient Level Line Selection

You can download the x86_64 binary to compute the saliency map representing hierarchical image simplification and segmentation Here. This application outputs the saliency map as a float image. The simplification and segmentation result can be obtained by thresholding this float image. Note that the image is twice as big as the original one and has a border for topogical and algorithmic purposes. Thus, any pixel with coordinates (x,y) in the original image is now at coordinates (2*(x+1), 2*(y+1)) in the saliency map.

Usage: ./saliency_map_mumford_ctos input[rgb] α₀ α₁ output[float]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)

In the previous binary, we use the absolute difference between values of neighboring pixels to compute the average of gradient's magnitude, which is used to sort the shapes. An alternative is to use a gradient image given by a sophisticated contour detection method (e.g., the Structured Edge) to compute the average of gradient's magnitude. You can download this x86_64 binary Here.

Usage: ./saliency_map_mumford_ctos_grad input[rgb] α₀ α₁ gradSE.pgm output[float]
α₀	Grain filter size before merging trees (0 to disable)
α₁	Grain filter size on the color ToS (0 to disable)
gradSE 	Gradient image given by a contour detection method (Gpb or SE)

• 

  Input RGB Image

• 

  Saliency map

Illustrations

Saliency map computation on BSDS500 database

The first test was performed on the BSDS500 database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

• From top to bottom: input images; saliency maps; slight/moderate/strong simplification.
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Saliency map computation on Weizmann database

The second test was performed on the Weizmann database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

• From top to bottom: input images; saliency maps; slight/moderate/strong simplification.
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 

Bibtex (lrde.bib)

@Article{	  xu.2015.prl,
  author	= {Yongchao Xu and Thierry G\'eraud and Laurent Najman<nowiki>},
  title		= {Hierarchical image simplification and segmentation based on Mumford-Shah-salient levelline selection},
  journal	= {Pattern Recognition Letters},
  year		= 2015,
  project	= {Image},
  abstract	= {Hierarchies are popular representations for image simplification and
                                    segmentation thanks to their multiscale structures. An example is the
                                    tree of shapes. Selecting meaningful level lines (boundaries of
shapes) yields to simplify image while preserving intact salient
structures. Many image simplification and segmentation methods are
driven by the optimization of an energy functional, for instance the
Mumford-Shah functional. In this paper, we propose an efficient
approach of hierarchical image simplification and segmentation based
on the minimization of some energy functional.  This method conforms
to the current trends that are to find hierarchical results rather
than a unique partition. Contrary to classical approaches which
compute optimal hierarchical segmentations from an input hierarchy of
segmentations, we rely on the tree of shapes, a unique and equivalent
image representation. Whereas, the construction of the input hierarchy
for the classical approaches is an interesting problem in
itself. Simply put, we compute an attribute function for each level
line that characterizes its resistance under the energy
minimization. Then we stack the level lines from meaningless ones to
salient ones through a saliency map using shape-space filtering
framework. Qualitative illustrations and quantitative evaluation on
Weizmann segmentation evaluation database demonstrate the good
performance of our method.},
  note		= {Submitted},
  optlrdepaper	= {https://www.lrde.epita.fr/dload/papers/xu.2015.prl.pdf}
		  
}