# Hierarchical image simplification and segmentation based on Mumford-Shah-salient level line selection

### From LRDE

- Authors
- Yongchao Xu, Thierry Géraud, Laurent Najman
- Journal
- Submitted
- Type
- article
- Keywords
- Image
- Date
- 2015-09-16

## Abstract

Hierarchies, such as the tree of shapes, are popular representations for image simplification and segmentation thanks to their multiscale structures. Selecting meaningful level lines (boundaries of shapes) yields to simplify image while preserving intact salient structures. Many image simplification and segmentation methods are driven by the optimization of an energy functional, for instance the celebrated Mumford-Shah functional. In this paper, we propose an efficient approach to hierarchical image simplification and segmentation based on the minimization of some energy functional. This method conforms to the current trend that consists in producing hierarchical results rather than a unique partition. Contrary to classical approaches which compute optimal hierarchical segmentations from an input hierarchy of segmentations, we rely on the tree of shapes, a unique and well-defined representation equivalent to the image. Simply put, we compute for each level line of the image an attribute function that characterizes its persistence under the energy minimization. Then we stack the level lines from meaningless ones to salient ones through a saliency map based on extinction values defined on the tree-based shape space. Qualitative illustrations and quantitative evaluation on Weizmann segmentation evaluation database demonstrate the state-of-the-art performance of our method.

## Materials

### Mumford-Shah Simplification on the Color Tree of Shapes

You can download the x86_64 binary to compute the Mumford-Shah simplification running on the color tree of shapes Here.

Usage: ./mumford_shah_on_ctos input[rgb] α₀ α₁ λ output[rgb] α₀ Grain filter size before merging trees (0 to disable) α₁ Grain filter size on the color ToS (0 to disable) λ Mumford-shah regularisation weight (e.g. 5000)

### Saliency Map Computation Relying on Mumford-Shah-Salient Level Line Selection

You can download the x86_64 binary to compute the saliency map representing hierarchical image simplification and segmentation Here. This application outputs the saliency map as a float image. The simplification and segmentation result can be obtained by thresholding this float image. Note that the image is twice as big as the original one and has a border for topogical and algorithmic purposes. Thus, any pixel with coordinates (x,y) in the original image is now at coordinates (2*(x+1), 2*(y+1)) in the saliency map.

Usage: ./saliency_map_mumford_ctos input[rgb] α₀ α₁ output[float] α₀ Grain filter size before merging trees (0 to disable) α₁ Grain filter size on the color ToS (0 to disable)

In the previous binary, we use the absolute difference between values of neighboring pixels to compute the average of gradient's magnitude, which is used to sort the shapes. An alternative is to use a gradient image given by a sophisticated contour detection method (e.g., the Structured Edge) to compute the average of gradient's magnitude. You can download this x86_64 binary Here.

Usage: ./saliency_map_mumford_ctos_grad input[rgb] α₀ α₁ gradSE.pgm output[float] α₀ Grain filter size before merging trees (0 to disable) α₁ Grain filter size on the color ToS (0 to disable) gradSE Gradient image given by a contour detection method (Gpb or SE)

## Illustrations

### Saliency map computation on BSDS500 database

The first test was performed on the BSDS500 database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

### Saliency map computation on Weizmann database

The second test was performed on the Weizmann database. Some samples are given below and full results are available in this archive. The results are obtained relying on the gradient image of Structured Edge.

## Bibtex (lrde.bib)

@Article{ xu.hierarchymsll, author = {Yongchao Xu and Thierry G\'eraud and Laurent Najman<nowiki>}, title = {Hierarchical image simplification and segmentation based on Mumford-Shah-salient level line selection}, journal = {Submitted for publication}, year = 2015, project = {Image}, abstract = {Hierarchies are popular representations for image simplification and segmentation thanks to their multiscale structures. An example is the tree of shapes. Selecting meaningful level lines (boundaries of shapes) yields to simplify image while preserving intact salient structures. Many image simplification and segmentation methods are driven by the optimization of an energy functional, for instance the Mumford-Shah functional. In this paper, we propose an efficient approach of hierarchical image simplification and segmentation based on the minimization of some energy functional. This method conforms to the current trends that are to find hierarchical results rather than a unique partition. Contrary to classical approaches which compute optimal hierarchical segmentations from an input hierarchy of segmentations, we rely on the tree of shapes, a unique and equivalent image representation. Whereas, the construction of the input hierarchy for the classical approaches is an interesting problem in itself. Simply put, we compute an attribute function for each level line that characterizes its resistance under the energy minimization. Then we stack the level lines from meaningless ones to salient ones through a saliency map using shape-space filtering framework. Qualitative illustrations and quantitative evaluation on Weizmann segmentation evaluation database demonstrate the good performance of our method.}, note = {Submitted}, optlrdepaper = {https://www.lrde.epita.fr/dload/papers/xu.hierarchymsll.pdf} }