Using histogram representation and Earth Mover's Distance as an evaluation tool for text detection

From LRDE

Abstract

In the context of text detection evaluation, it is essential to use protocols that are capable of describing both the quality and the quantity aspects of detection results. In this paper we propose a novel visual representation and evaluation tool that captures the whole nature of a detector by using histograms. First, two histograms (coverage and accuracy) are generated to visualize the different characteristics of a detector. Secondly, we compare these two histograms to a so called optimal one to compute representative and comparable scores. To do so, we introduce the usage of the Earth Mover's Distance as a reliable evaluation tool to estimate recall and precision scores. Results obtained on the ICDAR 2013 dataset show that this method intuitively characterizes the accuracy of a text detector and gives at a glance various useful characteristics of the analyzed algorithm.

Documents

Bibtex (lrde.bib)

@InProceedings{	  calarasanu.15.icdar,
  author	= {Stefania Calarasanu and Jonathan Fabrizio and S\'everine
		  Dubuisson},
  title		= {Using histogram representation and Earth Mover's Distance
		  as an evaluation tool for text detection},
  booktitle	= {Proceedings of the 13th IAPR International Conference on
		  Document Analysis and Recognition (ICDAR)},
  address	= {Nancy, France},
  month		= aug,
  year		= 2015,
  pages		= {221--225},
  abstract	= { In the context of text detection evaluation, it is
		  essential to use protocols that are capable of describing
		  both the quality and the quantity aspects of detection
		  results. In this paper we propose a novel visual
		  representation and evaluation tool that captures the whole
		  nature of a detector by using histograms. First, two
		  histograms (coverage and accuracy) are generated to
		  visualize the different characteristics of a detector.
		  Secondly, we compare these two histograms to a so called
		  optimal one to compute representative and comparable
		  scores. To do so, we introduce the usage of the Earth
		  Mover's Distance as a reliable evaluation tool to estimate
		  recall and precision scores. Results obtained on the ICDAR
		  2013 dataset show that this method intuitively
		  characterizes the accuracy of a text detector and gives at
		  a glance various useful characteristics of the analyzed
		  algorithm.},
  doi		= {10.1109/ICDAR.2015.7333756}
}