What is Autotest

The previous section highlighted that test suites are actual projects: they have to be maintained, extended etc. To ease their maintenance, there are a few tools, most notably DejaGNU. Unfortunately running a DejaGnu test suite requires DejaGNU, which itself requires TCL! As a consequence, given that few users installed DejaGNU on their machines, the DejaGNU test suites are rarely run, severely reducing part of their interest: exercising a package on a wide variety of platforms.

To make sure their users will always be able to run their test suites, many package maintainers write their test suite as a collection of hand written Bourne Shell scripts. Needless to say that this is long, tedious, unmaintainable etc.

History already faced this situation and provided an answer: people used to write long portable shell scripts to configure their package. Autoconf was invented to automatically create these configuring scripts from synthetic descriptions. Similarly, Autotest was invented to automatically create testing scripts. Autoconf is a configuration scripts compiler, Autotest is a testing scripts compiler. Because they share a significant part of code, Autotest is part of the package Autoconf.

An Autotest test suite is a series of test groups. A test group is a sequence of interwoven commands that ought to be executed together, usually because one creates data files that a later test in the same group needs to read. For instance, a Bison test group is typically composed of three tests: one which runs bison, one which runs the compiler, and one that runs the generated executable. They form a consistent group: it makes no sense to run the last step if the previous steps were not, nor to run the last steps if the previous ones failed.

In the following section, Running an Autotest Test Suite, we present the Autotest test suite from the user's point of view: "I'm installing a new package on my machine: how do I run its test suite?". The rest of the chapter is dedicated to writing and compiling an Autotest test suite.