Checking dnl and define

Now that our snacks have been digested, let's focus again on our main goal: designing a GNU M4 test suite, i.e., exercising the real core features of M4. We will write simple tests for a few basic builtins, and we will be sure to exercise them in some invalid way (voir Look for Realism).

Specifying the identity of the hosting package is also painful, in particular because a proper definition of the AT_PACKAGE_STRING must include its version. To factor the definition of these variables from now we will rely on the existence of package.m4:

     # Signature of the current package.
     m4_define([AT_PACKAGE_STRING],    [GNU Programming 2E])
     m4_define([AT_PACKAGE_BUGREPORT], [gnuprog2-devel@sourceforge.org])
     

     Example 12.12: A Simple package.m4
     

You don't have to m4_include it, or pass it as an argument to autom4te: autom4te --language=autotest automatically includes this file if present.

Two candidates are already laid in the test bed: dnl, being the most commonly used builtin¹, and define, so that we can test the user macro expansion. Exercising m4 typically consists in running it on some files. You are absolutely free to create the files the way you want, for instance using AT_DATA.

AT_DATA (file, contents)

Macro

Initialize an input data file with the given contents. Of course, the contents have to be properly quoted between square brackets to protect against included commas or spurious m4 expansion; no shell expansion of any sort is performed. The contents ought to end with an end of line.

Testing dnl is exceedingly simple: give it something to swallow, and observe it did:

     # Process with autom4te to create an -*- Autotest -*- test suite.
     # dnl.at -- Testing GNU M4 `dnl' and `define' builtins.
     
     AT_INIT([m4])
     
     AT_SETUP([Dnl])
     AT_DATA([[input.m4]],
     [[dnl This is killed.
     This is not
     ]])
     AT_TEST([[m4 input.m4]], [],
     [[This is not
     ]])
     AT_CLEANUP
     

     Example 12.13: dnl.at (i) -- A Broken Autotest Source Exercising dnl
     

     $ autom4te -l autotest dnl.at -o dnl
     dnl.at:8: error: possibly undefined macro: dnl
     dnl.at:11: error: possibly undefined macro: AT_TEST
     $
     

Arg! Yet another zealous useful feature: autom4te makes sure there are no suspicious tokens in the output which could result from improper quotation, or typing errors. And there is one indeed: the author of the test suite meant AT_CHECK, not AT_TEST. But in this test suite we really want to refer to dnl.

There are two means to explain this to autom4te.

One first solution consists in using the empty quadrigraph, @&t@, to mark valid occurrences of dnl in the output, as in:

     AT_DATA([[input.m4]],
     [[d@&t@nl This is killed.
     This is not
     ]])
     

But autom4te still complains, this time, being unable to find the source of the guilty dnl in dnl.at, its input, it reports the location in the output file:

     $ autom4te -l autotest dnl.at -o dnl
     dnl:209: error: possibly undefined macro: dnl
     $ sed -n 209p dnl
        1: dnl.at:6        Dnl
     

Aha! The culprit is no less than the filename! Therefore we have no other choice than using the second solution (except renaming the file): completely disabling the checking of dnl in the output. As matter of fact, we will use dnl so heavily while testing m4 that tagging each occurrence would obfuscate too much: just add m4_pattern_allow([^dnl$])².

The first solution is definitely the safest, because you tagged exactly the occurrences of dnl which are meant to be output. Any other accidental unexpanded dnl will still be caught. But sometimes simplicity and risks are to be preferred to strictness and safety.

And now for something completely different: define. Contrary to dnl, define has a precise arity: without arguments it is ignored, otherwise it takes one or two arguments, it should warn for any other arity (obviously we won't test them all). Let's first check by hand:

     $ cat define.m4
     define
     define()
     define(`one')one
     define(`two', `Two')two
     define(`three', `Three', `THREE')three
     $ m4 define.m4
     define
     
     
     Two
     error-->m4: define.m4: 5: Warning: define: too many arguments (ignored): 3 > 2
     Three
     

It is then a simple matter of separating the standard output from the standard error output, and just wrap this into a test case:

     AT_SETUP([[Define]])
     
     AT_DATA([[define.m4]],
     [[define
     define()
     define(`one')one
     define(`two', `Two')two
     define(`three', `Three', `THREE')three
     ]])
     
     AT_CHECK([[m4 define.m4]], [],
     [[define
     
     
     Two
     Three
     ]],
     [[m4: define.m4: 5: Warning: define: too many arguments (ignored): 3 > 2
     ]])
     
     AT_CLEANUP
     

     Example 12.14: dnl.at (ii) -- An Autotest Source Exercising define
     

Create the test suite, launch it: good, it passes with success. Let's try it on our fetal m4, not yet installed:

     $ ./dnl AUTOTEST_PATH=$HOME/src/m4/src
     ## --------------------------------------------------- ##
     ## GNU Programming 2E 0.0a test suite: Dnl and Define. ##
     ## --------------------------------------------------- ##
       1: dnl.at:7          ok
       2: dnl.at:17         FAILED near `dnl.at:35'
     ## ----------------------------------------------- ##
     ## ERROR: Suite unsuccessful, 1 of 2 tests failed. ##
     ## ----------------------------------------------- ##
     
     You may investigate any problem if you feel able to do so, in which
     case the test suite provides a good starting point.
     
     Now, failed tests will be executed again, verbosely, and logged
     in the file dnl.log.
     
     ## --------------------------------------------------- ##
     ## GNU Programming 2E 0.0a test suite: Dnl and Define. ##
     ## --------------------------------------------------- ##
     2. ./dnl.at:17: testing Define...
     ./dnl.at:35: m4 define.m4
     --- -   Tue Sep  4 18:04:30 2001
     +++ at-stderr   Tue Sep  4 18:04:30 2001
     @ -1,2 +1,2 @
     -m4: define.m4: 5: Warning: define: too many arguments (ignored): 3 > 2
     +lt-m4: define.m4: 5: Warning: define: too many arguments (ignored): 3 > 2
     
     2. ./dnl.at:17: FAILED near `dnl.at:35'
     ## ------------------- ##
     ## dnl.log is created. ##
     ## ------------------- ##
     
     Please send `dnl.log' to <gnuprog2-devel@sourceforge.org>,
     along with all information you think might help.
     

     Example 12.15: dnl Run on an Installed m4
     

Because it is the paragon of dynamic module based software, GNU M4 is built with Libtool; because of obscure but very well founded reasons which are beyond the scope of this chapter (FIXME: Ref to Libtool?.), bin/m4 is actually a shell script. It runs an executable named lt-m4. This is why the signature in the error message is "wrong". We have a serious problem, for our ultimate goal is to write a test suite shipped with GNU M4.

One possibility consists in adjusting the expected error messages to using lt-m4. This would prevent us from using our test suite on any other m4. In addition it clashes with an important motto: the test suite is a user, see Look for Realism.

Another is having the test suite be robust enough to work with different signatures, i.e., applying the same techniques as those we used in the previous section: save the standard error output, standardize it, check it. What a hassle! But why not, an AT_CHECK_M4 macro could hide those gory details.

For the time being, let us just imagine we didn't read Designing a Test Suite. We chose this solution, and proceed to other kinds of tests.