A Test System for L^{A}TEX-2.09

D. P. Carlisle Based on files originally written by F. Mittelbach^{∗}

Printed March 13, 1994

Aims of this Package

This set of files implements a scheme suggested by Frank Mittelbach for ensuring that new releases of L^{A}TEX do not inadvertently modify the behaviour of commands. This is definitely a non-trivial task, as L^{A}TEX is a large system, and in ‘fixing’ one bug, one often needs to modify the definitions of several ‘internal’ commands, which may affect many other commands which have no obvious connection with the original problem.

The Basic System

The idea is to have a suite of test files, each one exersising a particular set of related commands. These should be called in such a way that one can tell from the .log file (not the .dvi file) that the command has ‘met its specification’. The .log file is then edited to remove certain irrelevant information and will then be stored, as a .tlg file. Before a new release is issued, all the test files will be run through the new version, and the resulting .tlg files will be automatically compared with the saved original versions. Any tests which do not produce identical results will then be notified to the maintainer of L^{A}TEX, who can visually compare the .tlg files to see whether the differences are due to an ‘improvement’ or are the result of a newly introduced bug!

To distinguish test files from other files they have to have the extension .lvt (standing for L^{A}TEX validation test).

The Format of a Test File

If the functions to be tested need to be called inside the L^{A}TEX document environment, then start the file as usual with:

\documentstyle{article}
\begin{document}

∗Version v1.0a dated 92/06/28

If you are testing some commands from L^{A}TEX‘s ‘style file language’, then these lines may be omitted.

Then start the test procedure with

\input{test209}
\START

Everything in the .log file before this will be omitted, \START also prints a character table in the .log file, so that the .tlg file may be checked for errors caused be faulty email connections. After \START, @ is a letter, so you may use any commands that you may use in a style file.

If you like you can identify yourself as the original author of this test file by issuing the following commands.

\AUTHOR{my name}
\ADDRESS{my email address}

This will help us to get back to you later if necessary.

You should then declare the format that you are using, and any styles that you have input. This should include dates and/or version numbers, so that if at some future time, the test fails, any differences between the original versions and the new versions can be checked, to find the cause of the failure.

\FORMAT{LaTeX Version 2.09 <14 January 1982>}
\STYLE{article <14 Jan 92>}
\STYLEOPTIONS{array v2.0g}

You may then optionally ‘declare’ a collection of commands that make up the ‘module’ that you are testing. \declare@command\foo will report whether \foo is defined, undefined, or equivalent to \relax. Note that you may also declare commands which are not currently defined in L^{A}TEX, but which might be added. The system will then pick up the addition of such a command, and remind the maintainers to announce that there has been a change in the user interface. Thus if you were testing the array environment, you might want to declare \extrarowheight (currently defined in array.sty) or \hhline (currently defined in hhline.sty). Note that you would not declare \@tabularcr, as this is an internal command, and the fact that it is undefined in the array.sty version does not constitute a change in the interface.

After that you may have any series of L^{A}TEX commands, these should be called in such a way that any incorrect behaviour would show up in the .log file.

Any commands, such as those allocating registers, which produce lines in the .log file, which you do not want to be considered as part of the test, you should surround with the declarations \OMIT and \TIMO.

Finally the file should finish with either \END, or \end{document}. Only use the second form if you actually need to close the document environment, say to get the .aux files correct.

Tips

The test should be written to show that the command meets its specification. Ideally it should not fail if the command is correctly re-implemented in a different way. This means that the primitive tracing facilities like \tracingmacros and \tracingcommands should probably not be used, as these reveal the implementation details of the command.

If the command involves visual formatting, one way to get information into the .log file is to do all the commands inside \setbox0=\vbox{...}, and then \showbox0. Before showing the box you should set \showboxdepth and \showboxbreadth to suitable values so that just the right amount of information displayed. L^{A}TEX has a command called \showoutput which can be of some help here.

\tracingoutput or \tracingpages might also be used (with care), say if you were testing section headings at the top and bottom of pages. Other useful commands are \typeout{\meaning...}, which is like \show, but puts less junk

in the .log file, and \typeout{\the...}, which is like \showthe.

Please remember that it isn’t sufficient to produce a test file that shows the desired behavior in the .dvi file since this file will not be inspected by the test system later on. Nevertheless, it is often helpful to check the .dvi when creating a test file but you have to make sure that the relevant information is also displayed in the .tlg in the end.

What format to use

The format used to run the tests should not contain local specialties, i.e., you should either stick to a standard format generated from lplain or to a combination of lplain together with the new font selection. In any case please document the used format clearly with the above commands. If possible we would prefer if you use an new font selection format with either basefont or newlfont as base. For the present the standard American hyphenation patterns should be used.

Creating the .tlg File

If you are on a UNIX machine, a supplied shell script will run latex twice on your test file, then edit the .log to produce a .tlg file automatically. At present, on other operating systems, you will need to do this by hand.^{1} Run L^{A}TEX twice (using the format you have chosen and documented), afterwards delete all lines upto and including START-TEST-LOG, and all lines after, and including END-TEST-LOG. Similarly remove lines OMIT and TIMO, and everything in between. Finally replace the base part of the test file name whenever it shows up with “nothing”, e.g., turn ‘(tlen01.aux)’ into ‘(.aux)’. In this way the files stay valid if we have to rename them.

1We are grateful if we can include scripts for other operating systems to do this task.

Sending new files to the L^{A}TEX maintainers

It is important to realize, that only completely documented and checked files can be included into the automated checking system for L^{A}TEX2.09. The documentation, i.e., information what is supposed to show up in the .tlg file, etc., is very important, since if later such a file shows a change we need to be able to determine the reason for it. It is also important that we don’t have to re-check the test file, in other words we like to get both the test file and the hand-check result file, i.e., the .tlg file from you. Please send them to the address mentioned in

readme.val file.

If You Find a Bug

The basic idea of this test system is to ensure that future releases do not introduce new bugs. That is, it will normally be assumed that if the new release produces different results on a test to an old release, then the new release has a bug. Because of this, if you discover while producing the file, that L^{A}TEX acts incorrectly in some context, still submit the file, but note clearly in the file, and in the email message, that you consider the current behaviour incorrect. Otherwise you may ensure that this behaviour is maintained in all future releases!

Sample files

As an example we show a test file for the “length” module of L^{A}TEX2.09. Please note that it is important to document the desired output and also that only relevant information shows up in the .tlg file. For this reason the allocation information for \newlength is suppressed since it may change without making the interface invalid.

The file tlen01.lvt
% \iffalse meta-comment
%
% Copyright (C) 1992 by David Carlisle, Frank Mittelbach.
% All rights reserved.
%
% This file is part of the validate package.
%
% IMPORTANT NOTICE:
%
% You are not allowed to change this file.  In case of error
% write to the email address mentioned in the file readme.val.
%
% \fi
%
% Example test file for LaTeX2.09 testing the ``length''
% module.
%

\documentstyle{article}
\begin{document}
% Input the test macros for latex 2.09
\input{test209}
% Everything before this is ignored by the test system.
\START
\AUTHOR{David Carlisle}
% Declare the format used on the original run, as printed by
% LaTeX.
%
\FORMAT{LaTeX Version 2.09 <14 January 1992>}
% Article document style for this test. We need to have a
% minimal font setup, for the last few tests, otherwise we
% could have done this with no documentstyle.
%
\STYLE{article <14 Jan 92>}
% no document style options for this test
%
\STYLEOPTIONS{}
% We are using a format based on basefont.tex posible values
% here are: original oldlfont newlfont basefont
%
\FONTSELECTION{basefont}
% Optionally declare commands which are (or might be)
% defined in this `module' of LaTeX.  Each command declared
% will be recorded in the .tlg file as Defined, Undefined,
% or Relaxed (ie \let to \relax, which LaTeX treats for most
% purposes as being undefined)
%
\declare@command\newlength
\declare@command\setlength
\declare@command\addtolength
\declare@command\settowidth

% Use an already used name.
%
\newlength{\string}
% Check that it has not been over-written,
% \typeout{\meaning...} puts less junk in the .log file
% than \show.
%
% The output should be just ``\string'' since this
% is the primitive meaning of \string.
%
\typeout{\meaning\string}

% Get a length command: The .log file records the actual

% length register allocated, this is not part of the
% specification of \newlength, so omit it from the .tlg
% file.
%
\OMIT
\newlength{\la}
\TIMO
% Add some lengths.
%
\setlength{\la}{2pt}
\addtolength{\la}{5pt}
\addtolength{\la}{-50pt}
\typeout{-43pt = \the\la}
% check `rubber lengths' work
%
\addtolength{\la}{100pt plus 10pt}
\typeout{57pt plus 10pt = \the\la}

% check that \setlength does not look ahead for plus and minus
%
\setlength{\la}{5pt} plus 5pt
\typeout{5pt = \the\la}
% check that \addtolength does not look ahead for plus and
% minus
%
\addtolength{\la}{5pt} plus 5pt
\typeout{10pt = \the\la}

% check \settowidth's reaction to spaces
%
\tt
\settowidth{\la}{ }
\typeout{5.25pt = \the\la}
\settowidth{\la}{ x}
\typeout{10.5pt = \the\la}
\settowidth{\la}{x }
\typeout{10.5pt = \the\la}
\settowidth{\la}{x}
\typeout{5.25pt = \the\la}

% everything after this will be ignored by the test system.
% This line may be omitted, and replaced by \end{document}
% If it is necessary to run the \end{document} command, say
% to get the .aux files correct.
%
\end{document}

The file tlen01.tlg

This is the resulting output after applying the UNIX script maketlg to the file

tlen01.lvt.
This is a generated file for the LaTeX 2.09 validation system.
Don't change this file in any respect.

%% \CharacterTable
%%  {Upper-case    \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z
%%   Lower-case    \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z
%%   Digits        \0\1\2\3\4\5\6\7\8\9
%%   Exclamation   \!     Double quote  \"     Hash (number) \#
%%   Dollar        \$     Percent       \%     Ampersand     \&
%%   Acute accent  \'     Left paren    \(     Right paren   \)
%%   Asterisk      \*     Plus          \+     Comma         \,
%%   Minus         \-     Point         \.     Solidus       \/
%%   Colon         \:     Semicolon     \;     Less than     \<
%%   Equals        \=     Greater than  \>     Question mark \?
%%   Commercial at \@     Left bracket  \[     Backslash     \\
%%   Right bracket \]     Circumflex    \^Underscore    \_
%%   Grave accent  \`     Left brace    \{     Vertical bar  \|
%%   Right brace   \}     Tilde         \~}
%%

Author: David Carlisle
Format: LaTeX Version 2.09 <14 January 1992>
Main Style: article <14 Jan 92>
Style Options:
Font Selection: basefont
Defined   \newlength
Defined   \setlength
Defined   \addtolength
Defined   \settowidth
LaTeX error.  See LaTeX manual for explanation.
              Type  H <return>  for immediate help.
! Command name 'string' already used.
\@latexerr ...for immediate help.}\errmessage {#1}
l.64 \newlength{\string}
Your command was ignored.
Type  I <command> <return>  to replace it with another command,
or  <return>  to continue without it.
\string
-43pt = -43.0pt
57pt plus 10pt = 57.0pt plus 10.0pt
5pt = 5.0pt
10pt = 10.0pt
5.25pt = 5.24995pt
10.5pt = 10.49991pt
10.5pt = 10.49991pt
5.25pt = 5.24995pt
[1

] (.aux)

8