• Implementer

• Execute Developer Tests

Each Test Log should be made up of a series of entries that present an audit trail for various aspects of the test execution including, but not limited to, the following:

• the date and time stamp of when the event occurred
• a description (usually brief) of the event logged
• some indication of the observed status
• additional contextual information where relevant
• additional details relating to any anomalous or erroneous condition detected

The Test Log provides a detailed, typically time-based record that serves both as verification that a set of tests were executed, and provides information relating to the success of those tests. The focus is typically on the provision of an accurate audit trail, enabling post-execution diagnosis of failures to be undertaken. This raw data will subsequently be analyzed to help determine the results of some aspect of the test effort.

Automation tools often provide their own Test Log facilities, which can be extended or supplemented with additional logging provided both through custom user-routines and the use of additional tools.

The output may take one single or many different forms. Typically, Test Logs have a tabular or spreadsheet-like appearance, with each entry comprising some form of date and time stamp, a description of the event logged, some indication the observed status, and possibly some additional contextual information.

If you are using automated test tools, such as those found in the Rational Suite family of products, much of the above functionality is provided by default with the tool. These Test Log facilities typically provide the ability for the capture, filtering and sorting and the analysis of the information contained in the log. This allows the Test Log to be expanded in detail or collapsed to a summary view as required. The tools also offer the ability to customize and retain views of the Test Log for reporting purposes.

Where the logic that produces an automated Test Log simply appends new information to an existing log file, it will be necessary to provide sufficient storage to retain the Test Log file. An alternative solution to this approach is to use a ring buffer. A good explanation of using Ring Buffer Logging to help find Bugs is presented in a pattern catalog by Brian Marick. (Get Adobe reader.) This catalog provides an overview of other classic problems with using automated Test Logs.