Start by reviewing any existing Test Suite outlines, and determine which Test Suites are good candidates for implementation at the current time. Use the Iteration Test Plan, Test-Ideas List and any additional test definition work products as a basis for making your decision.

For each Test Suite you have selected for implementation, identify what Target Test Items and associated Tests are candidates for inclusion in the scope of the Test Suite.

Begin by considering the Test Environment Configuration and specific system start state. Consider what specific setup requirements there will be, such as the starting data set for dependent databases. Where one Target Environment Configuration will be used for various Test Suites, identify any configuration settings that may need to be managed for each Test Suite, such as the screen resolution of video displays or the regional operating system settings.

Now determine any specific relationships between the Tests. Look for dependencies where the execution of one Test included in the Test Suite will result in a system state change required as a precondition of another Test.

Once you've identified the relevant dependencies, determine the correct sequence of execution for the dependent Tests.

One of the main challenges in maintaining a Test Suite-especially an automated one-is ensuring that ongoing changes are easy to make. It's a good idea when possible and deemed useful to maintain a central point of modification for elements that are used in multiple places. That's especially true if those same elements are likely to change.

While the Tests themselves form natural units of modularity, assembly of the Tests into a Test Suite often identifies duplicate procedural elements across multiple Tests that could be more effectively maintained if the were consolidated. Take the opportunity to identify any general mechanics of the Tests that might potentially be refactored into a standard routine to assist ongoing maintenance.

Most test efforts require the use of one or more "utilities" that generate, gather, diagnose, convert and compare information used during test execution. These utilities typically simplify both complex and laborious tasks that would be prone to error if performed manually. This step relates to applying existing utility functions within the Test Suite, and identifying new utilities that are required.

It's a good idea to simplify the interfaces to these utilities, encapsulating as much complexity as possible within the private implementation of the utility. It's also a good idea to develop the utility in such a way that it can be reused where required for both manual and automated test efforts.

We recommend you don't hide the information that characterizes an individual test within these utilities: instead, limit the utility to the complex mechanics of gathering information, or comparing actual values to expected results etc. but where possible, pass the specific characteristics of each individual test in as input from-and return the individual actual results an output to-a controlling Test or Test Suite.

Determine the appropriate points within the Test Suite to provide recovery if the Test Suite fails during execution. This step gains importance where the Test Suite will contain a large number of Tests, or will run for an extended period of time-often unattended. While most often identified as a requirement for automated Test Suites, it is also important to consider recovery points for manually executed Test Suites.

In addition to recovery or restart points you may also want-in the case of automated Test Suites-to consider automated Test Suite recovery. Two approaches to auto-recovery are 1) basic recovery where the existing Test Suite can self-recover from a minor error that occurs in one of it's Tests, typically recovering execution at the next Test in the Test Suite or 2) sophisticated recovery that cleans up after the failed Test, resetting appropriate system state including operating system reboot and data restoration if necessary. As in the first approach, the Test Suite then determines the Test that failed and selects the next Test to execute.

Depending on the level of sophistication required, it will require effort to implement and stabilize recovery processing. You'll need to allow time to simulate a number of likely (and a few unlikely) failures to prove the recovery processing works.

In the case of automated recovery, both approaches outlined in the previous step have strengths and weaknesses. You should consider carefully the cost of sophisticated automated recovery, both in terms of initial development but also ongoing maintenance effort. Sometimes manual recovery is good enough.

You should take time to stabilize the Test Suite by one or more trial test executions where possible. The difficulty in achieving stability increases proportionally to the complexity of the Test Suite, and where there is excessively tight coupling between unrelated and and low cohesion between related Tests.

There is the possibility of errors occurring when Tests are executed together within a given Test Suite, that were not encountered when the individual Tests were executed independently. These errors are often the most difficult to track down and diagnose, especially when the are encountered halfway though a length automated test run. Where practical, it's a good idea to rerun the Test Suite regularly as you add additional Tests. This will help you isolate a small number of potential candidate Tests to be diagnosed to identify the problem.

Using the Traceability requirements outlined in the Test Plan, update the traceability relationships as necessary. Test Suites might be traced to defined Test Cases or to Test Ideas. Optionally, they may be traced to Use Cases, software specification elements, Implementation Model elements and to one or more measures of Test Coverage.

Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and that you did not simply consume vast quantities of paper. You should evaluate whether your work is of appropriate quality, and that it is complete enough to be useful to those team members who will make subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify that quality and completeness are "good enough".

Have the people performing the downstream tasks that rely on your work as input take part in reviewing your interim work. Do this while you still have time available to take action to address their concerns. You should also evaluate your work against the key input work products to make sure you have represented them accurately and sufficiently. It may be useful to have the author of the input work product review your work on this basis.

Try to remember that that RUP is an iterative delivery process and that in many cases work products evolve over time. As such, it is not usually necessary-and is often counterproductive-to fully-form a work product that will only be partially used or will not be used at all in immediately subsequent work. This is because there is a high probability that the situation surrounding the work product will change-and the assumptions made when the work product was created proven incorrect-before the work product is used, resulting in wasted effort and costly rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value. In project environments where presentation has importance and economic value as a project deliverable, you might want to consider using an administrative resource to perform presentation tasks.