15: Test Plan and Test Items

Test Plan
Test Plan is a sub set of test policy.

Introduction

1. State the purpose of the Plan. It is the executive summary part of the plan.
2. You may want to include any references to other plans, documents or items that contain information relevant to this project/process.
3. Create a references section to contain all reference documents. ??Project Authorization ??Project Plan ??Quality Assurance Plan ??Configuration Management Plan ??Relevant Policies and Standards ??For lower level plans, reference higher level plan(s)
4. Identify the Scope of the plan in relation to the Software Project plan that it relates to.
5. As this is the “Executive Summary” keep information brief and to the point.

Test Items

1. These are things you intend to test within the scope of this test plan. Essentially a list of what is to be tested.
2. This can be developed from the software application test objectives inventories as well as other sources of documentation and information such as: ???Requirements Specifications ???Design Specifications ???Users Guides ???Operations Manuals or Guides ???Installation Manuals or Procedures
3. This can be controlled and defined by your local Configuration Management (CM) process if you have one.
4. This information includes version numbers, configuration requirements where needed.
5. It may also include key delivery schedule issues for critical elements.
6. Identify any critical steps required before testing can begin as well, such as how to obtain the required item.
7. This section can be oriented to the level of the test plan. For higher levels it may be by application or functional area, for lower levels it may be by program, unit, module or build.
8. References to existing incident reports or enhancement requests should also be included.

Features To Be Tested

1. This is a listing of what is to be tested from the USERS viewpoint of what the system does. This is not a technical description of the software but a USERS view of the functions.

2. It is recommended to identify the test design specification associated with each feature or set of features.

3. Set the level of risk for each feature. Use a simple rating scale such as (H, M, L); High, Medium and Low.

Features Not To Be Tested

1. This is a listing of what is NOT to be tested from both the Users viewpoint of what the system does and a configuration management/version control view.

• Identify WHY the feature is not to be tested, there can be any number of reasons.
• Not to be included in this release of the Software.
• Low risk has been used before and is considered stable.
• Will be released but not tested or documented as a functional part of the release of this version of the software.

Approach

This is your overall test strategy for this test plan.

1. Overall rules and processes should be identified.
2. Are any special tools to be used and what are they? Will the tool require special training?
3. What metrics will be collected?
4. How is Configuration Management to be handled?
5. How many different configurations will be tested à?Hardware, Software
6. What are the regression test rules? How much will be done at each test level.
7. Will regression testing be based on severity of defects detected?
8. Specify if there are special requirements for the testing.
9. Only the full component will be tested. A specified segment of grouping of features/components must be tested together.
10. MTBF, Mean Time between Failures - if this is a valid measurement for the test involved and if the data is available.
11. Resource availability
12. Deadlines??
13. Are there any significant constraints to testing?
14. Are there any recommended testing techniques that should be used, if so why?

Item Pass/Fail Criteria

1. What are the Completion criteria for this plan? This is a critical aspect of any test plan and should be appropriate to the level of the plan. The goal is to identify whether or not a test item has passed the test process.
2. At the Unit test level this could be items such as: All test cases completed. A specified percentage of cases completed with a percentage containing some number of minor defects. Code coverage tool indicates all code covered.
3. What is the number and severity of defects located?
4. Is it possible to compare this to the total number of defects? This may be impossible, as some defects are never detected.

Suspension Criteria and Resumption Requirements

Know when to pause in a series of tests or possibly terminate a set of tests. Once testing is suspended how is it resumed and what are the potential impacts, (i.e. regression tests).

If the number or type of defects reaches a point where the follow on testing has no value, it makes no sense to continue the test; you are just wasting resources.

Specify what constitutes stoppage for a test or series of tests and what is the acceptable level of defects that will allow the testing to proceed past the defects.

Test Deliverables

What is to be delivered as part of this plan?

• Test plan
• Test case specifications
• Test procedure specifications
• Test logs
• Test Incident Reports
• Test Summary reports
• Test Incident reports
• Possible test tools to aid in the testing process
• Test data can also be considered a deliverable

One thing that is not a test deliverable is the software; that is listed under test items and is delivered by development.

Environmental Needs

1. Are there any special requirements for this test plan, such as:
2. Special hardware such as simulators, static generators etc.
3. How will test data be provided? Are there special collection requirements or specific ranges of data that must be provided?
4. Specific versions of other supporting software.
5. Restricted use of the system during testing.
6. Tools (both purchased and created).
7. Communications ???Web ???Client/Server ???Network ???Security

Responsibilities

Who is in charge? There should be a responsible person for each aspect of the testing and the test process. Each test task identified should also have a responsible person assigned. This includes all areas of the plan, here are some examples.

1. Setting risks.
2. Selecting features to be tested and not tested.
3. Setting overall strategy for this level of plan.
4. Ensuring all required elements are in place for testing.
5. Providing for resolution of scheduling conflicts, especially if testing is done on the production system.
6. Who provides the required training?
7. Who makes the critical go/no go decisions for items not covered in the test plans?
8. Who delivers each item in the test items section?

Staffing and Training Needs

Identify all critical training requirements and concerns.

1. Training on the product.
2. Training for any test tools to be used.

Schedule

1. Should be based on realistic and validated estimates. If the estimates for the development of the application are inaccurate the entire project plan will slip and the testing is part of the overall project plan.
2. As we all know the first area of a project plan to get cut when it comes to crunch time at the end of a project is the testing. It usually comes down to the decision, ‘Let’s put something out even if it does not really work all that well’. And as we all know this is usually the worst possible decision.
3. How slippage in the schedule to be handled should also be addressed here.
4. If the users know in advance that a slippage in the development will cause a slippage in the test and the overall delivery of the system they just may be a little more tolerant if they know it’s in their interest to get a better tested application.
5. By spelling out the effects here you have a change to discuss them in advance of their actual occurrence. You may even get the users to agree to a few defects in advance if the schedule slips.
6. It is always best to tie all test dates directly to their related development activity dates.

Risks and Contingencies

1. What are the overall risks to the project with an emphasis on the testing process?
2. Lack of personnel resources when testing is to begin.
3. Lack of availability of required hardware, software, data or tools.
4. Late delivery of the software, hardware or tools.
5. Delays in training on the application and/or tools.
6. Changes to the original requirements or designs.
7. The number of test performed will be reduced.
8. The number of acceptable defects will be increased.
9. These two items could lower the overall quality of the delivered product.
10. Resources will be added to the test team.
11. The test team will work overtime. This could affect team morale.
12. The scope of the plan may be changed.

You could just QUIT. A rather extreme option to say the least.

Approvals

Who can approve the process as complete and allow the project to proceed to the next

level (depending on the level of the plan).

At the master test plan level this may be all involved parties.

When determining the approval process keep in mind who the audience is. The audience for a unit test level plan is different than that of an integration, system level plan.