Level 1 Initial At TMMi level 1, testing is a chaotic, undefined process and is often considered a part of debugging. The organization usually does not provide a stable environment to support the processes. Success in these organizations depends on the competence and heroics of the people in the organization and not the use of proven processes. Tests are developed in an ad-hoc way after coding is completed. Testing and debugging are interleaved to get the bugs out of the system. The objective of testing at this level is to show that the software runs without major failures. Products are released without adequate visibility regarding quality and risks. In the field, the product does often not fulfill its needs, is not stable, or is too slow to work with. Within testing there is a lack of resources, tools and well-educated staff. At TMMi level 1 there are no defined process areas. Maturity level 1 organizations are characterized by a tendency to over commit, abandonment of processes in a time of crises, and an inability to repeat their successes. Also products tend not to be released on time, budgets are overrun and quality is not according to expectations. Level 2 Managed At TMMi level 2, testing becomes a managed process and is clearly separated from debugging. The process discipline reflected by maturity level 2 helps to ensure that existing practices are retained during times of stress. However, testing is by many stakeholders still perceived as being a project phase that follows coding. In the context of improving the test process, a company-wide or program-wide test strategy is established. Test plans are also being developed. Within the test plan a test approach is defined, whereby the approach is based on the result of a product risk assessment. Risk management techniques are used to identify the product risks based on documented requirements. The test plan defines what testing is required, when, how and by whom. Commitments are established with stakeholders and revised as needed. Testing is monitored and controlled to ensure it is going according to plan and actions can be taken if deviations occur. The status of the work products and the delivery of testing services are visible to management. For deriving and selecting test cases from specifications test design techniques are applied. However, testing may still start relatively late in the development lifecycle, e.g. during the design or even during the coding phase. Testing is multi-leveled: there are unit, integration, system and acceptance test levels. For each identified test level there are specific testing objectives defined in the organization-wide or program-wide test strategy. The main objective of testing in a TMMi level 2 organizations is to verify that the product satisfies the specified requirements. The purpose is also to clearly differentiate the processes of testing and debugging. Many quality problems at this TMMi level occur because testing occurs late in the development lifecycle. Defects are propagated from the requirements and design into code. There are no formal review programs as yet to address this important issue. Post code, execution based testing is by many stakeholders still considered the primary testing activity. The process areas at TMMi level 2 are: 2.1 Test Policy and Strategy 2.2 Test Planning 2.3 Test Monitoring and Control 2.4 Test Design and Execution 2.5 Test Environment Level 3 Defined At TMMi level 3, testing is no longer a phase that follows coding. It is fully integrated into the development lifecycle and the associated milestones. Test planning is done at an early project stage, e.g. during the requirements phase, by means of a master test plan. The development of a master test plan builds on the test planning skills and commitments acquired at TMMi level 2. The organization’s set of standard test processes, which is the basis for maturity level 3, is established and improved over time. A test organization and a specific test training program exist, and testing is perceived as being a profession. Test process improvement is fully institutionalized as part of the test organization. Organizations at this level understand the importance of reviews in quality control; a formal review program is implemented although not yet fully linked to the dynamic testing process. Reviews take place across the lifecycle. Test professionals are involved in reviews on requirements specifications. Whereby the test designs at TMMi level 2 focus mainly on functionality testing, test designs and test techniques are expanded, depending the business objectives, to also include non-functional testing, e.g. on usability and/or reliability. Chapter 2 TMMi Maturity Levels ©2009 TMMi Foundation. Version 2.0 Page 12 of 141 A critical distinction between TMMi maturity level 2 and 3 is the scope of the standards, process descriptions, and procedures. At maturity level 2 these may be quite different in each specific instance, e.g. on a particular project. At maturity level 3 these are tailored from the organization’s set of standard processes to suit a particular project or organizational unit and therefore are more consistent except for the differences allowed by the tailoring guidelines. Another critical distinction is that at maturity level 3, processes are typically described more rigorously than at maturity level 2. As a consequence at maturity level 3, the organization must revisit the maturity level 2 process areas. The process areas at TMMi level 3 are: 3.1 Test Organization 3.2 Test Training Program 3.3 Test Lifecycle and Integration 3.4 Non-Functional Testing 3.5 Peer Reviews Level 4 Management and Measurement In TMMi 4 organizations testing is a thoroughly defined, well-founded and measurable process. At maturity level 4, the organization and projects establish quantitative objectives for product quality and process performance and use them as criteria in managing them. Product quality and process performance is understood in statistical terms and is managed throughout the lifecycle. Measures are incorporated into the organization’s measurement repository to support fact-based decision making. Reviews and inspections are considered to be part of testing and used the measure document quality. The static and dynamic test approaches are integrated into one. Reviews are formally used as means to control quality gates. Products are evaluated using quantitative criteria for quality attributes such as reliability, usability and maintainability. An organization wide test measurement program provides information and visibility regarding the test process. Testing is perceived as evaluation; it consists of all lifecycle activities concerned with checking products and related work products. The process areas at TMMi level 4 are: 4.1 Test Measurement 4.2 Product Quality Evaluation 4.3 Advanced Peer Reviews Level 5 Optimization On the basis of all results that have been achieved by fulfilling all the improvement goals of the previous maturity levels, testing is now a completely defined process and one is capable of controlling the costs and the testing effectiveness. At TMMi maturity level 5, an organization continually improves it processes based on a quantitative understanding of the common cause of variation inherent in processes. Improving test process performance is carried out through incremental and innovative process and technological improvements. The methods and techniques are optimized and there is a continuous focus on fine-tuning and test process improvement. Defect prevention and quality control are practiced. Statistical sampling, measurements of confidence levels, trustworthiness, and reliability drive the test process. Amongst others “Defect Prevention” and “Quality Control” are introduced as process areas. The test process is characterized by sampling based quality measurements. A detailed procedure exists for selecting and evaluating test tools. Tools support the test process as much as possible during test design, test execution, regression testing, test case management, etc. Process re-use is also practiced at level 5 supported by a process asset library. Testing is a process with the objective to prevent defects. Process areas at level 5 are: 5.1 Defect Prevention 5.2 Test Process Optimization 5.3 Quality Control Note that the TMMi does not have a specific process area dedicated to test tools and/or test automation. Within TMMi test tools are treated as a supporting resource (practices) and are therefore part of the process area where they provide support
No comments:
Post a Comment