A great challenge for any organisation is effectively reviewing the testing practice they have in place in order to design a fair and balanced change programme. If you’re a Programme Test Manager joining a project with uncertainty around the maturity and efficiency of the process, practice and outcomes of the test management system in place – then a Testing Efficiency Review is a great place to start.
Conducting the review can prove a challenge for many reasons including:
- The difficulty of remaining and being seen to be approaching the review in an independently minded way without bias for reviewer’s interests.
- Critical self review of the software testing practice that is in place already that may identify strengths and weaknesses with co-workers and their teams.
- Risk of souring internal relationships and possible issues the required questioning and review will require.
- Limited knowledge of the domain under test that prevents the design of the most effective change programme.
Scope of the Review
A Testing Efficiency Review should specifically designed to eliminate all of these potential problems and deliver the most effective change programme related to how testing is conducted.
The review needs to be full-scope and designed to address practices, processes, team capabilities and aspirations of the organisation related to software testing and software quality assurance.
During the review a senior member of the test team of the Programme Test Manager themselves will gather information about the current status of testing and software quality assurance within an organisation or programme group.
The review is structured in two parts;
- In-team face to face discussions around “Key Discovery Areas” with relevant stakeholders to gather information and build trust in the process and the competency of the Programme Test Manager.
- Delivery of a clear and thorough written report that details the review findings and provides a series of recommendations to address areas of highest priority.
CMMi / TMMi
In my experience the best way to model the review and recommendations is in line with CMMi / TMMi – without aiming to drive compliance to this model. This isn’t a certification exercise, it is an internal audit and fact finding exercise to accelerate maturity and consistency. Instead, draw on proven models and best practices to make recommendations geared to the unique needs of the organisation.
Key Discovery Areas
When gathering information model the questions and fact finding around the following areas.
Process Definition and Management
- Organisational Process Definition and Focus
- Requirements Definition and Management
- Continuous Improvement and Feedback
- Documentation Configuration and Management
Project Definition and Management
- Project Planning and Tracking
- Communication and Inter-group coordination
- Organizational Structure and Team Roles
- Software/Testware Configuration Management and Change Control
- Peer Reviews and Quality Assurance
Testing Process and Tools
- Structure of Test Planning, Design, Specification and Execution Techniques
- Test Coverage and Requirements Mapping
- Test Management Tools
- Testing Tools and Automation
- Test Metrics, Measures and Reporting techniques and approaches
Charting the Results
Once the discovery is complete you’ll be able to easily map Maturity Levels against Discovery Areas, as in the example below. The definitions of the maturity levels can be set by the organisation or
CMMi/TMMi Compliance Level Shadow chart.
Developing a Programme of Change
After the review and results reporting a plan of action will need to be agreed for areas that are below the level of maturity needed. This is the Change Programme and it needs to be sponsored, resourced, planned, executed, tracked and delivered just like any project of work. The review is the map, but the organisation needs to decide what it does with that map.
Mark.
