Last update: 22/03/2016 21:18
For ages, I have been fortunate enough to be in the driving seat of several projects, some big, huge I'd say. I have had real hard times understanding and interpreting the test reports that were given to me. Trust my word!
Let me therefore explain what I would like to get, in my role of Project Manager:
I need to be able to extrapolate/interpret the report to figure out ? whether it meets the quality criteria that were defined for delivery and ? I need to know when I will be able to deliver. That's All, Folks!
Having said this, I would like to show you what sort of report I expect in an exemplary situation whose parameters are set forth as follows:
You get the picture? … THEN … The pie chart that I am expecting would correspond to the following figure: (understand: “should be similar to” or “should convey the same sort of information”):
The orange slice represents the tests that were NOT run (=25 tests upon 100).
The green slice represents the tests that were successful (=50 tests upon 100)
The red slice represents the tests that did fail (=25 tests upon 100)
Now, if we detail the red slice a bit further, we need to chop it down to the severity presented in the example.
|Severity||Count||Time to solve||Aggregated Time to Solve|
Remember that the average time to solve a bug is 0.5md! If I have decided not to let blocking bugs (either blocking or partial-blocking) hamper my production, it will require ( 3 + 5 ) x 0.5md to solve the defects of the first two rows! This is an estimation and I will take it as such: not more than a rough estimation!
This table makes it possible for me to inform my stakeholders that it would request 4 md to satisfy the quality criteria set forth in the Testing Strategy (I hope you have one!) I can even defend the idea of fixing more defects by including the “Non blocking” ones (even though 5% of “Non blocking defects” would still satisfy the exit criteria) for no more than 2.5 extra days.
That is really it! Keep it simple, guys. Simplicity is a difficult quality:
it requires empathy. Remember also what is stated in the Agile Manifesto …
Simplicity–the art of maximizing the amount of work not done–is