Last update: 22/03/2016 21:18

What a PM wants … from a Test Report

For ages, I have been fortunate enough to be in the driving seat of several projects, some big, huge I'd say. I have had real hard times understanding and interpreting the test reports that were given to me. Trust my word!

Let me therefore explain what I would like to get, in my role of Project Manager:

I need to be able to extrapolate/interpret the report to figure out ? whether it meets the quality criteria that were defined for delivery and ? I need to know when I will be able to deliver. That's All, Folks!

Having said this, I would like to show you what sort of report I expect in an exemplary situation whose parameters are set forth as follows:

You get the picture? … THEN … The pie chart that I am expecting would correspond to the following figure: (understand: “should be similar to” or “should convey the same sort of information”):

Tests performed and OK
Tests performed and NOK
Tests NOT performed

The orange slice represents the tests that were NOT run (=25 tests upon 100).

The green slice represents the tests that were successful (=50 tests upon 100)

The red slice represents the tests that did fail (=25 tests upon 100)

Now, if we detail the red slice a bit further, we need to chop it down to the severity presented in the example.

Severity Count Time to solve Aggregated Time to Solve
Blocking 3 1.5md 1.5md
Partial Blocking 5 2.5md 4.0md
Non Blocking 5 2.5md 6.5md
Cosmetic 12 6md 12.5md

Remember that the average time to solve a bug is 0.5md! If I have decided not to let blocking bugs (either blocking or partial-blocking) hamper my production, it will require ( 3 + 5 ) x 0.5md to solve the defects of the first two rows! This is an estimation and I will take it as such: not more than a rough estimation!

This table makes it possible for me to inform my stakeholders that it would request 4 md to satisfy the quality criteria set forth in the Testing Strategy (I hope you have one!) I can even defend the idea of fixing more defects by including the “Non blocking” ones (even though 5% of “Non blocking defects” would still satisfy the exit criteria) for no more than 2.5 extra days.

That is really it! Keep it simple, guys. Simplicity is a difficult quality: it requires empathy. Remember also what is stated in the Agile Manifesto … Simplicity–the art of maximizing the amount of work not done–is essential

Our website uses cookies to save your preferences. We kindly ask you to consent to our use of cookies when you first visit our website. If you do not consent, the sole recourse is to stop visiting our website. X