I’ve been working at Ve-hotech for years now. I first started being a developer and then moved to a project manager position…and as far as I can remember the tracking of bugs has always been a problem. The problem is not actually the bug tracking itself but to find a way to handle bugs when you cannot solve them as quickly as they get reported by end users (yes, we’ve been through pretty tough times…)

A classical approach for handling an important amount of bugs is to use a web-based bug tracking system. This is a pretty convenient way to centralize all the reported issues. The business stakeholders can then sort the bug list and prioritize it, allowing the team to pick up the next more important issues and solve them.

Although this approach might be compulsory when dealing with hundreds of bugs or when the team is not collocated, it surely is a pretty big overhead when the number of bugs is thin and the team and product owners are working at the same place.

Understanding that nobody really wanted to use a web interface for managing bugs, that the redmine instance we were using was beginning to get out of sync, and as we were moving to Kanban, I decided to morph the bug tracking system into a physical, visual, post-it driven bug backlog. Imagine a 2-meter high sheet of paper covered with little, yellow sticky notes – 80 or so.

The goal of this bug wall was to ease the work of product owners, since they could see everything at once and select the defects they wanted the team to solve more easily. New issues were added into a special area of the wall so that stakeholders could identify them and decide where to place them on the wall – and when to put them into the Kanban flow.

There were three main drawbacks with this bug tracking implementation :

1. The team – the people who know about the technical part – was not much involved into the prioritization/evaluation of bugs criticality
2. There was no simple way to get a whole-picture view of the product stability
3. Who can possibly sort 80 sticky notes?

To solve these problems, the boss (I wish I have thought of that first but…) came out with the idea of using a sort of criticality matrix.

We could actually dramatically improve how bugs were handled using two criteria:

1. Intrinsic severity : Does this bug jeopardize users’ data? Does it happen all the time? Is it highly visible? On the contrary is it only a highly improbable situation? Maybe we could not even see it by ourselves? etc. We decided that four levels of criticality were enough, each one having its own set of criteria.
2. Technical impact : How the team feels about this issue. Is there any identified risk? Is the fix difficult to implement? Do we need to rewrite an important part of the product? Do we even have a clue about how to debug this? etc. We don’t need any precise measures or calculations here. A simple gut feeling will do.

So the current implementation of the matrix is a 2×2-meter grid with horizontally the intrinsic severity, from A to D, and vertically the team feelings : 

As you can notice the first line is special. Bugs that cannot be quickly evaluated go there and need special attention as basically the team do not know where it comes from.

In the above example the two bugs in [A/???] are highly critical and must be dealt with as quickly as possible since, for example, they happen all the time and threaten user’s data, and the team don’t know at all how to solve them. On the contrary the four bugs in [D/:-)] are likely to be very improbable bugs that would be very easy to solve with no real impact on the rest of the product. There is no emergency to treat them.

As we wanted to have a global stability indicator, we affected each row and column an arbitrary coefficient : A=20, B=10, C=5 and D=1; ???=4, : -)=3 etc. In the above example, each [A/???] bug is worth 80 points (20×4) whereas [D/:-)] bugs are worth 1 point each. The above matrix can thus be estimated : 20*(2*4 + 2*2) + 10*(1*4 + 4*3 + 2*2 + 1*1) + 5*(1*4 + 4*3 + 4*2 + 2*1) + 1*(2*3 + 1*2 + 4*1) = 592

This approach facilitates decision making about scheduling while providing a good visualization and an easy-to-understand stability indicator. It can also break the last silo that might still exist in an Agile organization : the one between product owners and teams.

Do you use a visual tool for bugs with your teams? How do you prioritize them? Go share your experience and answer this follow-up question in pm.stackexchange.com