Using Relative Weighting to Determine Product Backlog Priority
The first step is to assign a weight to the relative benefit of a feature. A benefit is the advantage for users of having the feature in the final product. Next is to assign the relative penalty. The penalty is the consequence for users of not having the feature in the final product. (Assessing benefits and penalties accomplishes the same thing as the Kano method’s functional and dysfunctional form of the question.) The weights are arbitrary numbers that represent how your company values benefits and risks of features. It is very similar to how a teacher determines what weight to give homework, attendance, quizzes, and tests in determining the overall grade; it will vary from teacher to teacher. If you decide that the benefits outweigh the penalties, make the weight higher than the penalty by whatever ratio you see fit. If you decide that penalties outweigh benefits, adjust the weighting accordingly. In the example in table 2, we gave the benefit a relative weight of 2 and the penalty a relative weight of 1, meaning that the benefit will be a larger factor in determining the final priority.
In the same way, we determine the weight of cost percentage and risk percentage. In the following example, risk was not as much of a concern to the project, so cost percentage was given weight of 1 and risk percentage a weight of 0.5. (Note that, although benefit and penalty are weighted before the value percentage is calculated, cost and risk are weighted as percentages.) If you have a high-risk project, you might weight risk higher than cost.
Now that the weights are set, we ask the users to rate each feature’s relative benefit and relative penalty. Each benefit and penalty is rated on a set scale—Weigers recommends 1-9, but you could choose a different scale as long as you’re consistent. The relative benefit is the value that the feature will deliver; relative penalty is the potential negative impact of not doing the feature.
For example, let’s say one possible feature is “Make the widget comply with Sarbanes-Oxley regulations.” There is virtually no relative benefit to users, but the relative penalty to the business is great - not doing it could put the company out of business. As such, we might see a score of 1 or 2 for the relative benefit and a score of 8 or 9 for the relative penalty.
In the following example, users rated the relative benefit of the feature “Query status of a vendor order” as a 5. They rated its relative penalty as a 3. To derive the total value of that feature, we multiply the two numbers by their relative weights and add them together:
(Benefit * Weight) + (Penalty * Weight) = Total Value
(5 _2) + (3_1) = 13
In this case, the total value for the feature is 13 points.
When we get the total relative value and value percentage for the features, we move away from the users to get insight from the team. We ask the team to estimate the relative cost to implement each feature using the same scale. Karl Weigers explains, “Developers estimate the cost ratings based on factors such as the requirement complexity, the extent of user interface work required, the potential ability to reuse existing designs or code, and the levels of testing and documentation needed.”
After we estimate relative cost, we estimate relative risk. Again, Weigers says, “Developers estimate the relative degree of technical or other risk associated with each feature on a scale from 1 to 9. An estimate of 1 means you can program it in your sleep, while 9 indicates serious concerns about feasibility, the availability of staff with the needed expertise, or the use of unproven or unfamiliar tools and technologies. The spreadsheet will calculate the percentage of the total risk that comes from each feature.”
After we note the values for relative benefit, penalty, cost, and risk, we sum up each column. These totals will be used to calculate the percentages.
- Total value percentage: Divide the sum value of the relative benefit and penalty by the total sum at the bottom. In the following example, this is 13 / 154 = 8%.
- Relative cost percentage: Divide the relative cost value by the total relative cost sum at the bottom. In the following example, this is 2 / 42 = 4.8%.
- Relative risk percentage: Divide the relative risk value by the total relative risk sum at the bottom. In the following example, this is 1 / 33 = 3%.
- Priority: Divide the value percentage by (cost percentage * cost weight) + (value percentage * value weight). In the following example, this would be 8.4% / ((4.8% * 1) + (3% * 0.5). This gives the priority value (1.345).
After we obtain the priority values, we sort the priority column in descending order so that the highest priority items are at the top. As items are added to the product backlog or more information about a story emerges, we’ll need to reassess priority.
In the end, the sheet looks like this table:
By taking this approach, you can better understand the ranges that work for the team and for the stakeholders. It also helps to better ground discussions because it can be difficult to objectively factor in elements such as benefit, penalty, cost, and risk for each feature.
Weigers explains how to make the model more closely match your reality:
“Calibrate this model for your own use with a set of completed requirements or features from a previous product. Adjust the weighting factors until the calculated priority sequence agrees with your after-the-fact evaluation of how important the requirements in your test set really were. This model can also help you make trade-off decisions when you’re evaluating proposed new requirements. Estimate their priority using the model to tell you how they match up against existing requirements, so you can choose an appropriate implementation sequence. Any actions you can take to move requirements prioritization from the political arena into an objective and analytical one will improve the project’s ability to deliver the most important functionality in the most appropriate sequence.”
Karl Weigers has written two books that go into more detail on relative weighting: Software Requirements, Second Edition and More About Software Requirements: Thorny Issues and Practical Advice.