Pairwise ranking of objectives versus simple weighting

/
6a00e54efe8aa58834010536c22ebf970c-800wi.png

Traditional "project scoring" systems we see look like this... a list of projects in a spreadsheet scored against some sort of measurement criteria. Sometimes the criteria is weighted by importance.

The "weighting of criteria" approach does provide some degree of influence over the project scoring results, but it fails to capture the proportional relationships between criteria or what we like to call "Objectives."

Rather, we use a "pairwise"technique to compare the relative importance of one Objective over another. Further, this method of generating weighted values for each Objective provides dynamic group discussions between team members when facilitated correctly.

6a00e54efe8aa58834010536b8514e970b-500wi.png

In the project ranking example above I have five criteria or "Objectives" that I would like to achieve with my new product portfolio (of five projects). For example, "Strong Customer Engagement" is my most important Objective, i.e. I want to favor projects that have strong customer engagement. The measurement criteria for this Objective includes:

  • Low: No clear customer engagement

  • Med: Some evidence of customer engagement exist. No clear sign that the decision maker from customer side is engaged.

  • High: Senior management from both sides fully engaged.

In this case we went though a pairwise comparison of each Objective (with the product line management team). We see "Strong Customer Engagement" being compared to "Lead Customer Ranking" (above example). They reach a consensus that "customer engagement" was more important (strong) than "lead customer" with respect to achieving their goal of determining which development projects to fund. Pairwise analysis is a core element of Analytic Hierarchy Process (AHP).

The process is repeated for each cell intersection until all Objectives are evaluated. This mathematical process results in values for each Objective that sets their respective priorities with respect to one another and the overall goal statement.

Customer Engagement (34.7%) is about six-times more important than Technology Differentiation (6.3%). This also tells us that Customer Engagement and ROI are really the driving Objectives that will influence our project funding decisions. I also know from this that we've been 82% consistent in our pairwise judgments (>80% is what we are striving for in decision models).

6a00e54efe8aa58834010536c23ac4970c-320wi.png

Further, we can simulate the impact of changing Objective weightings on the project ranking (example, above).

I made Technology Differentiation much more important than any other Objective, notice how "Terra Project" dropped from second to last place in my development portfolio.

Pairwise Analysis permits us to explore the relationship between Objectives, not just the importance of a single Objective in addition to being able to study the proportional relationships between different Objectives.