Benchmarking: how to provide valuable comparisons around UX research | by Kai Wong

Take snapshots of problematic outcomes to help show the impact of UX

“[Data] gains it’s meaning from the comparison.” That quote helped me understand what was missing from my re-design recommendations.

When re-designing a product or application, you might have a completely different understanding of what that means compared to other team members. You might imagine it as a complete overhaul, while others think it means “keep mostly everything but touch up a few places.”

Change is difficult, especially when it involves significant effort. This is especially true if some team members were on the project’s previous iteration: they may prefer something other than scrapping everything they’ve done.

But this is where that quote led to a breakthrough. You see, I was making the wrong argument. In these cases, you don’t need to convince your team that your design solution is “Great.”

You must convince them it will be “Better than before.” To do that, we can use the power of benchmarking to make our point.

How benchmarking helps with the problem of qualitative comparison

Qualitative user research offers us many insights, but one of its disadvantages is that it doesn’t offer us an easy comparison point.

With quantitative research, for example, we can quickly look at a metric at two points and make a statement like “We are performing 8% better now compared to 6 months ago.”

On the other hand, if “4/5 found this interface frustrating” 6 months ago, and it’s down to “3/5 users” now, that means next to nothing. It could be that you improved the interface, or you just so happened to choose a more resilient user.

This is where benchmarking can help you. UX benchmarking evaluates a product’s user experience using metrics to gauge performance against a meaningful standard.