WHY MEASURE UX?
In the working environment, we measure things in order to make performance visible because we want to evaluate how well we’re doing, define targets and allocate resources for achieving those targets. Making performance visible means it can be managed objectively in order to uphold a certain standard or improve the bottom line. Performance measurement improves communications internally among employees, as well as externally between the organization and its customers.
WHAT IS THE USERINDEX?
The USERindex is a standardised means of measuring the user experience of a digital interface. While usability refers to how successfully a user can carry out a task with a given product, User Experience takes a broader view, taking the user’s individual perception of his entire interaction with the product and even the brand into consideration. Today, the success of a digital interface is determined not only by the functions it offers but also, and maybe even more importantly, by how the user feels about interacting with it.
HOW DO WE MEASURE USER EXPERIENCE?
We have developed the USERindex because we want to measure the user experience of a digital interface in a repeatable, scientific manner. This enables the product owner to determine the quality of the user experience. As a result, the product owner can derive insights for improvements, monitor the evolution of the user experience over time and benchmark it against other products.
“When you can measure what you are speaking about, and express it in numbers, you know something about it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarely, in your thoughts advanced to the stage of science.”
― William Thomson, 1st Baron Kelvin
What is the USERindex?
The USERindex is a standardised means of measuring the user experience of a digital interface.
We say “digital interface” because we are well aware that today, when we speak of “digital” we are no longer only referring to computers and desktop/web applications; we are referring to a series of devices such as wearables and smartphones as well as products which are recently being equipped with digital interfaces such as cars and fridges, for example.
While usability refers to how successfully a user can carry out a task with a given product, User Experience takes a broader view, taking the user’s individual perception of his entire interaction with the product and even the brand into consideration. Today, the success of a digital interface is determined not only by the functions it offers but also, and maybe even more importantly, by how the user feels about interacting with it. The subjective, more emotional factor is becoming the deciding factor. The USERindex takes this into consideration.
Each dimension is in turn defined by aspects which comprise both very objective facets of good user experience such as the accuracy of the information and the responsiveness of the interface, as well as very subjective ones such as whether the user liked using the interface.
View image below.
The USERindex comprises 10 questions which have been worded neutrally, avoiding jargon and which can be applied to any digital interface, be it a web application or a wearable device. This makes the index very powerful because it can be used to compare the user experience of different products even if they are very dissimilar in function and interface type.
We believe that an exceptional user experience is determined by 4 factors (which we call USERindex dimensions): Usefulness – Satisfaction – Ease of use – Reliability
We researched our choice of positively expressed statements (as opposed to a mix of negative and positive statements) and deliberately decided to go with them because we don’t believe that we need to negatively word the statements in order to achieve more accurate results. This has also been proven in a study where the SUS was deployed in both its original format of both negative and positive statements as well as an entirely positively worded version and the variance was found to be negligible (SUStisfied? Little-Known System Usability Scale Facts). We believe that the 4 dimensions are correlated and thus improving one dimension will have an impact on the others.
| DIMENSION | ASPECT | STATEMENTS |
|---|---|---|
| Reliability | Performance | I felt the site responded quickly |
| – | Accuracy | I found the information on the site to be accurate |
| Usefulness | Effectiveness | The site does what i need it to do |
| – | Usefulness | The information on the site was useful to me |
| Ease of use | Ease of use | I found the site easy to use |
| – | Navigation | It was easy to find what I was looking for on the site |
| – | Learnability | It didn’t take me long to learn how to use the site |
| – | Efficiency | It didn’t take me long to complete my task on the site |
| Satisfaction | Appeal | I found the design of the site appealing |
| – | Engagement | I liked using the site |
Summarising:
-
The USERindex measures more than just the usability of a digital interface; it takes the emotional aspect of user engagement into consideration
-
The USERindex is a tool for measuring the holistic user experience of a digital interface
-
The USERindex can be used to compare the user experience of different digital interfaces regardless of the devices or technology they run on
-
The USERindex can be used to continuously monitor the evolution of the user experience of your digital interface over time.
Why measure UX?
Today, we are constantly measuring how we perform
We have smart devices that record the number of steps we take, the GPS coordinates of a hike we walked, the depth and temperature of a dive we made. Cheap data storage allows us to accumulate this data, use it to plot our progress in charts and determine correlations. The cloud enables us to easily share our data with friends and like-minded strangers in order to compare achievements, harvest accolades and find new insights.
In the working environment, we measure things in order to make performance visible because we want to evaluate how well we’re doing, define targets and allocate resources for achieving those targets. Making performance visible means it can be managed objectively in order to uphold a certain standard or improve the bottom line. Performance measurement improves communications internally among employees, as well as externally between the organization and its customers.
So the question actually should not be ‘why measure user experience?’, it should be ‘why aren’t you measuring the user experience of your digital interface yet?‘
How do we measure user experience?
We have developed the USERindex because we want to measure the user experience of a digital interface in a repeatable, scientific manner. This enables the product owner to determine the quality of the user experience. As a result, the product owner can derive insights for improvements, monitor the evolution of the user experience over time and benchmark it against other products.
Why not use the SUS?
We like the SUS and we have used it extensively over the past years, but we find it lacking in the following areas:
-
it was developed prior to the emergence of the web (and digital interfaces such as mobiles, wearables and intelligent fridges)
-
it does take the design of a digital interface into consideration
-
it does not measure the user satisfaction.
In our digital era, where features and functions of a digital product are comparatively similar, you differentiate yourselves by the way you engage with your user. This dimension is lacking in the SUS.
“If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”
― H. James Harrington
How to use the USERindex
Ask your users to rate the 10 USERindex statements on a 5 item Likert scale
The USERindex is very simple to use. Ask your users to rate the 10 USERindex statements on a 5 item Likert scale (from strongly disagree/somewhat disagree/neutral/somewhat agree/strongly agree) and then use this USERindex Calculator spreadsheet to calculate the overall USERindex as well the individual score of each dimension.
When to use the USERindex
-
The USERindex can be used during formative usability tests to determine, mould and monitor the user experience during the development phase.
-
After go-live, it can be used in summative usability tests to validate the expected user experience; to determine a benchmark value and monitor its improvement during the subsequent product lifecycle.
-
It can also be used to compare the user experience of different digital interfaces in a competitive analysis.
How to interpret the score
The value of USERindex ranges from 1 to 5. A digital product with an exceptional user experience will have a USERindex of 4.7 or above; anything between 4.2 and 4.7 is good and a USERindex of below 4.2 means you have uncovered areas for improvement.
If the USERindex is used in a test environment, questions about performance will not reflect realistic use by end users. To minimize the impact of non-production-like circumstances, the score calculation in the USERindex Calculator can be configured accordingly.
Some tips:
-
Make the questions optional in order to avoid forcing users to evaluate an aspect they might not find applicable.
-
Replace the word ‘site’ with a more meaningful term if you feel ‘site’ does not work for your digital product. For instance, use ‘mobile site’ for a mobile website, ‘app’ for a native app, ‘watch’ for connected watch, etc…
Downloads:
Use this spreadsheet for entering your user data and generating charts.
Unformatted statements for copying into your online survey
Printable questionnaires
About Us
The USERindex was created by 2 user experience professionals
Why we created the USERindex
In our usability testing projects, we’ve used a number of different metrics over the years. While all metrics have their merits, we’ve been looking for something that is ‘quick and easy’ (think SUS), but also collects information on today’s metrics such as appeal and satisfaction. And we want it to be freely available (we don’t believe anyone should be paying for a good metrics system).
Our vision for the USERindex
We created and fine-tuned the USERindex, and find that it’s working well for us. We encourage anyone to use it. Please refer back to the USERindex.org site for further information. If you wish to anonymously submit your data to be added to the historical database, please send your file to feedback at userindex.org.
News:
Contact Us
Do you have questions or feedback you would like to share with us? Then mail us at feedback at userindex.org. We’d love to hear from you.
Share your data
If you wish to anonymously submit your data to be added to the historical database, please send your file to feedback at userindex.org. We will use your data to continuously improve our analysis of the correlations between the dimensions.