Latest US poll: 63% say gas prices are affecting their household budget a great deal or a moderate amount
See the pollMaxDiff (best-worst scaling)
When people rate a list, almost everything comes out as important. MaxDiff asks them to pick the most and least important from small sets, so you get a ranking with clear gaps between items.
picked most−picked least
The problem
Rating scales let people say yes to everything. You end up with a list of near-identical scores that cannot tell you what to cut. MaxDiff takes that option away. Each screen shows a handful of items, and people choose the one that matters most and the one that matters least. Over a short run of screens, those choices add up to a ranking with real distance between items.
Asked to choose: screen 3 of 12
Which would you value most, and least?
Pick the one you would value most and the one you would value least.
Play with the model
Switch the audience and watch the ranking re-sort. Flip to most and least picks to see which features split opinion.
Switch the group: the ranking re-sorts. The bar ends show each score's range; overlapping ranges are a tie.
Drawn on the worked example further down: Example: 16 feature ideas for a home workout app
Scores are relative: they show which items beat which, not whether any of them matters.
Items whose ranges overlap are shown as a tie, not forced into an order.
Below about eight items, a simple ranking question often does the job.
In CatalyseAI today
In CatalyseAI today, a Ranking question puts a short list in order and a Distribution question shares 100 points across items, and every report compares up to five audience groups with significance markers.
How the model works
Claims, features, benefits, messages, flavors or names. Items should be the same kind of thing and should not overlap, so we tidy the wording before anything goes live.
The list: 16 feature ideas
Same kind of thing, no overlaps: the wording is tidied first
The list is split into small sets, usually of four or five items. Every item appears about equally often, alongside every other item, and in each position on the screen.
A balanced design
Screen 112081505
Screen 214040801
Screen 314161112
Screen 409041115
Screen 501051506
Screen 605121016
16 items in sets of 4: 12 screens, each item seen 3 times
One set per screen and two taps: most and least. It works on a phone and gets around the way different people use rating scales differently.
Screen 3 of 12
Which would you value most, and least?
MostLeast
Plans that adapt to your progress
10-minute express workouts
Workouts with no equipment
Workouts that play offline
Every item gets a score. Scores add up to 100 across the list, so you can see how far ahead the leaders are and where items tie.
Scores that add up to 100
The leaders, the ties and the gaps between them
The deliverables
Every CatalyseAI report downloads in these formats.
Every item scored and sorted, with a line showing where the average item sits.
How often each item was picked as most and as least important, which shows items that split opinion.
Scores for each audience group side by side, to show where one message will not fit everyone.
Use the results to find the small set of items that appeals to the largest number of people.
A score for every person and item as a CSV file, for your own analysis.
The terms, in plain words
Worked example
A product team has sixteen feature ideas for the next release of a home workout app and room to build about four. The study screens for US adults who used a fitness app in the past month. Each person sees twelve screens of four features and picks the one they would value most and the one they would value least. The report ranks all sixteen, shows which leaders are clearly ahead and which sit in a tie, and compares paying subscribers with people on the free plan.
FAQ
Ten to thirty is the comfortable range. Below about eight, a simple ranking question often does the job. Well above thirty, the study needs more people so that each item is seen often enough.
It depends on the list. Twelve items in sets of four, each shown three times, means nine screens. Twenty items in sets of five means twelve. We keep the number low enough that people stay focused.
A common guide is at least 300 people in total, and around 200 for each group you want to read on its own. Long lists need more.
Not on its own. Scores are relative: they show which items beat which, not whether any of them is important. An anchored version adds a question that splits the list into items that matter and items that do not. Ask us if you need that line.
Putting every item in order gets hard past about eight items, especially on a phone, and the middle of the ranking becomes guesswork. MaxDiff breaks the job into small, quick choices.
Related
Share your claims, features or messages. We will check the wording, suggest the set size and tell you how many people the list needs.
Nearest office: Orlando, FL (corporate office)