Latest US poll: 63% say gas prices are affecting their household budget a great deal or a moderate amount
See the pollData quality · Response checks
In a CatalyseAI study, every respondent is checked for rushing, straight-lining, gibberish and off-topic answers, and video answers are checked while they are recorded. Each flag comes with a reason, and you decide who stays in the data.
Creamy, and not too sweet. I would put it in my coffee.
On topic · 0.94sdfg sdfg kjhkjh
GibberishI need a new phone soon.
Off topic · 0.06Video answer, 0:41
Too dark: record againWhy it matters
Some people rush. Some tick the same box all the way down a grid. Some type "asdf" or paste one sentence into every open question. Some record a video in the dark, or talk about something else. Left in the data, these answers flatten real differences between groups and fill word clouds with noise.
Finished far faster than everyone else.
FlaggedOne answer on every row of a grid.
FlaggedRandom letters in an open question.
FlaggedAn answer about something else.
FlaggedA video nobody can see.
FlaggedThe gate
Follow 1,000 completes of an oat drink survey through the checks, layer by layer.
Video answers are checked as they are made: the camera is on, the person stays in view and there is enough light. If a video fails, the person is told why and asked to record it again.
Too dark: move somewhere brighter and record again.
AI reads each typed or recorded answer and judges whether it is about the question you asked.
Relevance from 0 to 1: is the answer about the question you asked?
Once answers are in, each respondent is compared with everyone else: time on each question against that question's median, one value down a whole grid, numbers far outside the normal range, the same text pasted into several questions.
The validation summary shows a pass or fail for each check, with the reason. Mark people Complete or Terminate one at a time, upload a list of IDs, or auto-reconcile so passes become Complete and fails become Terminate.
Eight checks
Every flag names the check that caught it, so you can see why before you decide.
The interview, or a single question, took far less time than is normal for this sample. You can also set a fixed minimum length.
The same answer on every row of a grid or rating battery.
A typed answer is missing or too short to mean anything, or the same text appears in several questions.
Keyboard mashing or random characters.
Offensive language in a typed answer.
A typed or recorded answer is not about the question.
A number, or the interview length, far outside the normal range for this sample.
The camera is on, the person stays in view and there is enough light.
What you get
One row per respondent, with their status, interview length and a pass or fail badge for each check. Hover or tap a badge to see why it failed.
One row per respondent, with their status, interview length and a pass or fail badge for each check. Hover or tap a badge to see why it failed.
Open a respondent to see each answer, the time spent against the median and the check it failed. A second table counts failures per question, so a confusing or overlong question stands out.
Force a pass or a fail and give a reason. Every change is logged, and undo puts the earlier status back.
In CatalyseAI, a complete you accept is paid for and a complete you reject is refunded.
Download the validation summary as a CSV to send to a sample supplier or keep with the project files.
How a speeder is defined
Against your own sample. A whole interview finished in less than half the median time is flagged, and each question is compared with its own median time. If you set a minimum length for the survey, anything faster is flagged too.
Flagged as speeders: under half the median time
Example
A grocery brand runs a CatalyseAI study with a brand rating grid, two open questions and one short video answer. After fieldwork, the Validation tab shows a handful of people who gave every brand the same rating, a few open answers that are keyboard mashing, and two respondents on the same device. The per-question table also shows many people rushing one long grid, which tells the team the grid was too long, not that the people were careless. The researcher keeps one flagged respondent whose short answer was genuine, notes why, and auto-reconciles the rest.
Many people rushed Q3. The grid is too long; the people are not careless.
On our own panelQuality fails earn nothing on our panelOn PocketsFull, members are paid for completes, not for clicks.
How members are paidFAQ
No. Answer checks flag people in the validation summary and you decide, one by one or with auto-reconcile. The exception is video: a recording that fails the camera or light check goes back to the person to record again. FraudCheck, which runs when people arrive, can terminate them on entry if you choose.
Against your own sample. A whole interview finished in less than half the median time is flagged, and each question is compared with its own median time. If you set a minimum length for the survey, anything faster is flagged too.
Yes. Force a pass or a fail and add a reason. The change is logged, and undo restores the respondent's earlier status.
Yes. The checks are worked out again from the stored answers each time the summary opens, so medians and duplicate matches always reflect the current sample.
Related
Book a demo and we will walk you through the Validation tab, the reasons behind each flag and how reconciliation works.
Nearest office: Orlando, FL (corporate office)