A tiny change in wording can change survey findings

.
It has been demonstrated repeatedly that a small change in survey question wording can change respondent answers. For accuracy, it is important to use the exact survey question wording and exact response category wording when reporting on survey findings — to the extent possible.

.

Real World Survey Example

Not what they said: “One-In-Four Satisfied Customers Don’t Come Back”

April 21, 2024

LINK TO ARTICLE

The author of this article analyzes this one survey question:

“If you were to rate a customer experience on a scale of 1 to 5 – where 1 is bad, 2 is fair, 3 is average or satisfactory, 4 is good, and 5 is excellent – how likely is it that you would return to this company or brand if you rated them a 3?” [Never, Not Likely, Not Sure, Likely and Very Likely]

There are a number of issues with both this question and with the survey response scale. Here we’ll focus just on paraphrasing.

The author reports that  23% of customers who have a “satisfactory experience will not likely or never come back.”  This finding is reasonable; however, the author warns that a large minority of “satisfied customers” don’t come back.

The author has paraphrased “a satisfactory customer experience” as “a satisfied customer.” In everyday English, ‘satisfactory’ is a lukewarm, adequate rating, while ‘satisfied’ is a positive rating.

By using “satisfactory” interchangeably with “satisfied”, the author presents an interesting, surprising and actionable finding, albeit one for which he has no data and is most likely untrue.