After years of working with customer satisfaction surveys and assessment projects, OpenMet has observed a series of patterns of errors or misunderstandings that many people make when addressing this type of project.
Without considering survey format, which we have already discussed in other entries on this blog, this post offers a short list or checklist – although far from exhaustive – of the 10 most common mistakes made when designing satisfaction surveys.
Initial errors in the design of satisfaction surveys
1. Not to define or be clear about the aim of the survey
Although it may seem absurd, the main problem when it comes to designing a survey in many cases is not really knowing what we want. Wanting to evaluate overall satisfaction (in which case, perhaps an NPS question would suffice) is different to wanting to know the exact causes of the dissatisfaction.
Wanting a system to input information into a dashboard or corporate control panel is different to a survey whose ultimate aim is to evaluate the services of the departments that work with customers.
So we need to be as precise as possible when we define the aim of the evaluation, the type of results analysis we want, which KPIs and metrics to use, and the type of action we intend to take once we have the results.
2. Thinking that we already know what customers think and what is important for them
This is a very common mistake in companies and leads to the design of questionnaires based on partial knowledge of the customers. This may result, for example, in the right questions not being asked or that critical aspects for customers – which the company may be unaware of – are not asked.
A good way to solve this problem is to carry out a more qualitative preliminary phase to the study dealing with a broad horizon of topics, where we can search for the aspects and main vectors that really do govern customer satisfaction.
3. Making satisfaction surveys too long or too short
In an attempt to maximize the number of answers, or so as not to tire customers, a survey can sometimes be so short that the results cannot be used for effective decision-making. On the other hand, if we want too many details, we can fall into the trap of making our surveys too long, which results in poor participation or poor quality answers. To solve this, there are several methods that we can use to design the survey with fewer questions.
Mistakes made with the questions of satisfaction surveys
4. Considering only evaluation questions
And not thinking about the demographics or information that we already have in the customer database. Questions that evaluate specific aspects are just as important as questions that give us information on the profile of the person answering.
If we don’t add any of these, our analysis of the results will be far poorer and, hence, we will be less capable of fixing problems.
5. Confusing questions or answers
We should reread all questions several times and make sure that we don’t make typical mistakes like asking two things in the same sentence, asking incomprehensible questions (or which the CEO of the company can understand, but none of the company’s customers can), asking questions with abbreviations or acronyms that not everybody knows, or using double negations that produce uncertain responses.
Answers are also important. We must not use similar or repeat answer options, or scales of strange or unpopular answers (e.g. scales from 1 to 10 when everybody has used the scale of 0 to 10 since they were children).
And we must take care to make sure that everybody is perfectly clear on what each possible answer means. We also need to make sure that the number of answers is not so long that the customer gets distracted.
6. Not empathizing with the customer
If we want the customer to feel comfortable answering, we need to get as close as possible to them.
Therefore, we should take care with the type of treatment that we adopt in our questions (friendly vs. formal), our gender bias (e.g. don’t use “he” in every sentence), in the aesthetics or language (the customer has to be able to choose the language if we are targetting customers who use different languages or if they are potentially multilingual).
7. Forgetting open questions
We should always leave an open field for customers to include what they want without having to adhere to the restrictions of closed questions.
Even if the sample is very big and the results may end up with more text than a novel, there are systems available today that can make a good analysis of suggestions.
Statistical errors in satisfaction surveys
8. Making guesses and attempting to make decisions based on samples that are too big or too small
Failing to give statistics the attention they deserve can turn out to be expensive if we want to make business decisions. To be able to guess the opinions of our customers based on results, we need to understand concepts such confidence interval and margin of error of the sample.
There are still many customers who take very important decisions based on samples that are too small, or they spend a lot of money asking lots of customers when they could potentially make the same decisions surveying far fewer people.
9. Wanting to make decisions based on an accuracy that is less than sampling error
This is a common error when we want to monitor KPIs or metrics associated with the survey. Decisions are often made based on variations in KPIs that are less than the sampling error.
There is no sense in working with a sampling error of 5% and then producing a report on results with metrics of 0 to 100 with 2 decimals and making decisions based on small variations in these values.
10. Using biased samples without realizing it
It is pointless doing a customer survey if it turns out that those of a particular type (e.g. in the north) answer far more and we use the results to make estimates for all customers (north, south, etc.).
The aim is to obtain a sample of answers that is representative of the population. And if it is not, we will have to correct the results suitably, e.g. using weightings.
As we said at the start, this list is not exhaustive but these are some of the most typical errors. There are of course many more aspects to take into account if we don’t want to make mistakes in our customer satisfaction surveys, but we’ll leave those for another blog post☺.
