Satisfaction Surveys People Actually Answer

Thursday, 16:40, at Harbourview Clinic. Marcus runs the three-person desk that supports the clinic's two sites, and he has the month's survey results open: three responses, from 260 closed tickets. The survey is the one he inherited. It has seven questions and lives on the portal behind a login. It goes out two days after the ticket closes and sends two reminders. One of the three responses this month reads, in full, "I don't remember this ticket." Aisha, the practice manager, has asked for "a satisfaction figure for the board", and Marcus is looking at a figure built from two people and a shrug.

A satisfaction survey is the question the desk asks the requester at the end of a ticket. The design of that question decides how many people answer and what their answers are worth. The design that works is short enough to describe in one breath: one question, sent with the resolving reply, no login, no reminders, an optional comment box. Each of those five choices is a trade, and this article is about the trades. It is part of our Measuring Satisfaction series, which starts with what the score actually measures and comes back, in every article, to the same point. The number is only as good as the people who answered.

It covers why people do not answer, the one question and which one to ask, and sending it with the resolving reply. Next come why there must be no login, the optional comment box, which is where the value lives, and the case against reminders. Then it covers what a good response rate looks like for a small desk, as a worked example in ranges. Launching the survey and keeping the wording still, a short war story and the survey text to copy finish the article.

Why People Do Not Answer

People do not answer because answering costs more than they think the desk deserves. That sounds harsh and is simply the arithmetic of a full inbox. A requester whose printer now works has already received the thing they wanted. The survey asks them for a second thing, their time, and offers nothing back except the vague sense of having helped. If the second thing costs ten seconds and one click, a fair number of people will pay it. If it costs a login, a page load, seven questions and a scroll, almost nobody will. The few who do are the ones with a strong feeling to express.

Timing adds to the cost. A survey sent two days after the close arrives when the ticket has left the requester's mind. The first thing they have to do is remember what it was about. Some cannot. Those who can have to reconstruct a feeling about an interaction that is over, and reconstructed feelings are blander and less reliable than fresh ones. Marcus's "I don't remember this ticket" is not a rude answer; it is an accurate one.

Length, logins and reminders each remove a slice of the people who would otherwise have answered, and the slices are not random. The login removes the people who do not know their portal password, which is a group the desk should want to hear from. The length removes the busy and the mildly content. The reminders remove goodwill, which is harder to see in the numbers and more expensive. A survey that demands effort does not sample the requesters. It samples the requesters who had a point to make. The inherited seven-question survey at Harbourview had, for two years, been collecting a monthly census of the aggrieved and the very grateful.

One Question, and Which One

Ask one question. One question can be answered in the time it takes to decide whether to answer at all, which is the only window the desk gets. Two questions are a form. Seven questions are an obligation, and most people meet obligations to the help desk by closing the tab. The temptation to add a second question is permanent, because every stakeholder has one thing they would like to know. The only defence is a rule: one question, written on the definition card, and no exceptions for the steering group. Not "just one more". Not this quarter.

The question should ask about this ticket, not about the desk, and about the handling rather than the outcome alone. "How did we do on this request?" asks the requester to react to what just happened, which is what they can actually report on. "How satisfied are you with IT?" asks them to summarise a relationship, which they will do from memory and mood. "Was your issue resolved?" asks for a fact the desk should already know from the reopen rate. It gets a yes from people who were kept waiting a week and a no from people whose problem was outside the desk's gift. Keep the wording plain, keep it short, and keep it about the ticket in hand.

Offer a small number of answers, worded as labels, and let the requester choose one. Whether your tool shows numbers, faces or words is its business and its documentation's; the design point is the same either way. The labels should be unambiguous at a glance: a requester should never have to work out whether a four is good. The series' first article explains why the labels are not quantities and why the share of good answers is a better headline than an average. The survey design only has to make sure that "good" is a label somebody can click without thinking.

Asked With the Resolving Reply

Send the question in the message that tells the requester the fix is in. That message, the resolving reply, is the one piece of mail from the desk that the requester is certain to open. It is the one that says their problem is over. It arrives while the ticket is fresh, while the feeling is fresh, and while the requester is already reading something from the desk. The survey costs them one more glance and one click. Nothing else the desk sends will ever be opened as reliably. Spending that moment on anything other than the question is a waste of the best moment you have.

This is how many tools do it, and the detail matters for what the score means. The survey goes out with the reply, so a ticket closed without a reply is a ticket whose requester was never asked. That ties the response rate to the quality of the close, which is a feature rather than a flaw. It is why Avoiding the Silent Close is required reading for anyone who wants a satisfaction figure worth printing. If you run QuerySolv, the rating request rides on the resolving reply. The whole of the survey's mechanics reduces to one habit: resolve with a message, every time. Whatever your tool does, check how it sends the question, because if it is tied to the reply, every silent close is a vote never cast.

The resolving reply itself has to be worth reading, or the question underneath it will not be. A reply that says "Closing this ticket" and then asks "how did we do?" is asking the requester to rate four words. A reply might say what was wrong, what was done, and what to do if it comes back, and then ask the question. It is asking them to rate a conversation they just had. The shape of a good resolving reply is in What a Proper Close Looks Like; the survey is the last line of it, not a separate transaction.

No Login

The answer must be one click from the message, with no login, no portal, no "please sign in to continue". The reason is cost, again. A login is the most expensive single thing you can ask of a requester. It demands a password they may not have, a page they may not use, and a reason to bother that the desk has not supplied. The tools that get answers put the choice directly in the message or one click behind it, and identify the ticket from the link rather than from the person. The requester clicks "good", a page says thank you, and that is the whole transaction.

The trade is that a link anyone can click is a link anyone can click. If the requester forwards the resolving reply to a colleague, the colleague can rate the ticket; if the mail is opened by an assistant, the assistant can. In practice this almost never matters, because almost nobody rates a ticket that was not theirs. The few who do are usually the person whose problem it really was. The loss from a login, which is most of your respondents, is far larger than the loss from the occasional borrowed click. Take the trade. Then make sure that the link does not do anything beyond recording an answer, so that the worst a stranger can do is have an opinion.

No login also means no account, no password reset for the survey, and no ticket raised about the survey, which at Harbourview had happened. Twice. (A ticket asking for help rating a ticket is a sentence I had hoped never to write and have now written.)

The Optional Comment Box

Offer a comment box, and make it optional. The box is where the value of the survey lives, because the click tells you how someone felt and the comment tells you why. The why is the only part the desk can act on. A month of thirty clicks and eight comments is worth more than a month of sixty clicks and none. A desk reading its survey results should read the comments first, the counts second and the average, if anyone insists on one, last. The articles later in this series on acting on the feedback and on reading around the bias are both, in the end, about the comment box.

Optional is the important word. A required comment turns one click into a writing task. The writing task removes exactly the people whose click you most wanted. These are the ones who were fine, who would have clicked "good" and gone, and who now go without clicking. The people who stay to write are the ones with something to say, which is the bias you were trying to dilute. Make the box visible, make the prompt short and plain, and let the empty box be a complete answer.

Word the prompt as an invitation, not a form field. "Anything you want to tell us?" draws more than "Comments". "What could we have done better?" draws complaints and suppresses praise, which is a choice some desks make deliberately and should make knowingly. Whatever the prompt, say that a person reads every comment, because that is the only reason anyone writes one, and then make it true. A comment box nobody reads is a suggestion box in a locked corridor, and people can tell.

No Reminders

Ask once. Do not send a reminder. If your tool sends them by default, find the setting and turn it off. Some tools send none, some send one, some can be made to send several, so check yours. The case for reminders is that they raise the response rate, which is true in the narrow sense that a third e-mail produces more clicks than two. The case against is everything else. A reminder reaches someone who has already decided not to answer, and asks again. The answer it gets is partly an answer about the ticket and partly an answer about being asked twice. A reminder is a second survey about whether you liked the first one.

The people a reminder recruits are not the silent middle you were hoping for. The silent middle delete the reminder too. The reminder recruits the people who were nearly annoyed enough to answer the first time and are now annoyed enough. It also recruits the people who will click anything to make a recurring message stop. Both groups rate lower than they would have the first time, and neither tells you anything about the ticket. The response rate rises, the score falls, and the fall is an artefact of the survey, not of the service. That is the worst available outcome for a metric: more data, less truth.

Goodwill is the other cost, and it is the one that does not show in the numbers. Every message the desk sends that is not about the requester's problem spends a little of the requester's patience with the desk. A reminder spends it for nothing. The requester who ignored the survey was not withholding anything; they were busy, and the second ask tells them the desk values its chart over their time. Ask once, at the right moment, in the right message, and accept that most people will not answer. They were never going to. The ones who do are enough.

What a Good Response Rate Looks Like for a Small Desk

The response rate is the number of answers divided by the number of tickets that were offered the question. For a small desk it is the most important figure in the satisfaction report, because it says how many people the score describes. Every figure in this section is from the fictional desks in this series, stated as a range, and is offered as a shape to recognise rather than a target to hit. Your desk, your requesters and your resolving replies will produce their own number. The question is whether yours is in the region where the score can be read, and the arithmetic below shows what that region is.

Harbourview's inherited survey ran at about one answer per hundred closes: three in a typical month, from around 260 closes, of which perhaps 200 had a reply that carried the link. Marcus replaced it with one question in the resolving reply, no login, no reminders, optional comment. The first full month produced 31 answers from 240 tickets closed with a reply, which is about thirteen per hundred asked. The following quarter settled between nine and fourteen per hundred, with the spread depending mostly on how many tickets closed silently that month. Nothing else changed. The same requesters, the same desk, ten times the answers.

Across the fictional desks this series uses for its arithmetic, consider a one-question survey on the resolving reply with no login. It lands somewhere between one answer in twelve and one in five of the tickets asked, most months. Below one in twenty, something mechanical is usually wrong. That might be tickets closing without a reply, a link that leads to a login, a question people cannot parse, or a reply that nobody opens because it looks automated. Above one in three, look for a technician asking for ratings in person, or a department with a culture of answering everything, before you celebrate. The table below shows how each of Marcus's design changes moved the Harbourview rate and what it did to the honesty of the answers. That is the column that matters more.

What the rate means for reading the score follows directly. A desk closing 300 tickets a month with a reply, at one in ten, gets about thirty answers a month. That is enough to read as a share with a count beside it, though not enough to cut by team or by technician. A desk closing sixty a month gets five or six, which is a handful of anecdotes. It should report the quarter rather than the month, in words rather than decimals: "fourteen of seventeen answers this quarter were good, and here are the three that were not." The small-numbers problem, and how to read a month with nine responses, is the subject of the next article. The design job here is to make the handful as large and as honest as the desk's volume allows.

Launching It and Keeping the Wording Still

Tell people the survey exists before the first one arrives, in one short message to all staff. Explain that the desk will ask one question at the end of each ticket. It takes one click, there is no login, and a person reads every comment. The announcement does two things. It stops the first survey being mistaken for junk mail, and it tells the requesters what the click is for, which is the only incentive the desk can offer. Do not promise a prize, a draw or a league table. Promise that someone reads it, and then read it.

Fix the wording and leave it alone. The question, the labels and the comment prompt are part of the metric's definition. A change to any of them starts the trend again from zero, with a note in the report saying so. The urge to tweak is strongest in the second month, when the first results arrive and someone suggests that "how did we do?" would read better as "how satisfied were you?" It might. It would also make the second month incomparable with the first, and the third with the second if the suggestion recurs. Write the wording on the definition card, date it, and treat a change as a project with a start date and a line on every chart.

Brief the technicians, because the survey rides on their replies and they will have views. The brief has three parts. The resolving reply now carries the question, so it must be a real reply. Nobody is to ask a requester for a good rating, in person, by phone or in the reply itself. The per-person figures, when there are enough answers to produce any, are for coaching and not ranking. That is a promise the desk lead has to keep in public. The third part is the one technicians listen for. A survey introduced as a scoreboard will be gamed by the second week. The gaming looks like excellent customer service aimed at the people most likely to click.

A Short War Story: The Three Reminders at Pinecrest Academy

At Pinecrest Academy, a school with about two hundred staff and one IT lead called Ruth, the ticketing tool offered survey reminders as a setting. Ruth, wanting a response rate the governors would take seriously, set it to three: one with the resolving reply, one after two days, one after five. The response rate went from about one in ten to nearly one in four within a month. The governors were impressed until the score itself fell from mostly good to roughly half good. The comments read "fine, now stop e-mailing me", "third time asking", and, from the head of maths, a single word that Ruth chose not to include in the report. A member of staff complained formally to the head about "IT spam". The head asked Ruth to switch the survey off entirely. It stayed off for the rest of the term, which meant a term's worth of figures were unusable. The figures from the reminder months had to be discarded as well. Ruth relaunched the following term with one question and no reminders, and spent her first governors' meeting explaining why the chart had a gap and a reset in it. A reminder does not measure satisfaction; it measures patience, and patience runs out first.

The Survey Text

The template below is the resolving reply with the one question underneath it, written as a single message, because that is how it reaches the requester. The upper part is the reply and follows the shape of a proper close. It says what was wrong, what was done, how to tell it worked, and what to do if it comes back. The lower part is the survey, and it is deliberately short. It has one question, three labels as an example, an optional comment prompt, and one line saying that a person reads it. The rules block at the end is for the desk, not the requester, and is the part to pin above the monitor. Adjust the labels to whatever your tool shows; keep everything else.

RESOLVING REPLY WITH THE ONE-QUESTION SURVEY
(sent as the public reply that resolves the ticket; nothing follows it)

Hi ______________,

What was wrong: ________________________________________________
What we did: ___________________________________________________
How to tell it worked: _________________________________________
If it comes back, reply to this message and the ticket reopens
with everything we already know about it.

______________ (technician), ______________ (desk)

----------------------------------------------------------------
How did we do on this request?

      [ Good ]        [ Okay ]        [ Poor ]

Anything you want to tell us? (optional) _______________________
One click, no login. A person on the desk reads every comment.
----------------------------------------------------------------

RULES FOR THE DESK (not sent)
  One question. No second question for anyone.
  Same wording on every ticket. Changes are dated on the definition card.
  Sent once, with the resolving reply. No reminders.
  Comment optional, never required.
  Answer link needs no login and records an answer only.
  Nobody asks a requester for a good rating, ever.
  Comments are read within one working day by a named person.
  Response count and close count are printed with every score.

Marcus sent the all-staff note on a Monday, switched the survey over the same afternoon, and had more answers by Friday than the old survey had collected in a quarter. Aisha got her figure for the board: twenty-six of thirty-one answers good, from 240 tickets closed with a reply, five comments attached, two of them worth acting on. It was a smaller number than the governors of Pinecrest had been shown and a far larger one than Harbourview had ever had. For the first time it described people who remembered the ticket.

Frequently Asked Questions