Measuring the Brand: The Kitchen Test and the Annual Pulse

Friday, 15:50, at Stonebridge Insurance, a year after the claims director's printer. Farah runs the five-person desk and has spent the year doing most of what this series describes. That includes the five-person audit, the monthly note, the rewritten acknowledgement, and a recovery playbook that has been used twice. Her director, Ben, has just asked a reasonable question. "Has it worked?" Farah has a dashboard that says tickets are closing faster and a satisfaction score that has drifted up a little. Neither of those is what Ben asked. He asked whether people say something different about IT in the kitchen now. Farah realises she has been changing a sentence for a year. She has never decided how she would know if it had changed.

Measuring the brand means tracking a small set of signals that move when the sentence people say about IT moves. It also means reading them together against the desk's own past rather than anyone else's. None of the signals is the brand. A reputation is a sentence in someone's head, and no tool counts those. But a sentence casts shadows: in what people say when asked and in how it felt to get help. It casts shadows in the thanks that arrive without a survey and in whether people use the desk's doors or go round them. And it casts shadows in what a whole organisation says once a year when invited to. This article is part of our IT's Internal Brand series. And it is the last one, because measuring is how you find out whether the other five did anything.

It covers what you are measuring and what you are not. Next come the kitchen test, which is the audit repeated, and the experience numbers, borrowed from the series that owns them. Then come unsolicited thanks, counted; the walk-up share and the shadow channels; and the annual pulse of five questions. Finally, it covers reading the five signals together on one page, a short war story, and the brand scorecard to copy.

What You Are Measuring, and What You Are Not

You are measuring change in the desk's own signals over time. And you are not measuring the desk against any other desk, any benchmark or any target. The brand is specific to this organisation, this history and this printer. And the only comparison that means anything is with the same organisation a year ago. A desk that starts with a reputation for silence and ends with one for honesty has done something remarkable. And no external number would show it. Set the baseline in the first quarter, and compare with that.

You are also not measuring people. None of the five signals in this article is cut by technician. The reason is the same one that applies to every per-person number in this library. A reputation is made by the desk as a whole, by its tools and its habits and its hours. Attaching it to an individual produces ranking, not insight. Where a signal points at one person's work, that is a coaching conversation, held privately, under the rules in Satisfaction by Category, and Carefully by Agent. The scorecard shows the desk. The desk shows the person, if it must, somewhere else.

Finally, none of the signals is a target. The moment "unsolicited thanks per month" becomes a number the desk is asked to raise, technicians will start fishing for thanks. And the signal will measure the fishing. The cures for a number that has become a goal are in Metric Gaming and Its Cures. And the simplest one applies here: the scorecard is read, discussed and acted on, and nobody's objectives mention it. A brand that is being measured for a bonus is being measured for the bonus.

The Kitchen Test

The kitchen test is the five-person audit from the first article in this series, repeated once a year with five different people, and scored. The questions do not change. The fifth one asks, "describing IT to a new colleague in one sentence, what would you say?" That is the one that gets scored, because it is the brand in the person's own words. Score each answer one of three ways: a warning ("it'll be Thursday"), neutral ("they're fine"), or a recommendation ("they'll tell you before you've noticed"). Write the three counts on the scorecard beside last year's. Five sentences cannot carry a percentage, so do not compute one. Three warnings down to one is a result. "Twenty percent fewer warnings" is a joke with decimals.

Keep the five sentences, verbatim, year on year, because they are the most persuasive artefact the desk will ever have. A page might show "ask Lena, IT takes a week" above "they get back to you within the hour now", from different people a year apart. Such a page says more to a director than any chart. It also tells the desk which sentence it is now fighting. The sentence changes as the desk improves, from speed to tone to scope, and the audit tells you which one is current. The method of asking, choosing the five and listening without defending is in Why IT Has a Brand Whether It Wants One or Not and is not repeated here.

Between audits, there is the informal kitchen test, which is simply the desk lead spending time where people talk and listening. Ten minutes a month in the kitchen, the canteen or the open-plan corner where a department sits is not a measurement, and it is not nothing. It catches the sentence as it changes, months before the audit confirms it. And it catches the new sentence the desk has not heard yet. (It also catches what people think of the new printers, which is free.)

The Experience Numbers

The second signal is how getting help felt, which the experience series measures properly and this article only borrows. The satisfaction score, trended by quarter and read with its response count, is the first part. The effort question, "how easy was it to get this sorted?", is the second. And the moments that matter in a ticket's life, from the first reply to the close, are the third. How to measure each honestly, without a named instrument and without being fooled by who answers, is in Measuring Experience Honestly. The brand scorecard takes two numbers from it: the satisfaction trend and the effort trend. Each is a count of good, middling and poor responses by quarter rather than an average.

The experience numbers are the only signal on the scorecard that most desks already have. They are not enough on their own because they come from people who raised a ticket and answered. That is the desk's friendliest audience. The brand lives among the people who did not raise a ticket, or did and did not answer. The other four signals exist to hear from them. The rating request may go out with the resolving reply, as it does in QuerySolv and in many tools. If it does, and satisfaction appears in the standard reports, the trend is a report rather than a project. Read it beside the response count, every time, because a score from nine people is a score from nine people.

Read the experience numbers for direction, not level. A satisfaction trend that has crept up for four quarters while response counts held steady is a signal that the moments are getting better. A score that jumped in one quarter on half the responses is a signal that something changed in who answered. The traps in reading a survey that only some people answer are in Survey Response Bias and How to Read Around It, and the scorecard inherits them. The brand article adds only this: if the score is up and the kitchen test is not, believe the kitchen.

Unsolicited Thanks, Counted

Unsolicited thanks is any thank-you that arrives outside the survey. It might be an e-mail to the desk lead about a technician or a reply to the monthly note. It might be a mention in another department's team meeting that gets passed on, a message copying someone's manager, or a card. Nobody asked for it, which is what makes it the purest brand signal the desk has. A survey response is prompted and takes a click. An unprompted thank-you took effort from someone who had nothing to gain. It almost always names the moment that caused it, which is the second thing the desk wants to know.

Count them with a tally, one line each, in a note or a spreadsheet kept by the desk lead. Record the date, the department it came from, the moment it refers to and nothing about who it praises. The last part matters. The tally is a count of moments, not of people. A line that says "classroom visit, start of term, got the whiteboard working before the lesson" is more useful than a name. It tells the desk which kind of moment builds the brand. Over a year the moments cluster. Stonebridge's clustered around two things: the follow-up a week after a problem, and the loan laptop that arrived with the user's files already on it. Neither was in the service level. Both were in the brand.

Read the count as a small number, because it is one, and read its absence as carefully as its presence. A desk that gets two or three a month and then gets none for a quarter has had a bad quarter that the survey may not have caught. A desk that gets none at all may be doing nothing anyone notices. Or it is doing it through a channel where thanks has no way to arrive. And the second is more common than people think. A desk that signs nothing with a name receives no thanks addressed to one. Count the complaints that reach the desk lead's manager the same way, as the mirror image, with the same columns. The two tallies side by side are the brand's own weather report.

The Walk-Up Share and the Shadow Channels

The fourth signal is the share of the desk's work that arrives through its own doors and gets logged. Compare it against the share that arrives by hallway, direct message to a technician or e-mail to a personal address and does not. People go round the desk when they do not trust the desk, and they stop going round it when they do. That makes the walk-up share a brand signal in disguise. A falling share of unlogged work, with total work steady, means people have decided the queue is the faster route. That decision is the brand changing, measured in behaviour rather than in words.

The measurement itself belongs to the walk-up series and is not rebuilt here. Counting the hallway, logging it with its own source, and tracking the shift from unlogged to logged work month by month is in Measuring the Shift to Logged Work. The scorecard takes one number from it: the share of logged requests that came through the desk's own doors. These did not get logged after the fact from a hallway or a technician's inbox. If you run QuerySolv, every door already lands in one queue. Those doors are the web form, e-mail, chat and the hallway request a technician logs on the spot. So the share is a question of which source each ticket carries. In any tool, it needs the source recorded honestly, including when the honest source is "walk-up, logged afterwards".

The companion number is the quiet department. Tickets per head by department, compared across departments, shows which parts of the organisation are not using the desk at all. A department that raises a third of what its size predicts has either no problems or another route. The method is in Per-Department and Per-User Patterns. The brand reading is that a quiet department coming up toward the others over a year is a department that has started to trust the desk. And that is worth a line on the scorecard with the department named and no individual in it.

The Annual Pulse of Five Questions

The annual pulse is a five-question survey sent once a year to everyone in the organisation, not just to people who raised a ticket. It goes out in the same week each year and is anonymous. Four questions are answered on a scale and one is answered in a sentence. The pulse is the only signal on the scorecard that hears from the people who never contact the desk, which is the audience the brand is mostly made of. It is short because long surveys are answered by the people with the most to say, and the desk already hears from them.

The four scaled questions are these. "When you need IT help, do you know where to go?" "The last time you asked, did you hear back when you expected to?" "While you waited, were you told what was happening?" "Did the people you dealt with treat you as a colleague?" Each is answered from "no" to "yes" on a five-step scale, reported as counts in each step rather than as an average. An average of a five-step scale is a number that describes nobody. The fifth question is the kitchen test's fifth question, asked at scale: "In one sentence, how would you describe IT to a new starter?" Read every answer. Score them warning, neutral or recommendation, the same way as the audit. Keep the sentences.

Run it the same week each year and send one reminder. Report the response count beside every result. A pulse answered by a tenth of the organisation is a pulse of the tenth who felt strongly. Do not add questions. The moment the pulse asks about the portal, the new laptops and the phone system it becomes an IT survey, and IT surveys are deleted. Five questions, one sentence, one week a year. The desk at Stonebridge got its first pulse back from a little under half the organisation. Farah had been told that was unusually high. She attributed it to the questions being answerable in under a minute. The results were less flattering than the satisfaction score and more useful than anything on the dashboard.

Reading the Five Signals Together

No single signal means anything on its own. And the scorecard exists to put the five on one page so that they can be read against each other. The table below says where each comes from, how often it is read, what moves it and what a change usually means. The right-hand column is the one to argue with; it is drawn from desks I have run and watched. And your organisation will teach you its own readings within two years.

Some pairings are worth knowing in advance. Satisfaction up and the pulse flat means the desk is getting better for the people who already use it. The improvement has not yet reached the ones who do not. In that case, the walk-up share and the quiet department are where to look next. Thanks up and satisfaction flat means the moments that build the brand are not the ones the survey measures, which is common. In that case, the tally will say which they are. Everything up and the kitchen test unchanged means the sentence has a lag, which it always has. In that case, the audit next year will probably catch up; keep going. Everything flat for two years means the desk changed its reports and not its moments. The scorecard cannot fix that. It can only say so.

What to Do With the Scorecard

Read it with the team once a quarter and with the director once a year. In both meetings the question is the same: which moment should the desk change next? The scorecard is a diagnostic, not a report card. And the output of reading it is one change, with an owner and a date, written at the bottom of the page. A desk that reads its scorecard and changes five things has changed nothing it can measure. A desk that changes one and watches the signals for two quarters learns what that change did. The brand improves one moment at a time, which is slow. It is also the only speed at which the brand has ever been seen to improve.

Give the director the sentences, not the counts. Ben's question at Stonebridge was whether people said something different now. The answer that satisfied him was a page with last year's five sentences above this year's. It also had two pulse results side by side, and the thank-you tally's two clusters named. The counts were on the page too. He did not read them. He read "they come and tell you before you've noticed" under "it'll be Thursday", and asked what the desk was going to change next.

A Short War Story: The Thank-You Tally at Pinecrest Academy

Owen ran the two-person desk at Pinecrest Academy, a school with about 90 staff. Owen kept a thank-you tally for a year mostly to see whether anyone would notice the desk at all. By the summer the tally had twenty-three lines, and nineteen of them referred to the same moment: the first week of each term. That was when the desk walked every classroom before lessons and checked the screen, the sound and the login. The desk also left a card with a name and an extension on the teacher's desk. The following year the walk-round was dropped to free up time for a laptop refresh. Nobody announced the change. And the tally for the autumn term was one line. The complaints reaching the head about "IT not being around" rose from none the previous year to four. The head raised the desk's reputation, unprompted, in Owen's appraisal. Owen restored the walk-round the next term at a cost of about two days per term. The tally and the head's view recovered inside two terms. The tally had not measured the brand; it had found the one moment the brand was made of. And that was the moment the desk had cut without looking.

The Brand Scorecard

The scorecard below is one page, filled in quarterly. It has the five signals, their baseline from the first quarter, this quarter's reading and the trend. At the bottom are the sentences, verbatim, year on year, and the one change the desk will make next. Keep every quarter's copy. The scorecard's value is in the stack, not in any single page. The stack is what you hand the director when they ask whether it has worked.

IT BRAND SCORECARD — ONE PAGE, QUARTERLY
Desk: ______________   Quarter: ____________   Baseline set: ____________   Filled in by: ______________

1. THE KITCHEN TEST (yearly; fifth-question sentences scored)
   Warning ____   Neutral ____   Recommendation ____   (last year: ____ / ____ / ____ )
   Informal listen this quarter, one line: _____________________________________________

2. THE EXPERIENCE NUMBERS (quarterly; counts, not averages; response count beside each)
   Satisfaction: good ____  middling ____  poor ____   of ____ responses   (baseline: ____/____/____ of ____)
   Effort ("how easy was it"): easy ____  middling ____  hard ____   of ____   (baseline: ____/____/____ of ____)

3. UNSOLICITED THANKS (tally of moments, no names)
   This quarter: ____   Last quarter: ____   Baseline quarter: ____
   Moments they refer to, grouped: _________________________________________________
   Mirror: complaints reaching the desk lead's manager this quarter: ____  (baseline: ____)

4. THE WALK-UP SHARE AND THE QUIET DEPARTMENT
   Logged work through our own doors: ____ of ____ requests   (baseline: ____ of ____)
   Logged afterwards from hallway / direct message / personal inbox: ____   (baseline: ____)
   Quietest department for its size: ______________ at ____ per head vs desk-wide ____ per head

5. THE ANNUAL PULSE (yearly, same week; counts per step; response count)
   Responses: ____ of ____ staff   (last year: ____ of ____)
   Know where to go ............ no ____ ____ ____ ____ ____ yes
   Heard back when expected .... no ____ ____ ____ ____ ____ yes
   Told what was happening ..... no ____ ____ ____ ____ ____ yes
   Treated as a colleague ...... no ____ ____ ____ ____ ____ yes
   One-sentence answers: warning ____   neutral ____   recommendation ____

THE SENTENCES (verbatim; keep every year's)
   Last year, most repeated: "________________________________________________"
   This year, most repeated: "________________________________________________"

READ TOGETHER
   What moved, and what did not: _____________________________________________
   The one moment we will change next: _______________________________________
   Owner: ______________   By: ____________   Re-read the signals on: ____________
   Not cut by technician: [ ]   Not anyone's target: [ ]   Response counts shown: [ ]

Farah filled in the first scorecard the week after Ben's question. The baseline came from the audit she had done a year earlier and the survey data she already had. She had to reconstruct the walk-up share from ticket sources for three months, which was the only part that took real effort. The page did not say the brand was fixed. It said that the sentence had moved from the wait to the tone and that the thanks clustered around follow-ups. It also said that the claims department was still the quietest in the building for its size. The change at the bottom was a walk to the claims department. It had, after all, started with their printer.

Frequently Asked Questions