Outlier Analysis: Reading the Ten Slowest Tickets Every Month

The first working day of the month, 08:30, at Bellweather Logistics. Dev runs a desk of four for two depots and a head office. Before anything else he sorts last month's resolved tickets by resolution time, longest first. He reads the top ten with his coffee. Ticket 7731, thirty-eight working days: a docking station on back order, chased once in week one and never again. Ticket 7702, twenty-nine days: "waiting on requester". But the requester had replied on day three. The reply had sat unread under the status. Ticket 7688, twenty-two days: a depot scanner that lost its network settings after every firmware update, four attempts, a vendor case, a real fix at the end. Ticket 7719, nineteen days: a licence request waiting for approval from a manager. The manager was on leave for two of those weeks. The deputy had not been told. Dev writes one letter against each ticket as he goes. By 09:30 he has ten letters, and seven of them are the same letter.

An outlier is a ticket whose resolution time sits far beyond the rest of the month's. The outliers together are the tail: the slow end of the sorted list. It is where the 90th percentile lives and the complaints come from. Outlier analysis is reading the ten slowest resolved tickets each month and finding where the time went in each. Sort each into one of four causes. Fix the cause rather than the ticket. The ticket is already resolved; nothing can be done for it. The cause will be back next month unless something is done about it. The ten slowest tickets are the cheapest way a desk has of finding out what that something is. This article is part of our Time to Resolution series, and it is about that one hour a month.

It covers why the tail is worth reading when the median and percentile are already on the report. It then explains how to pull the ten and how to read a history. Next come the four causes, with how each shows in the history and what fixes it. After that comes the quarterly tally, with a table, followed by the outlier that turns out to be a process problem. It finishes with what not to do with the ten, a short war story, and the outlier review sheet to copy. The numbers are Bellweather's and are an example from a fictional desk, not a benchmark.

Why Read the Tail at All

The median says what a typical ticket is like. The 90th percentile says how long the tail is. Neither says why. A 90th percentile of forty hours is a fact about the month; it is not a cause, and nothing can be done about a fact. The ten slowest tickets, read one at a time with the question "where did the time go?", produce ten causes, and causes can be fixed. The summary figures are the subject of Median and Percentiles, Not Averages. This article is what you do once you have them and want the tail to be shorter next month.

The tail is also where the desk's reputation is decided. Nobody remembers the median ticket; it was fine. The people who tell the regional director that IT takes a fortnight are in the tail, and each of them is one ticket on this list. Reading the ten is, among other things, reading the complaints before they are made. The advantage is that the history says what happened rather than what was remembered. (The history is the only witness on the desk that does not improve the story.)

One hour, once a month, is the whole cost, and it is the best-value hour on the desk. Over a year, Dev's review has produced a chase rule for suppliers and a deputy on every approval. It has also produced a loan pool of docking stations and a change to the leave checklist. All of these came from letters written against ticket numbers. A desk that skips the review will improve the wrong things, confidently.

How to Pull the Ten

Take the tickets with a resolved stamp in the month, on the clock the definition card names. Usually that means business hours from created to resolved with no pauses, as What Time to Resolution Actually Measures sets out. Sort by resolution time, longest first. Take the top ten. That is the list, and it is deliberately resolved tickets only. Tickets that are still open and old are a different list with a different purpose, the aging report. The sweep that works through them is in The Stale-Ticket Sweep. The tail review reads tickets whose story is finished, because a finished story has an ending that explains it.

Ten is a count, not a percentage, and that is on purpose. On a desk resolving five hundred tickets a month, ten is a fiftieth of them. On a desk resolving fifty, it is a fifth. In both cases it is an hour's reading, which is the quantity the review is actually sized by. A desk of one, resolving twenty tickets a month, reads five. Do not take the slowest tenth either. On a busy desk that is fifty tickets, and the review will be skipped by month three. Ten tickets, every month, read in full.

Reading a history means finding the longest gap. Every ticket history is a list of stamped events: created, replied, status changed, assigned, note added, escalated, resolved. Read the stamps and find the longest stretch between two of them in which nothing happened. The gap is the cause, almost every time. The events either side of it say what the ticket was waiting for and from whom. A twenty-nine-day ticket with a three-day gap and a twenty-six-day gap has one cause. It is whatever was happening during the twenty-six days. In ticket 7702's case that was nothing, because a reply had landed and the status had not changed. If you run QuerySolv, the event history on each ticket already lists every status change, reply and handoff with its stamp. So the hour is spent reading rather than reconstructing. In a tool that scatters the history across mail and notes, the hour is longer. The first improvement is to stop scattering it.

Cause One: Waiting on Someone Inside the Organisation

The first cause is a ticket waiting for a person inside the organisation who is not the desk. It may be the requester, who was asked for information. It may be a manager, whose approval was needed. Or it may be another team, whose part of the fix came first. It shows in the history as a long gap in a waiting status. In its most expensive form, the gap continued after the wait was over. The reply arrived and nobody noticed. Or the approval was given by e-mail to the technician and never reached the ticket. Ticket 7702 was this. So was 7719, in its approval form. So were five of Dev's ten that morning, which is why seven letters matched.

The fix depends on who was being waited for, and there are three. For the requester, ask everything on the first contact rather than in instalments. Chase on a fixed cadence with a close-if-silent rule. That way, a ticket waiting on a requester is never waiting without a date. For an approver, every approval request names one person, a date by which it is needed, and a deputy for when that person is away. On desks without a workflow engine the convention is a note on the ticket and a calendar reminder. This is described in Approvals Without a Workflow Engine. For another team, the handoff carries a named owner and a date. The ticket stays on the desk's waiting view until the other team's part is done. In every case the test is the same. A ticket in a waiting status with no date and no name is not waiting. It is drifting.

The hidden half of this cause is the reply that did not change the status, and it deserves its own check. Many tools reopen a resolved ticket when the requester replies. But fewer change the status of a ticket that is waiting. So the answer arrives and the ticket stays filed under "waiting on requester" with nobody looking. Test yours with a reply from a personal address. If the status does not move, add "waiting tickets with a newer reply" to the daily sweep. Ticket 7702 lost twenty-six days to a reply that had been there all along. That is a long time to wait for something you already have.

Cause Two: Waiting on a Vendor

The second cause is a ticket waiting for a supplier. It may involve a part on order, a licence transfer, a support case with a software vendor, or a line installation from a carrier. It shows as a long gap in a vendor-waiting status, sometimes with the vendor's reference number on the ticket and often without. Its signature is one chase in the first week followed by silence. The technician assumed the supplier would say when the part shipped. The supplier assumed nobody minded. Ticket 7731's docking station was this. Of its thirty-eight working days, about thirty were the gap between the first chase and the second.

The desk cannot shorten a courier, and the honest first fix is to stop pretending otherwise and record the supplier's own promise. When the part is ordered, the ticket gets the order reference, the date the supplier promised, and a chase date two working days after that. The chase date goes in the waiting view. A supplier that misses its own date is chased on the day, not when someone remembers. The escalation path to the supplier, and what to do when the chase is ignored, is covered in the article on escalating to vendors and other teams. The second fix is to tell the requester, in the words When the Fix Is Weeks Away suggests. A requester who knows the part is due on the fourteenth does not phone on the eighth.

The third fix is the one that actually shortens the wait for the requester. It is a loan pool: a small stock of the things that most often go on back order. These are issued on the day so that the requester is working while the supplier is finding the box. Bellweather bought a loan pool of four docking stations after the third month in which a docking station appeared in the tail. It cut the requester's wait on those tickets from weeks to hours. It left the supplier's wait exactly where it was, which no longer mattered to anyone but the supplier. The gap that a loan pool cannot help with is the one after the part arrives. The box sits in the post room for three days because nobody linked the delivery to the ticket. Put the ticket number on the order so that the post room can.

Cause Three: Forgotten

The third cause is a ticket that was waiting for nothing. It shows as a long gap with no waiting status, or with a waiting status and nothing actually awaited. A sudden burst of events and a resolution follow, usually on the day the requester phoned. The owner was on leave, or had left. The ticket was in the wrong department, where everyone agreed it was somebody else's. The status said "on hold" and the note said nothing. The ticket was the hard one at the bottom of a personal queue that kept being left for a quieter afternoon. The ways a ticket arrives in this state are catalogued in Keeping Tickets From Hiding, and the tail review is where the catalogue gets its monthly count.

Forgotten is the only cause that is entirely the desk's and entirely free to fix. That makes it the most embarrassing letter on the sheet and the one to write honestly. The fix is routine rather than clever. Look at the unassigned view and the waiting view daily. Make a rule that tickets are released before leave and reassigned on a leaver's last day. Read everything older than the follow-up interval weekly. None of this costs anything. All of it has to be done by a named person on a named day or it is not done. Two of Dev's ten that morning were forgotten. Both would have been found in a day by a sweep the desk had been meaning to start.

Distinguish forgotten from waiting by asking what the ticket was waiting for and whether anyone could have said so at the time. "Waiting on the requester" with a question in the history and no answer is waiting. "Waiting on the requester" with no question anywhere is forgotten with a label on it. The label is the more worrying part. It means the status was being used to make the ticket go quiet. When the tally shows forgotten tickets wearing waiting statuses, the fix has two parts. Make a rule that every waiting status carries a note saying what is awaited and from whom. Take a monthly sample of ten to check that the notes are true.

Cause Four: Genuinely Hard

The fourth cause is a ticket on which the time was spent working. It shows as a history with no long gaps: attempts, notes, an escalation, a vendor case for a real fault, a workaround, a second attempt, a fix. Ticket 7688's scanner was this: four firmware attempts across three weeks, a case with the scanner maker, and a configuration change at the end that fixed it for good. There is nothing to fix on a hard ticket. The right response is to accept that some tickets take time. Write down what was learned so that the next one takes less. The learning goes into a knowledge base article. Or, if the fault will recur, it goes into a problem record with a known workaround, as Known Errors and Workarounds describes.

Hard is the cause people claim first and the history supports least, so grant it last. A technician asked why a ticket took three weeks will say it was a hard one. Sometimes it was, and the way to know is the gaps. A hard ticket has events every day or two. A ticket that was hard for three days and forgotten for eighteen is a forgotten ticket with a hard beginning. Read the stamps before accepting the letter. The tail review exists to separate slow-for-a-reason from slow-for-no-reason, and the separation only works if "hard" has to be earned.

The Tally: A Quarter of Tails

The letters are worth little for one month and a great deal for three. Three months of tails is thirty tickets, and thirty tickets is a pattern. Keep a running tally by cause, with the tickets' hours added up beside the count, and read it each quarter. Bellweather's first quarter under the review is below. The two waiting causes hold most of the tickets and most of the hours. This is what this one fictional desk found, not a rule about desks. Whether yours matches is what your own tally is for. The point of the tally is the "first fix" column, where the quarter's one improvement comes from.

Read the hours column against the count column, because they rank the causes differently. Waiting inside had the most tickets; waiting on vendors had the most hours, because a part on back order waits longer than an approver on leave. Which to fix first depends on what the desk is trying to shorten. That may be the number of requesters who have a bad month, or the length of the worst wait. Bellweather chose approvals first because the fix was free and took a week. It chose the loan pool second because it cost money and took a quarter to argue for. Both are the subject of the last article in this series, Improving Resolution Time Without Rushing Anyone. The tally is where that article's list of candidates comes from.

The Outlier That Is a Process Problem

When the same sub-cause appears in the tail three months running, it is not an outlier; it is the process, seen from the slow end. An approval that waits for a monthly meeting will put a ticket in the tail every month for as long as the meeting is monthly. A new starter form that HR sends the day before the start date will put a starter in the tail every time there is a starter. A supplier with no agreed delivery time will supply a tail ticket with every order. The signature in the tally is a sub-cause with the same shape each time: same step, same team, same length of gap, different ticket number. The ticket numbers are the evidence, and three months of them is a case.

The fix for a process outlier is a change to the process. The process is usually not the desk's to change. So the desk's job is to show the pattern in a form the owner of the process can act on. Show the tickets, the gap in each, the hours added up, and one sentence on what would remove the gap. That is a page, not a meeting, and the form of such a page is the subject of Evidence Over Anecdote. The war story below is a process outlier, found by the tally, fixed by a page. The alternative is to keep resolving the tickets individually and keep writing the same letter. That is how a desk spends a year being slow at something that was never its fault and never going to change.

A related case is the outlier that is a category. The tally shows one category supplying most of the hard or vendor letters month after month. Hardware repairs that always wait for parts are not ten problems. They are one process with ten examples. The honest response is to treat that category as a different kind of work. Give it its own expectations, its own stated timescale to requesters, and its own cut in the monthly table. Do not let it sit in the desk-wide tail being read as failure. The tail review finds it. The category cut, which is the subject of Resolution Time by Category and Priority, owns it from then on.

What Not to Do With the Ten

Do not take the ten out of the month's figures. Once the slow tickets have been read and explained, the temptation is to report the median and percentile "excluding known outliers". The answer is that the median already excludes them, in the sense of not being moved by them. The 90th percentile is supposed to include them, because the tail is what it measures. Explained is not the same as excused. A desk that reports its tail only after removing the tail has reported nothing. Leave the figures alone and put the explanations beside them.

Do not turn the review into a tribunal. The technician whose name is on a tail ticket is the witness, not the accused. They know what the ticket was waiting for. They will say so if the question is "where did the time go?" They will not if the question is "why did this take so long?" The causes are nearly always structural. Attach them to people and, within two months, tail tickets will be resolved prematurely to keep them off the list. That is a worse problem than the one it replaced. The per-person rule in this series applies here as everywhere. A technician's figures are for coaching, in private, on a sample of at least thirty. The tail review is not where that happens.

Do not fix the tickets. They are resolved; the requester has been helped or has given up; there is nothing to do for them. Fix one cause, the one with the most letters or the most hours. Leave the other causes for the following months. A desk that tries to fix all four at once will do none of them well. It will not know, when the tail shortens, which change did it. One cause, one change, one month, then read the ten again. The review is a loop, not an audit, and it is the loop that improves the desk.

A Short War Story: The Monthly Meeting at Harbourview Clinic

At Harbourview Clinic, the desk lead, Rosa, began the tail review in the spring. In each of the first three months, she found the same thing. Three or four of the ten slowest tickets were requests for software a clinician needed. Each waited between four and seven weeks in an approval status. In every case, the gap ran from the day the request was raised to the day of the clinic's monthly finance meeting. The meeting approved any software purchase above a small amount and met on the last Thursday. A consultant who had asked for a dictation licence in the first week of one month waited six weeks for it. She typed her clinic letters by hand in the meantime. The practice manager later estimated that at around forty hours of a consultant's time, against a licence that cost a fraction of that. Eleven other clinicians had similar waits across the quarter. Rosa wrote one page: the eleven ticket numbers, the gap in each, the hours added up, and a proposal. The twelve most-requested applications would be approved once a year as a standing list. The finance meeting approved the list at its next sitting, which was the last time it saw a software request. The following quarter's tail had no approval tickets in it at all. A cause that appears in the tail three months running is not a slow ticket. It is a process. A process is fixed by a page with ticket numbers on it, not by resolving the tickets faster.

The Outlier Review Sheet

The sheet below is the monthly hour as a form, one copy per month. The top section is the list. The ten rows are the reading, one letter and one line each. The tally carries the letters into the quarter. The last section is the single change chosen and the date it will be measured. Keep the sheets together. Three in a row are the first pattern the desk will have seen in its own slow tickets. The sheet from a year ago, read against this month's, is the proof that the loop works. Nothing on the sheet names a technician; the owner column is for the fix, not the ticket.

OUTLIER REVIEW SHEET — TEN SLOWEST RESOLVED TICKETS, ONE MONTH
Desk: ______________________   Month: __________   Reviewer: ______________   Review date: __________
Clock: business hours, created to resolved, no pauses   Resolved tickets in the month: ____   Read: ten [ ]  five [ ]

Cause letters:  I = waiting on someone inside (requester / approver / other team)
                V = waiting on a vendor      F = forgotten (waiting for nothing)      H = genuinely hard

 #   Ticket   Hours    Longest gap (from -> to, and how long)       Cause   What was awaited, and from whom
 1   ______   ______   __________________________________________   ___     ______________________________________
 2   ______   ______   __________________________________________   ___     ______________________________________
 3   ______   ______   __________________________________________   ___     ______________________________________
 4   ______   ______   __________________________________________   ___     ______________________________________
 5   ______   ______   __________________________________________   ___     ______________________________________
 6   ______   ______   __________________________________________   ___     ______________________________________
 7   ______   ______   __________________________________________   ___     ______________________________________
 8   ______   ______   __________________________________________   ___     ______________________________________
 9   ______   ______   __________________________________________   ___     ______________________________________
10   ______   ______   __________________________________________   ___     ______________________________________

CHECKS MADE WHILE READING
   [ ] every "waiting" letter has a question or request in the history; otherwise it is F
   [ ] every H has events every few days with no long gap; otherwise it is F with a hard start
   [ ] replies that arrived during a waiting status without changing it: ____ tickets
   [ ] tickets whose category lives in the tail every month: ____  (handed to the category cut)

THIS MONTH'S TALLY                 RUNNING QUARTER
   I ____ tickets  ______ h          I ____ tickets  ______ h
   V ____ tickets  ______ h          V ____ tickets  ______ h
   F ____ tickets  ______ h          F ____ tickets  ______ h
   H ____ tickets  ______ h          H ____ tickets  ______ h

SUB-CAUSE SEEN THREE MONTHS RUNNING (a process, not a ticket)
   ___________________________________________________   Process owner: ______________   Page sent on: __________

THE ONE CHANGE THIS MONTH
   Cause chosen: ___   Change: ________________________________________________   Owner: ______________
   Starts: __________   Measured on: __________   Expected to move: median [ ]  90th percentile [ ]  tail count [ ]

NOT DONE WITH THIS SHEET
   [ ] no ticket removed from the month's figures     [ ] no technician named in the tally or the change

Dev's first sheet read seven I, two F and one H. The one change was a deputy on every approval, written into the request form by lunchtime. The following month's ten had three I, and the month after that, one. The sheets live in a folder in order. A new technician who joined in the autumn was given them to read on her second day. That is a better introduction to a desk than any process document. Every line is a ticket number, and every ticket number has a story.

Frequently Asked Questions