Automating the Desk Without Losing Your Judgment

Wednesday, 09:30, at Ashgrove Housing. Femi has set aside the morning for something the desk has never done, which is to look at what its automation rules did last month. He filters for every ticket a rule touched and gets 1,912 actions across 640 tickets. The rule that raises a ticket's priority after two days without an update has made 212 of the 260 open tickets Urgent, including one about a stationery order. The routing rule that sends housing officers' tickets to the housing systems team has sent nothing there since the request form was redesigned in the spring. That is because the form choice it reads no longer exists. Those tickets have been landing in facilities, who have been politely leaving them. The keyword hint rule has written "subject mentions password; check account status" on 340 tickets, including every one that actually was a password reset. None of the rules has done anything other than exactly what it was told. Nobody has told them anything for eight months.

Judgement, on a help desk, is the act of reading a situation and deciding what it needs. It is the one thing a rule cannot do and the one thing automation quietly removes if nobody guards it. Every rule replaces a moment when a person would have looked at a ticket. Most of those moments were routine, and replacing them is the point. A few were the moment somebody would have noticed that the rule was wrong. And those moments have to be put back on purpose, as checkpoints, audits and logs. Otherwise, the desk ends up with a set of rules doing confident, consistent, unexamined things to its tickets. This article is part of our Ticket Workflow Automation series. It is about putting the moments back.

It covers what judgement does on a desk and why rules erode it. Next come the four human checkpoints no rule may cross, then the monthly audit of what the automation did. We cover the misfire log and how to read it for patterns, then switching a rule off without shame. Next comes rule ownership and what happens when the owner leaves. We finish with keeping the team's judgement in practice, a short war story, and the automation review sheet to copy.

What Judgement Does, and How Rules Erode It

A person reading a ticket does three things a rule cannot. They notice what is missing, they notice what is odd, and they notice that the rule itself is misbehaving. A technician who opens a ticket about a stationery order marked Urgent frowns, and the frown is the whole value. A rule that marked it Urgent does not frown. It has no idea that stationery orders are not urgent, no idea that it has done this two hundred times. It has no way to learn either fact, because learning is not among its three parts. Trigger, condition, action. The frown is not in there.

Rules erode judgement in two ways, and the first is simply by removing the look. Before the priority rule, someone at Ashgrove glanced at old tickets every few days and decided which ones mattered. After it, nobody did, because the rule was handling it, and "handling it" meant making all of them the same. The second way is subtler: a field a rule has filled in looks decided. A technician who sees Urgent on a ticket assumes a person judged it so, and works accordingly. And the rule's guess is now carrying the authority of a decision nobody made. The erosion is not that technicians get worse at judging. It is that they stop being asked to.

The guard against both is to decide, in advance, which moments must always have a person in them. The other guard is to build the audit that checks the rules are not drifting around those moments. The first is the checkpoints. The second is the monthly review. Neither takes long. Both are skipped on most desks, because a rule that is running feels like a job that is done. It is not; it is a job that is being done by something that will never tell you when it starts doing it wrong.

The Four Human Checkpoints

A checkpoint is a moment in a ticket's life where a person must look before anything further happens, whatever the rules have done up to then. There are four, and the table below gives each one its reason and its guard. Rules may run before a checkpoint and after it. They may not stand in for it. If a rule is doing something that a checkpoint says a person must do, the rule is above the judgement line described in What to Automate First, and What Never to Automate, and it comes off.

Two of the four tend to get argued about. Auto-close is the usual argument. The answer is a reminder at an agreed interval, a second reminder, and then a close that says what happened and invites a reply. That is a routine a person designed, with the person's judgement built into the intervals and the wording. A close with no reminder is a rule deciding that the requester has finished. Remediation on live systems is the other, and the answer is the same as in the ladder. Use a button first, for months, and a trigger only when the log of button presses has earned it. The checkpoints are not anti-automation. They are where the automation hands back.

The Monthly Audit of What the Automation Did

Once a month, pull every action a rule took and look at a sample of it. Most tools record a rule's action in the ticket's history in some form. That could be a note, an event, or a field change with the rule's name on it. Most can filter or report on it. If yours cannot, the rule's owner keeps a count by other means, such as a tag the rule adds or a search for the note it leaves. Count actions per rule for the month, then open ten tickets per rule, chosen at random rather than from the top. And ask three questions of each. Did the rule do what the register says it does? Was what it did right for this ticket? Would a person, reading this ticket, have done something different?

The three questions catch three different failures. The first catches drift: a rule whose condition no longer matches what it was written to match, like Ashgrove's routing rule reading a form choice that had been renamed. The second catches misfires: the rule did what it says, and what it says was wrong for this ticket, like the priority bump on the stationery order. The third catches the quiet erosion: the rule did something defensible, but the person would have done something better. And the gap between the two is what the desk is paying for the rule. Ten tickets per rule, three questions each, is about an hour for a desk with a dozen rules. It is the hour that turns automation from a thing that was set up into a thing that is run.

Read the counts as well as the samples. A rule that fired two thousand times in a month on a desk that opened six hundred tickets is doing something to tickets more than three times each. Whatever it is, nobody intended it. A rule that fired zero times has either finished its job or stopped matching, and only the sample will say which. A rule whose count doubled since last month has met a change in the desk's traffic that nobody told it about. If you run QuerySolv, every event on a ticket sits in its history and the reports include reopen and escalation rates. That means finding out what happened to a ticket last month is a read rather than a reconstruction. A rising reopen rate is the earliest sign that something automated is replying or closing wrongly. In any tool, a count per rule per month, written on the review sheet, is the minimum, because the second month's count means nothing without the first.

The Misfire Log

A misfire is any time a rule did something a person had to undo. And the misfire log is one line per misfire, written by whoever undid it, at the time. Date, rule, ticket number, what the rule did, what it should have done, what it cost, and whether the rule was changed. It lives wherever the register lives, which for most desks is a shared document or a page in the knowledge base. It is read in full at the monthly audit. Writing the line takes thirty seconds. Not writing it means that the audit relies on memory, and memory is the thing that turned "the rule has been fine" into eight months of nobody looking.

The log is only useful if writing in it is safe. A technician who undoes a misrouted ticket and logs it is reporting a fault in a rule, not in a colleague. The desk lead's response to a log entry is "thank you" and never "who wrote that rule?" The person who wrote the rule may well be the one logging the misfire, and should be. The rule's author is the person most likely to notice that it has started doing something odd. A desk where logging a misfire feels like an accusation has a log with nothing in it and rules that have been "fine" for years. The monthly audit treats every entry as a free lesson, because that is what it is.

The log also feeds back into the register. Each misfire line names a rule, and the register's entry for that rule gets the count. A rule with no misfires in six months is trusted and reviewed quarterly. A rule with one a month is watched. A rule with three of the same kind in a month is wrong in a way the log has just described. And the fix is usually written in the log's own "what it should have done" column, by someone who was looking at the ticket at the time. The log does the diagnosis. The audit only has to read it.

Reading the Log for Patterns

Single misfires are noise. Patterns are the signal, and four patterns cover most of what a log will ever show. The first is the same rule misfiring the same way: three tickets in a month where the keyword rule matched a word with its other meaning. That rule is wrong, not unlucky, and it becomes a hint or comes off. The second is misfires clustering around one requester, one department, or one system. For example, the routing rule is fine for everyone except the partner whose domain changed, or the form that was redesigned. The rule reads a fact that has changed for one group, and the fix is to update the fact, not the rule.

The third pattern is misfires that cluster in time: a burst on one afternoon, usually the afternoon something else changed, like a mailbox being migrated or a form being edited. The rule did not change; the world did, and the log is the only place that records when. The fourth is the pattern of absence. This is a rule that never appears in the log and never appears in the audit sample doing anything useful either. It fires, it acts, nobody undoes it, and nobody can say what the desk would lose if it stopped. That rule is a candidate for the month-off test in the next section, and most desks have two or three of them.

The same reading applies to the automated things that feed the desk from outside, such as alerts that become tickets. And the monthly look at those is in Reviewing the Integration Monthly. The patterns are identical. A rule that files, routes, flags or opens tickets is a rule that will drift the moment the thing it reads changes. And the log is how you find out which month that was. (It was, in my experience, always the month before anyone looked.)

Switching a Rule Off Without Shame

The off switch is a feature. And a desk that cannot bring itself to use it is a desk that has made its rules into furniture. Every rule in the register exists because someone thought it would help, and most do, and some stop. And the right response to a rule that has stopped helping is to switch it off and see what happens, with the same lack of ceremony as changing a filter. The rule failed. The person who wrote it did not. They wrote a sensible rule for the desk as it was, and the desk moved, and the rule did not, because rules cannot.

The month-off test is the honest way to do it. Switch the rule off, note the date in the register, and watch for a month. Does anything land in the default that used to be routed? Does anything age that used to be flagged? Does anyone ask where the hint notes went? If nothing happens, the rule was decoration and stays off. If something happens, you have learned what the rule was actually doing, which is often different from what the register said. In that case, you write that down and switch it back on with a better description. Either result is a win. A rule that is on because nobody dared switch it off is a rule whose purpose is unknown, and unknown purposes are how automation accumulates.

Tell the team in one line and move on. "The priority bump is off from today; I've logged why; shout if anything looks odd." No post-mortem, no slide. The culture to aim for is the one where switching a rule off is as routine as switching one on. And the only way to get there is to do it in public a few times without anyone being blamed. The wider principle is that the desk hands work to automation and takes it back without drama when the automation stops earning it. The same principle governs the newer kinds of assistance in What Not to Hand Over. The tools differ. The off switch is the same shape.

Rule Ownership and Succession

Every rule has an owner by name. And the owner is the person who answers the three audit questions for it each month, reads its misfire lines, and decides whether it stays. A rule without an owner is a rule nobody is looking at, which is the condition every failure in this article starts from. The owner is not necessarily the author. Authors leave, and the commonest way a desk acquires a rule it cannot explain is that the person who could explain it is no longer in the building. Put rules on the leaver checklist beside tickets and passwords. When someone leaves, every rule they own is reassigned, with a one-line note on what it does, or switched off.

Ownership also means the owner is told when the rule misfires, which is what makes the log work. The technician who undoes a misroute writes the log line and mentions the owner. The owner reads it that day, not at the end of the month. If the same thing has happened twice this week the owner does not wait for the audit. A rule with an attentive owner misfires for a day. A rule with no owner misfires for eight months and is found by a desk lead with a filter and a free Wednesday morning, which is how Femi found his.

Keeping the Team's Judgement in Practice

Judgement is a skill, and skills that are not used fade. A desk where rules do all the routing and all the flagging is a desk where the technicians have stopped reading tickets at first touch. When the rules fail, as they will the month something changes, the people who have to catch the failure have been out of practice for a year. Keep the practice deliberately. Every technician triages by hand for a week each quarter, rules or no rules, and reads the default department's unassigned list as part of it. New technicians triage by hand for their first month before they are shown what the rules do. That way, they learn what the rules are for by having done the job the rules replaced.

Keep the hint pattern, too, where it fits, because a hint keeps the person in the decision. A rule that writes "subject mentions password; check account status" on a ticket and leaves the category blank is asking the technician to decide, with help. A rule that sets the category has decided for them, and after a hundred tickets the technician has stopped noticing the field at all. Ashgrove's hint rule had drifted into noise by writing the same note on tickets that did not need it. The fix was not to remove the hint but to narrow it, so that the note appeared only where it would change what the technician did. A hint that is always there is wallpaper. A hint that is sometimes there is advice.

Finally, watch the desk's own quiet-failure signs, because automation produces them in a new form. The seven ways a desk goes wrong without an outage, in The Seven Ways Support Quietly Fails, all have an automated cousin. The black hole becomes the misrouted queue nobody reads, the silent close becomes the auto-close, and metric theater becomes a priority field that says Urgent on everything. The dashboard stays green throughout, because the rules are all running, and running is what dashboards measure. A rule that is running is not a rule that is right. Only the audit knows the difference.

A Short War Story: The Urgent Stationery at Bluewater Bank

At Bluewater Bank, the desk lead, a man called Tomasz, added a rule that raised a ticket's priority by one level every two business days it went without an update. He reasoned that old tickets deserved attention. For a month it worked as intended, nudging a handful of forgotten tickets up the list. By the third month, with nobody auditing it, every ticket older than a week had climbed to Urgent. The priority field had become an age counter, and the technicians had learned to ignore it. That was because an Urgent ticket about a stationery order teaches you quickly that Urgent means nothing. On a Thursday morning a branch raised a genuinely urgent ticket: the card payment terminals had stopped working and the branch could take only cash. Triage set it to Urgent, where it joined 190 others. It was picked up the following afternoon. By then the branch had spent a day sending customers to the cash machine across the road. The branch manager had sent an e-mail to the regional director with the ticket number in the subject line. The review found that the rule had done exactly what it said for eleven weeks, and nobody had looked at what it did. It also found that a priority field with 190 Urgent tickets in it is a field that has stopped carrying information. Tomasz switched the rule off that afternoon, wrote the first line in a misfire log that had not previously existed, and put the monthly audit in his calendar. His note on the register says that the rule had not failed. The rule had succeeded at the wrong thing, in public, for three months, and nobody had asked it to stop.

The Automation Review Sheet

The sheet below is the monthly audit on one page, filled in by the desk lead with each rule's owner, from the counts, the ten-ticket samples and the misfire log. The first block is the list of checkpoints, confirmed each month because the quickest way to lose one is to have a rule quietly start crossing it. The second block is one row per rule and is where the hour goes. The third block is the decisions, and the last line of it is the one that matters. That line records the rules switched off this month, with the date, so that the month-off test has a start.

AUTOMATION REVIEW SHEET — MONTHLY, ONE PAGE
Desk: ______________________   Reviewed by: ______________   Month: __________
Tickets opened this month: ____   Rule actions this month, all rules: ____

1. CHECKPOINTS STILL HELD BY A PERSON (tick each, or name the rule crossing it)
   [ ] First touch: no rule sets priority; triage reads every new ticket   crossed by: ________
   [ ] Requester messages: rules send only the acknowledgment            crossed by: ________
   [ ] Irreversible actions: buttons a person presses, never triggers    crossed by: ________
   [ ] Close: no close without a reminder; category confirmed at close   crossed by: ________

2. ONE ROW PER RULE (ten random tickets each)
   Rule: ______________   Owner: __________   Rung: [ ]ack [ ]route [ ]timer [ ]remediate
     Actions this month: ____   Last month: ____   Change explained? [ ] yes: ________ [ ] no
     Sample of 10:  did what the register says ____/10   right for the ticket ____/10
                    a person would have done differently ____/10
     Misfire log lines this month: ____   Same kind as each other? [ ] yes [ ] no
     Fact the rule reads (mailbox, form, domain, keyword) still true? [ ] yes [ ] no
     Decision: [ ] keep  [ ] narrow: ________  [ ] make it a hint  [ ] month-off test  [ ] retire

   Rule: ______________   Owner: __________   Rung: [ ]ack [ ]route [ ]timer [ ]remediate
     Actions this month: ____   Last month: ____   Change explained? [ ] yes: ________ [ ] no
     Sample of 10:  did what the register says ____/10   right for the ticket ____/10
                    a person would have done differently ____/10
     Misfire log lines this month: ____   Same kind as each other? [ ] yes [ ] no
     Fact the rule reads (mailbox, form, domain, keyword) still true? [ ] yes [ ] no
     Decision: [ ] keep  [ ] narrow: ________  [ ] make it a hint  [ ] month-off test  [ ] retire

   (repeat per rule)

3. MISFIRE LOG SUMMARY
   Lines this month: ____   By rule: ______________________________
   Patterns: [ ] same rule, same way  [ ] one requester or group  [ ] one afternoon  [ ] none
   Cost of the worst one (hours, delay, or a named consequence): ______________________

4. OWNERSHIP
   Rules without an owner: ____   Reassigned to: ______________   Or switched off: [ ]
   Leavers this month: ______________   Their rules reassigned on their last day: [ ] yes [ ] no

5. DECISIONS THIS MONTH
   Narrowed or changed: ______________________________
   Made into hints: ______________________________
   Month-off test started (rule, date): ______________________________
   Month-off test ended (rule, result): ______________________________
   Retired: ______________________________
   Judgement practice: technicians who triaged by hand this quarter: ______________

Next review: __________   Register updated: [ ]   Team told in one line: [ ]

Femi's first sheet at Ashgrove took most of the Wednesday, because it was eight months of looking done at once. The priority bump went off that day. The routing rule was pointed at the form choice that now exists. The hint rule was narrowed to fire only when the subject mentions a password and the body does not say "reset". The second month's sheet took fifty minutes. The third took forty, found one misfire, and recorded the stationery order's return to Normal priority, where it had always belonged.

Frequently Asked Questions