Device and Model Hotspots: Lemon Models, Bad Batches and Bad Rooms

Monday, 08:40, at Stonebridge Insurance. Lukas has three tickets before nine. They all say the same thing in different words: "second monitor not detected", "screen only works if I unplug and replug the dock", "dock dead, laptop not charging". Jo, who runs the desk of five, looks over his shoulder. She notices what he has not had time to: all three requesters sit on the fourth floor. The fourth floor was refitted in the spring, and the refit came with sixty new docking stations, all one model, all from one delivery. The desk's quarterly hunt had already put that dock model fourth on the device list, with the note "check". Jo has the note. What she does not yet know is which of three different problems she is looking at, and the three have three different fixes.

A device hotspot is a class of equipment that produces tickets out of proportion to how many of it there are. It comes in three shapes that look alike from the queue. They look nothing alike once you know the question to ask. A lemon model is a model that is bad everywhere it is installed. A bad batch is a model that is fine in general and bad in one delivery, one serial range or one firmware revision. A bad room is good kit in a place that kills it. The device cut from Finding Hotspots in the Ticket Data gives you the line. This article is about reading the line correctly. Replacing sixty docks when the problem is one floor's power is the kind of mistake that is remembered at budget time. This article is part of our Hotspot Analysis series. The per-device lemon list is a different tool. There, one machine with a dozen tickets gets replaced on a rule. The list lives in Tickets as Asset Intelligence. This article is about classes of device, not individuals.

It covers the three shapes and why the distinction matters, then telling them apart in the data with three pivots and one swap. Next come the lemon model, the bad batch, and the room with the failing kit. Then we cover separating age from model, the case in hours, and keeping the model column honest. We finish with a short war story and the device hotspot table to copy, in that order.

Three Shapes, Three Fixes

The shape decides the fix, and that is the only reason to care about the shape. A lemon model is fixed by not buying any more of it and replacing what you have, fastest where the rate is worst. A bad batch is fixed by a conversation with the supplier about one delivery, usually under warranty. The fix replaces or re-flashes that batch alone while the rest of the fleet carries on. A bad room is fixed by facilities, or the electrician, or the person who owns the air conditioning. Buying new equipment for the room fixes nothing, because the new equipment goes into the same room. Three hotspots that produce identical tickets, three different people to talk to, three different sums of money. Get the shape wrong and the desk spends the budget and keeps the tickets.

The queue cannot tell the shapes apart, because the ticket records the symptom and the symptom is the same. "Dock not charging" is what a lemon dock says and what a bad-batch dock says. It is also what a perfectly good dock says when plugged into a floor whose sockets were wired by the lowest bidder. The information that separates them is on the asset side: which model, which delivery, which room. If the desk has those three facts for each ticket, the shape falls out of the data in twenty minutes. If it has none of them, the shape is a guess. The guess is usually "lemon", because that is the one that requires nobody outside IT to be wrong.

Telling Them Apart: Three Pivots and a Swap

Three pivot tables separate the shapes, in order. The first is tickets by model, divided by the number of each model. That is the rate that found the hotspot in the first place. At Stonebridge, dock model D had fifty-three tickets in the quarter from a hundred and eighty units. That was about three per ten, against under one per ten for the older model. The second is tickets for that model by delivery or serial range. The sixty fourth-floor docks arrived in one spring order. The other hundred and twenty arrived in two earlier orders. And the spring order had forty-four of the fifty-three tickets. The third is tickets for that model by location: the fourth floor had forty-four, the other floors nine. The second and third pivots point the same way, and that is exactly the ambiguity. The spring batch is on the fourth floor. Is it the batch or the floor?

The swap test answers the question the pivots cannot, and it costs an afternoon. Take four docks from the suspect group and four from a good group, swap them between locations, label them, and wait a fortnight. If the fourth-floor tickets follow the docks downstairs, it is the batch. If the tickets stay on the fourth floor and the imported good docks start failing, it is the room. If both happen a little, it is both. That is rarer than it sounds and usually means the batch has a weakness the room exposes. Jo swapped four docks on the Monday afternoon. By the second Friday, three of the four docks that had gone downstairs had tickets. None of the four that had come up had any. It was the batch. The fourth floor was innocent, which pleased the facilities manager and surprised everyone else.

Write the swap down as a small experiment in the hotspot's definition card, with the serial numbers, the dates and the result. Its result is the evidence the supplier conversation will rest on. And in six months someone will ask why the desk was so sure. A swap is not a controlled trial, and a fortnight is a short window for a device that fails once a month. Where the rate is low, run it for longer or swap more units. The cost is an afternoon either way, which is less than the cost of being confidently wrong about sixty docks.

The Lemon Model

A lemon model is bad everywhere, and the tell is that splitting its tickets by anything else fails to find a pattern. Split by location and every floor has roughly the same rate. Split by delivery and every delivery is bad. Split by user and the tickets follow the machine rather than the person. The symptoms are several and start early. A laptop model that is a lemon produces docking, battery, wireless and hinge tickets from its first months. An honest model produces nothing much until its batteries age. Meridian Parts' model B, the fictional laptop that runs through this series, was a lemon by this test. There were nineteen machines across four departments and three floors. All produced tickets at six times the fleet rate, from the first quarter after purchase.

The first fix for a lemon is to stop buying it. And this is the cheapest fix in the whole of hotspot analysis because it costs nothing and prevents the hotspot from growing. The desk is usually not the buyer, so the fix is a sentence to whoever is: "model B produces tickets at six times the rate of the rest of the fleet; please take it off the standard list until we have replaced the ones we have." That sentence, with the rate behind it, is rarely argued with. The replacement of the existing units is a bigger conversation, built in hours in the way the next article but one describes. And the replacement can be staged: worst users first. Here, "worst" means most tickets per machine, which the per-device count gives you for free.

Beware the lemon that is really a mismatch. A model that is fine in the office and dreadful in the field is not a lemon. It is the wrong tool for the field. And the fix is a rugged model for the field staff and the ordinary model for everyone else. A model that is fine for most users and dreadful for the designers is not a lemon either. It is underpowered for one group, and the rate split by department will say so. The lemon test is that nothing explains the rate except the model. When something else does, name that instead, because it is cheaper to fix a mismatch than to replace a fleet. (Suppliers prefer the second interpretation. They are not neutral on the question.)

The Bad Batch

A bad batch is one delivery of a good model. The tell is a serial range or a delivery date that carries the tickets while the rest of the model does not. Manufacturing is done in runs. A run can have a component from a different sub-supplier, or a firmware revision that was fixed the following month. Or it can have a shipping incident that nobody mentioned. The fix is narrower than a lemon's and often cheaper, because a batch is a warranty conversation rather than a budget conversation. The desk's job is to give whoever holds the supplier relationship a tight case. Include the serial range, the count of units in it, and the count of tickets. Include the rate against the same model from other deliveries, the swap result, and three example tickets with their resolution notes.

Keep the batch together in the record from the moment you suspect it. Tag the serial numbers on the card, and note the delivery and the purchase order reference. Ask the technicians to write the serial on every ticket from the model for the next quarter. That way, the count keeps rising with the evidence. A supplier who is told "we think the spring delivery is faulty" will ask for serials and dates. A desk that has them in a list closes the conversation in one e-mail. And a desk that has to go and find them loses a month. The warranty position of each unit matters here too. And a desk that keeps it visible on the ticket has the batch conversation half-prepared before it starts.

Where the batch's fault is firmware, the fix may be a file rather than a return. That is worth asking the supplier about before anything is boxed up. Stonebridge's sixty docks had a firmware revision one step behind the other deliveries. The supplier sent the update. The desk applied it to the four swapped docks first and watched them for a fortnight. Then it did the other fifty-six on a Saturday. The tickets stopped. Fifty-three tickets a quarter, roughly thirty hours of desk time and a floor's worth of irritation, fixed with a file and a sandwich. Not every batch is that kind. Enough of them are that the question is always worth asking first.

The Room With the Failing Kit

A bad room is good equipment in a place that breaks it. The tell is that the location cut shows one place hot across several models at once. Heat is the usual culprit. Examples are a server cupboard that became a store room, or a training room where the sun lands on twelve machines every afternoon. Another is an office over the kitchen. Dust and damp come next, from workshops, packing halls, loading bays and anywhere with a roller door. Power is third, and the most insidious. A floor with a tired distribution board or a daisy-chain of extension leads produces tickets that look exactly like hardware faults. And the hardware is innocent. Connectivity is fourth: the one switch with the failing port, the cabling run next to the lift motor, the access point behind the fire door.

The room's fix belongs to someone outside the desk, and the desk's contribution is the evidence and the swap. Facilities managers hear "the fourth floor has a problem" from everyone and act on none of it; they act on "fourteen devices of three models on the fourth floor have produced sixty tickets this quarter against nine for the same models elsewhere, and when we swapped four of them with the third floor the tickets stayed on the fourth". That sentence names a rate, a comparison and an experiment, and it is very hard to answer with "the floor is fine". It also makes the desk the organisation's only source of evidence about buildings. That is a strange thing for a help desk to be and an underrated one.

Some bad rooms cannot be fixed, and then the fix is to choose equipment for the room rather than the room for the equipment. A packing hall that is hosed down every evening will always be damp. And the answer is sealed, rated devices, not a conversation with the cleaners. The failure signals that let a desk catch failing hardware before the ticket, disks, batteries and temperatures, are the subject of Failing Hardware Signals. And they are worth watching in a bad room most of all, because a room that cooks machines will cook the replacements too.

Separating Age From Model

Old machines fail more than new ones, and a device cut that ignores age will call the oldest model a lemon every time. A five-year-old laptop model produces battery, disk and hinge tickets because it is five years old, not because it was badly made. The comparison that reveals a real lemon is against peers of the same age, or against the same model's own rate when it was younger. Add the purchase year to the pivot. If the rate climbs with age across every model at roughly the same slope, the desk has a refresh-cycle question and not a hotspot. If one model is high at every age, it is a lemon. If one model's rate jumps after a particular year's purchase, it is a batch.

The refresh-cycle question is a real one and deserves its own answer, in hours. A fleet whose oldest third produces half the hardware tickets is telling the organisation what its replacement interval should be. And the hours spent on those tickets are the argument for shortening it. That argument is not a hotspot fix; it is capacity planning. And it goes to the budget holder with the rate by age as its evidence. The hotspot hunt's contribution is to stop a desk from "fixing" age by replacing one model with another of the same vintage. Otherwise the desk will do that, confidently, twice.

The Case in Hours

The device hotspot's case is tickets times hands-on hours, plus requester time, over a year, set against what the fix costs. The fifth article in this series, Killing the Source, Not the Symptoms, builds the full version. For a device, the short version is often enough. Stonebridge's dock batch produced fifty-three tickets a quarter, about half an hour hands-on each. That meant twenty-five to thirty desk hours a quarter and a hundred or so a year. Add sixty people losing a few minutes to a dead screen on a bad morning. The firmware fix cost a Saturday. Meridian's model B produced sixty-six tickets a quarter at forty minutes. That meant roughly forty-five desk hours a quarter and close to two hundred a year, from nineteen machines. The budget holder can put a price beside nineteen laptops and two hundred hours. They will not need the desk's help to reach a conclusion.

State the hours as a range and the tickets as a count, and let the money be someone else's sentence. The desk knows hours because the tickets hold them. It does not know what an hour of a claims handler's time is worth. When it guesses, the guess becomes the argument instead of the hotspot. "Sixty-six tickets, a hundred and fifty to two hundred desk hours a year, nineteen machines" is a case. The same with a currency figure attached is a debate about the figure.

Keeping the Model Column Honest

Every device hotspot in this article was found through a model column. And the column is only as good as the habit that fills it. Most small-team help desks, QuerySolv among them, keep no asset register of their own. The device reaches the ticket through a field on the form or a line in the note. And the desk decides what goes in it. Decide once: the model name as the supplier writes it, from a short list, and the serial number where the technician can see it. A free-text field collects "the new docks", "dock", "Dock D" and "docking station (grey)" within a month. The pivot treats them as four models. A drop-down with six entries collects six models. The difference is twenty minutes of setup and the whole of next quarter's hunt.

Record the delivery, too, somewhere the desk can reach it, because the bad batch is invisible without it. A spreadsheet of purchase orders with model, count, date and serial range is enough. It is the kind of record a finance team already holds and will share if asked kindly. Machines move, get rebuilt and get renamed. The ways of keeping device data truthful over time are the subject of Keeping Asset Data Honest. The single rule that matters most for hotspots is the first one there. The tag goes on the ticket at the time, not from memory on Friday.

A Short War Story: The Packing Hall Office at Tollgate Foods

Tollgate Foods was a food distributor with a packing hall and two depots. It had a small office at the end of the hall where the shift supervisors worked. Over three years its desk of two replaced the office's four computers seven times, each time with a different model. Each model in turn was judged unreliable. Disks failed, fans seized, and one machine stopped working altogether in a way nobody could explain. The desk lead, Nadia, had the model column in her tickets and had used it. What she did not have was the location split. The office was filed under the depot's site code along with forty other machines that were fine. A new technician, Rob, ran the location cut by room rather than by site. Rob found that the four machines in that office had produced more hardware tickets in a year than the forty in the depot. Rob walked down there with a thermometer. The office shared a wall with the blast chiller's compressor room. It ran at a temperature nobody would have tolerated in a server cupboard and was hosed down with the rest of the hall every evening. Seven rounds of replacement, perhaps thirty desk hours a year and the supervisors' trust in their computers, had been spent on a room. A sealed, rated machine on a bracket and a conversation with the cleaning contractor fixed it for good. The lesson Nadia wrote on the card was the shortest in the desk's history: split by room before you blame the model.

The Device Hotspot Table

The table below is the record for one device cut, one row per model or batch that the hunt flagged. It is laid out so that the shape is decided on the page rather than in a meeting. The first six columns come from the pivot and the asset record. The split columns are where the three shapes separate; the swap column is the experiment; the last three are the decision. Fill in the peer rate every time, because a row without one is a count, not a hotspot. Keep the completed table with the quarter's hunt worksheet, and carry unresolved rows forward to the next quarter unchanged. That way, a batch that is "being discussed with the supplier" for three quarters is visible as exactly that.

DEVICE HOTSPOT TABLE — ONE PERIOD
Desk: ______________   Period: ____________   Prepared by: ______________

Row 1
  Model / batch: ________________________   Units: ______   Tickets: ______
  Rate (tickets per 10 units): ______    Peer rate (same kind, same age): ______
  Symptoms seen (top three): __________________ / __________________ / __________________
  Split by delivery or serial range:  one delivery hot [ ]   all deliveries alike [ ]
  Split by location:                  one place hot [ ]      all places alike [ ]
  Split by purchase year:             rate rises with age [ ]   high at every age [ ]
  Swap test run? [ ] no  [ ] yes, from ____________ to ____________
    Result: tickets followed the devices [ ]   tickets stayed in the room [ ]   unclear [ ]
  Shape:  [ ] lemon model  [ ] bad batch  [ ] bad room  [ ] old fleet  [ ] single lemon machine
  Hands-on hours per ticket: ______   Desk hours per year (range): ______ to ______
  Fix proposed: ______________________________________   Owner outside the desk: ______________
  Status: [ ] evidence gathering  [ ] case sent  [ ] fix agreed  [ ] fix done, watching  [ ] cooled

Row 2
  Model / batch: ________________________   Units: ______   Tickets: ______
  Rate (tickets per 10 units): ______    Peer rate (same kind, same age): ______
  Symptoms seen (top three): __________________ / __________________ / __________________
  Split by delivery or serial range:  one delivery hot [ ]   all deliveries alike [ ]
  Split by location:                  one place hot [ ]      all places alike [ ]
  Split by purchase year:             rate rises with age [ ]   high at every age [ ]
  Swap test run? [ ] no  [ ] yes, from ____________ to ____________
    Result: tickets followed the devices [ ]   tickets stayed in the room [ ]   unclear [ ]
  Shape:  [ ] lemon model  [ ] bad batch  [ ] bad room  [ ] old fleet  [ ] single lemon machine
  Hands-on hours per ticket: ______   Desk hours per year (range): ______ to ______
  Fix proposed: ______________________________________   Owner outside the desk: ______________
  Status: [ ] evidence gathering  [ ] case sent  [ ] fix agreed  [ ] fix done, watching  [ ] cooled

Carried forward from last period unchanged: ______________________________________________
Models taken off the standard purchase list this period: _________________________________

Jo's row for dock model D was filled in over two weeks. The pivots were done on the Monday, and the swap on the Monday afternoon. The result came on the second Friday, and the supplier's firmware file the week after. The shape box said bad batch, and the fix said "firmware, then watch". The status reached "cooled" the following quarter when the model's rate came down to the fleet's. Lukas had three dock tickets before nine on the Monday it started. He had none in the whole of the quarter after, which he described as suspicious.

Frequently Asked Questions