The Coverage Calendar and the Single Point of Failure

Wednesday, 10:15, at Bluewater Bank. The handsets in two branches have gone into a restart loop after an overnight update. The branch managers are phoning the desk on their mobiles. Grace, who runs the four-person desk, knows exactly who deals with branch telephony. There is Dev, who is on leave until Monday. His leave was approved in the spring. Then there is Mo, who covers for Dev and is on a two-day course. Grace approved that course herself, last Tuesday, in about four seconds. Both absences are on the shared calendar. The calendar shows two people away. It does not show that they were the only two people who knew the telephony system, because the calendar does not know what anybody knows. Grace looks for the runbook. There is a folder called Telephony with a licence certificate in it.

A coverage calendar is a calendar that shows, for each working day, not just who is in but which duties have someone in who can do them. It is built from two things: the absence list everyone already has, and a coverage matrix. The matrix is a grid of the desk's duties against the people who can perform each one. A single point of failure is anything the desk cannot do without, and on a small desk the single points of failure are mostly people. This article is part of our Handoffs & Coverage Transitions series. This article is about seeing the gap on the day the leave is approved rather than on the Wednesday morning.

It covers why someone being in is not the same as being covered. Next come the coverage matrix, who covers what and to what level, and the bus test. Then comes the cross-training minimum that turns a matrix of red into a matrix of amber. The article covers the calendar that combines absences with the matrix, with a picture, and approving leave against it. Next come the two-person and one-person desk, where every row is a single point of failure, and keeping the matrix current. It ends with a short war story and the coverage matrix template to copy, in that order.

Someone Being In Is Not the Same as Being Covered

A desk is covered on a given day when every duty it might be asked to perform has at least one person in who can perform it. That is a different question from whether the desk is staffed. Bluewater's desk was staffed on Wednesday: Grace and Sunita were both in, both busy, both competent, and neither of them could do anything about the handsets. Two people in and the desk uncovered for one duty, which happened to be the duty that mattered that morning.

Staffing is counted; coverage has to be looked up. The absence calendar answers "how many people are in", and it answers it well, and every small desk has one. What no absence calendar answers is "can the people who are in do the things that might come up". That requires knowing what each person can do. That knowledge lives in people's heads, in the vague form of "Dev does the phones". The vagueness is the problem. "Dev does the phones" is true on the days Dev is in and silent on the days he is not.

The gap is almost always visible in advance and almost never looked for. Dev's leave was approved months ahead. Mo's course was approved a week ahead. On the Tuesday Grace approved the course, both absences were on the calendar and the gap existed, in the sense that anyone who knew the matrix could have seen it. Nobody did, because the matrix was not written down. A thing that is not written down cannot be checked in four seconds, which is how long leave approvals take. (It is also how long they should take, once the check exists.)

Who Covers What: The Coverage Matrix

The coverage matrix is a grid: duties down the side, people across the top, and in each cell how well that person can do that duty. Three levels are enough. Owner means this is their job and they can handle anything it throws up. Can cover means they can do the routine work and the common failures without help. Runbook only means they have never done it alone but could follow written steps for the most likely cases. A blank cell means no. Most desks find, on the first pass, that the matrix is a diagonal of owners with a great deal of white space around it.

Build the duty list from the work rather than from job titles. Pull the last three months of tickets and sort them by what the desk did about them; the piles are the duties. Add the standing duties that never become tickets. They include backups and restores, the directory and accounts, the firewall, and the telephony. They also include the card terminals, the branch systems, the monthly reports, and the supplier relationships. Ten to fifteen rows is normal for a desk of up to ten people. Forty rows means the list has been built from the asset register, and it will never be kept current. Merge until a row is something one person could be asked to "look after".

Fill it in with the team in the room, and fill it in honestly, which is harder than it sounds. People overstate what they can cover, out of helpfulness, and understate what they own, out of modesty, and the matrix ends up looking better than the desk. The test for "can cover" is a question: "If this broke on a Wednesday and the owner was away, would you fix it or phone them?" If the answer involves phoning them, the cell is runbook only at best. The matrix is not a performance review. It is a map of where the desk will fall over, and a flattering map of that is worth nothing.

Here is Bluewater's first honest matrix, drawn on the Wednesday afternoon after the handsets came back. Read down any column and you see what one person's absence removes from the desk. Read across any row and you see how many people can hold that duty up. Two rows had one name each. One of those rows was branch telephony.

Every row in that table has two names or more except the last. That looks reassuring until you notice that for the telephony row the two names were Dev and Mo. Both were away on the same Wednesday. Two names is the minimum for a row to be safe, and a minimum is only a minimum if the two are never away together. That is what the calendar is for, and it is the next section but one.

The Bus Test

The bus test asks, for each person, what the desk could not do if that person were hit by a bus tomorrow. The phrasing is grim and traditional. You may prefer "won the lottery" or "went to a competitor", which have the same effect on the desk and better effects on morale. Run it down each column of the matrix: strike the person out and read what each row is left with. Any row that goes to zero names is a single point of failure, and the person in that column is carrying it.

There are two kinds of single point of failure, and the second is worse. The first is skill: only Dev knows how the telephony system is configured. That can be fixed with a runbook and an afternoon. The second is access: only Dev has the administrator credentials, or only Dev's account is on the supplier's contact list, or the licence portal sends its codes to Dev's phone. Skill can be written down. Access has to be granted, in advance, by someone with the authority to do it. It is the thing most often discovered missing at 10:15 on a Wednesday. Check both for every row. A runbook that ends "log in as Dev" is not a runbook.

Run the test on yourself first. The lead of a small desk is usually the biggest single point of failure in the room. They hold supplier contracts, approvals, the budget, and the knowledge of why things are the way they are. Leads are reliably the last column anyone strikes out. Grace's column at Bluewater removed the supplier renewals from the desk entirely. Nobody else knew when anything expired. The hero pattern, where one person quietly becomes the desk, is described in The Seven Ways Support Quietly Fails. The bus test is how you find out which column the hero is in before their next holiday finds out for you.

The Cross-Training Minimum

The minimum is two names at "can cover" or better on every row, and a runbook for every row regardless. The minimum is two, not one, because one is a single point of failure by definition. It is not three, because a four-person desk cannot keep three people current on fifteen duties. The attempt produces a matrix full of optimistic cells. Two names and a runbook means that when one is away the other is in. When both are away someone can follow the steps for the common cases and knows who to phone for the rest.

Cross-training is done in three stages, and skipping the middle one is how "can cover" cells turn out to be "runbook only" on the day. First the learner watches the owner do the work, twice, while the owner writes the runbook as they go. That is because a runbook written by the person who knows the job always leaves out the step they no longer notice. Second, the learner does the work with the owner watching, which is where the missing step is found. Third, the learner does it alone, with the owner reachable, and the cell becomes "can cover". Each stage is an hour or two. The whole row is a month, done alongside the normal work.

One row a month is the sustainable pace. The order is by risk: the rows with one name first, then the rows where the two names are often away together, then the rest. A desk with fifteen rows and four single-name rows has four months of urgent work and a year of the rest. That is fine, because the matrix at month four is already a different desk from the matrix at month zero. Writing it all down is the half of the job that survives the person leaving. The discipline of doing so is in Documentation as Your Second Person.

Skills decay. A "can cover" cell that has not been exercised in a year is a "runbook only" cell with a better label. The person in it has usually noticed and not said. Once a year, have each cover do the duty for a day while the owner does something else. It costs a day and it finds the cells that have quietly gone stale. It also finds the runbooks that still say "log in as Dev", because Dev changed roles in the spring and nobody updated the page. (The page was marked "reviewed". The reviewer was Dev.)

The Calendar That Shows Gaps Before They Happen

The coverage calendar is the absence calendar read through the matrix. For each working day and each duty, count the people who are in and have "can cover" or better for that duty. Two or more is covered. One is thin. Zero is a gap. The result is a grid of duties against days, coloured, and the gaps are the red cells. Bluewater's week with Dev on leave and Mo on a course looks like the picture below. Two duties drop to nobody on Wednesday and Thursday. One duty is a single person all week. The whole thing was knowable on the Tuesday before.

Build it with whatever you have, because the arithmetic is simple and the tooling does not matter. A spreadsheet works. One sheet holds the matrix, and one holds the absences by person and day. A third counts, for each duty and day, the people who are both in and able. Most spreadsheet tools do this with a count formula that checks two conditions, and the formula is written once and copied. If spreadsheets are not your desk's habit, a shared calendar with one entry per absence and a printed matrix on the wall will do. In that setup, the lead reads the week ahead every Friday with the matrix beside it. The method is less important than the Friday.

Look at it on a fixed day for the two weeks ahead, and look at it whenever an absence is added. The first look catches gaps that have crept in through sickness and short notice; the second is the one that stops gaps being created. Both take under five minutes on a desk of ten. A red cell two weeks out is a problem with options. You can move a course, bring the cross-training forward, arrange outside cover, or accept the risk for a day with a written plan. A red cell discovered at 10:15 is a problem with none.

Approving Leave Against the Calendar

The rule is that no duty drops to zero on a working day. Leave is approved after that rule has been checked, not before. The check is the four seconds: open the week, look for red, approve. When there is red, the answer is not "no". The answer is "yes, once we have fixed Wednesday". Wednesday is usually fixed by moving a course, swapping a day, bringing a cross-training session forward, or putting a supplier on standby. People's holidays are rarely the thing that moves. Courses, projects and "I'll just work from the other site that day" move easily, and they are what create most of the double absences.

First come, first served is a queueing rule, not a coverage policy, and desks that use it for leave end up with August covered by whoever asked last. The calendar makes a better rule possible: anyone can book any day on which the duties they hold still have cover. With that rule published, people check the calendar themselves before asking. That moves the four seconds from the lead to the person booking and makes the lead's job a confirmation. It also makes the conversation about a clash a conversation about a red cell, which is a thing on a screen. It is no longer about whose holiday matters more, which is a thing nobody wins.

Accepting a gap is allowed, and sometimes right. A duty that has produced two tickets in a year does not need two people in every day. A desk that treats every red cell as forbidden will refuse leave it should grant. The discipline is that the gap is accepted on purpose, in writing, with a plan: "Card terminals have nobody Wednesday and Thursday; the supplier's support line is the cover; branch managers have the number; Mo is reachable by phone for anything the supplier cannot do." That paragraph in the calendar entry is the difference between a risk and a surprise. The branch managers would much rather have the number than the surprise.

The Two-Person and the Solo Desk

On a two-person desk every row has at most two names, and the calendar is red whenever both are away. That is why two-person desks should never both be away and usually are, at some point, because sickness does not read the calendar. The matrix still earns its place: it says which of the two holds each row. That tells the one who is in what they are about to be asked to do alone, and which runbooks to have read. The cross-training minimum for two people is simply that each can cover everything the other owns at runbook level, and that the runbooks exist. That is a year of Friday afternoons. It is also the whole of the desk's resilience, so it is worth the Fridays.

On a one-person desk every row is a single point of failure. The matrix has one column, and the honest coverage calendar is red on every day the one person is not in. There is no fix for this inside the desk, and pretending otherwise is how solo administrators end up answering tickets from a hospital bed. The fix is outside. It includes a manager who can triage and re-promise, so that requesters hear something true. It includes a supplier or an outside firm on a contract that includes cover. And it includes a written "if I am out" page that says where everything is and who to phone. The matrix is the document that makes the case for those arrangements, because a page of red cells is an argument that a budget meeting can understand.

Be honest with management about what a small desk's calendar shows. A desk of two has coverage most days and none on the days both are away. A desk of one has a plan, not coverage. Say so, with the matrix, before the Wednesday. Management seldom refuses to fund cover for a gap it can see on a page. It reliably refuses to fund cover for a gap it was never shown, after the gap has become an incident report with a name in it.

Keeping It Current

The matrix goes stale in three ways and needs a review that catches each. People join and leave: a joiner is a blank column until trained, and a leaver takes a column away on their last day. That is the moment to re-run the bus test on every row they touched. Duties appear and disappear: a new branch system is a new row, a retired one is a row to delete. And cells drift: skills decay, access lapses, runbooks rot. A quarterly review of fifteen minutes, with the team, catches most of it, and the annual cover-for-a-day exercise catches the rest.

Add one habit that keeps the matrix honest between reviews. After any incident where the desk struggled because the right person was away, ask one question in the review: "Who else could have done this, and why didn't they?" The answer updates a cell, or writes a runbook, or grants access. Bluewater's answer on the Wednesday was "nobody, because the telephony runbook was a licence certificate", and the matrix got a new row of work that afternoon. Desks that ask the question after every such morning find their matrix has stopped producing those mornings within a year. Desks that do not ask it have the same Wednesday every spring, with different handsets.

A Short War Story: Exam Week at Larkfield Academy

Larkfield Academy, a secondary school with a three-person IT team, found its single point of failure on the first morning of external examinations. The wireless in the two exam halls refused to authenticate the candidates' loaned laptops. The only person who understood the exam-hall network profile, Omar, was at home with a chest infection. The other two technicians could see the controller and could not log in to it, because the administrator account was Omar's. The supplier's support line would not act on a call from anyone not on the contract's named-contact list, which was Omar. Two exams started forty minutes late, on paper, and the exams officer spent the afternoon filing an irregularity notice with the exam board. Omar had owned the exam-hall row for four years and nobody had ever filled in a second cell. That was because the exam-hall network only mattered for three weeks a year, which turned out to be exactly the three weeks it mattered most. Larkfield wrote the matrix the following week and put a second named contact on every supplier contract. It moved every administrator credential into a shared vault and scheduled the cover-for-a-day exercise for the week before each exam season. A single point of failure does not fail on an average day; it fails on the day the duty is needed.

The Coverage Matrix

The template below is the matrix and the bus test on one page, with the leave-approval rule and the review dates at the bottom. Fill in the duties from the tickets and the standing duties, the people across the top, and the levels honestly. Then strike out each column in turn and write what goes to zero. The rows with one name are the first month's cross-training, and the sheet is the plan for it.

COVERAGE MATRIX — ONE PAGE
Desk: ______________________   Drawn: __________   Next review: __________
Levels:  O = owner   C = can cover alone   R = runbook only   blank = no

DUTY                          | ________ | ________ | ________ | ________ | names at C or O
------------------------------+----------+----------+----------+----------+----------------
Triage and the queue          |          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
______________________________|          |          |          |          |
Supplier contracts, renewals  |          |          |          |          |
Backups and restores          |          |          |          |          |

BUS TEST (strike out one column at a time)
  Without ________ the desk cannot: ________________________________________
  Without ________ the desk cannot: ________________________________________
  Without ________ the desk cannot: ________________________________________
  Without ________ the desk cannot: ________________________________________

ACCESS CHECK (a runbook that needs one person's login is not a runbook)
  Duties where only one person holds the credentials or supplier contact: __________________
  Fixed by: shared vault [ ]   second named contact on contract [ ]   second admin account [ ]

RUNBOOKS
  Rows with a runbook: ____ of ____     Rows with none, by risk: ______________________

CROSS-TRAINING PLAN (one row a month, single-name rows first)
  Month 1: ______________ learner ________   Month 2: ______________ learner ________
  Month 3: ______________ learner ________   Month 4: ______________ learner ________

LEAVE RULE
  No duty drops to zero names on a working day. Approver reads the week before approving.
  Accepted gaps are written in the calendar entry with the plan and the outside contact.

REVIEW
  Quarterly with the team: ______   Annual cover-for-a-day: ______   After any "only X knew" incident: [ ]

The rows pre-printed on the sheet, triage, supplier renewals and backups, are the three most often left off. The first is left off because everyone assumes everyone can do it. The other two are left off because they belong to the lead, and the lead does not think of themselves as a row. Grace added her own column to Bluewater's matrix last, and found that supplier renewals had one name. The following month Sunita learned the renewals calendar. That month, Grace booked a holiday for the first time in two years without taking her laptop. The handsets were fine by 14:30, after the supplier walked Grace through a reset over the phone. The runbook now has eleven steps and two people who have done them.

Frequently Asked Questions