A Knowledge Base Health Check in Thirty Minutes

Monday, 09:05, at Bellweather Logistics. Theo took over the desk lead role on Friday, when his predecessor left for a job with a shorter commute. The operations director has just caught him in the corridor. "Is the knowledge base any good? We paid for that module." Theo does not know. He knows it has two hundred and sixty articles, because the number is on the front page. He knows that nobody mentioned it during his handover. That took forty minutes and was mostly about the supplier contract for the warehouse scanners. The stand-up is at 09:35. He has thirty minutes, a help desk tool, and a question that deserves a better answer than "I think so".

A knowledge base health check is a repeatable half-hour look at ten signs of sickness. Each has a number you can pull from the tool or the team, and a threshold that says whether the sign is present. Each also has a fix that points at one article in this series. It is not an audit of every article, and it is not a survey of what people think of the knowledge base. It is ten measurements and a score. This article is part of our Knowledge Base Fundamentals series. It is the one to read on the day you inherit a knowledge base, and again in the first week of every quarter after that.

It covers what the check is and is not, then the ten signs. These fall into the three groups that the earlier articles in this series will make familiar: nobody reads, nobody writes, nobody owns. There is also one sign that the desk has stopped trusting it. Next come the numbers to pull and where to find them when the tool does not simply hand them over. Then come the fix for each sign, in a table, and scoring the result and what to tell the director. Finally come the thirty minutes, minute by minute; a short war story; and the health check sheet to copy, in that order.

What the Check Is, and Is Not

The check measures whether the knowledge base is doing its three jobs. Those are answering the user before the ticket, answering the technician faster than memory, and remembering the fix after the person leaves. The check does so by looking at the symptoms of the three habits that kill it. Each sign is something you can count or look up in a couple of minutes. Examples are articles opened from tickets, searches that return nothing, articles with no owner, and articles about systems that no longer exist. A sign is present or it is not, by a threshold the check states. Ten signs, ten yes-or-no answers, and a score out of ten in which lower is healthier. The whole thing is designed to fit in the half hour between a corridor question and a stand-up.

It is not a quality review of the articles themselves. You will not read two hundred and sixty articles in thirty minutes. You do not need to, because an article nobody opens is not improved by being well written. And an article everyone opens gets corrected by the people who open it. The check looks at use, coverage, ownership and trust, which are properties of the collection rather than of any article in it. Nor is it a survey. Asking users whether the knowledge base is helpful produces polite answers from people who have not opened it. The numbers do not have manners. (The director's question was also a survey of one, and the honest answer was "nobody knows".)

Run it three times: the week you take over a knowledge base, the first week of every quarter after the sweep, and whenever somebody senior asks whether it is any good. The first run gives you a baseline and a fix list. The quarterly runs tell you whether the fixes worked, in the same ten numbers. That means "the knowledge base is better" can be said with a before and an after. A knowledge base does not complain when it is ill. It simply stops being opened, quietly, and the check is how you notice before the director does.

The Ten Signs

The first three signs say that nobody reads. Sign one: no article was opened from a ticket in the last month. The number is the count of ticket replies and internal notes in the last thirty days that contain a link to a knowledge base article. The threshold is zero. A knowledge base that was not opened for a single ticket in a month is not part of the desk's work. Sign two: search returns nothing useful for the top ticket type. Type the requester's words for the commonest ticket into the knowledge base search. If the first result does not answer it, the sign is present. Sign three: the only views come from the desk. If the tool reports views, look at who viewed. If every reader was a technician, job one is not being done, and the sign is present. Together these three say whether anyone is in the building.

The next three signs say that nobody writes. Sign four: nothing new in a quarter. The number is the date of the newest article. If it is more than three months old, the sign is present. That is because the queue has certainly produced a new repeating question in that time. Sign five: top ticket types with no article. Take the ten commonest ticket types from the volume report and check each for an article. Three or more without one is the sign. Sign six: the fixes live in heads and chat. Ask the desk, out loud, "what are the five things only one person here knows how to do?" Then check whether any of the five is written down. Fewer than two of five written is the sign. The five names you just collected are the next five articles. The "ask Dev" test, from the first article in this series, is the same sign measured by ear.

The next three signs say that nobody owns. Sign seven: nobody can name the owner. Pick five articles at random and ask who is responsible for each being right. If the answer for three or more is a shrug, a team name, or a person who has left, the sign is present. Sign eight: half the articles are past review or have no review date. The number is the count of articles with a review date in the future divided by the total. Below half is the sign, and "there are no review dates" is the sign with a bow on it. Sign nine: articles about systems that no longer exist. Search the knowledge base for the names of the last three systems the organisation switched off. Any hits that are not clearly marked archived are the sign. These three say whether anyone would notice if an article went wrong.

The tenth sign says that the desk has stopped trusting it, and it is the one most desks have without knowing. Sign ten: corrections typed under links, and reopens clustered on articles. Search the last month's replies for phrases like "ignore step", "the button has moved", "it's actually under" within a few lines of a knowledge base link. Also look at reopened tickets for a ticket type whose replies linked the same article. Either finding is the sign. A desk that sends an article with a correction underneath has decided the article is wrong and is routing around it rather than fixing it. That habit, left alone, becomes a knowledge base the desk uses only as a place to point requesters while telling them the truth by hand.

The Numbers to Pull, and Where

Most of the ten numbers come from the help desk tool rather than from the knowledge base, which is the point: a knowledge base is measured by the tickets. Three come from the ticket search: links to articles in replies and notes over the last thirty days, correction phrases near those links, and reopened tickets by type. One comes from the volume report: the top ten ticket types. If you run QuerySolv, the reports give volume and reopen rates and every reply and note sits in the ticket's event history. So those four are searches rather than estimates. In most tools they are at worst a saved filter and a sort. The knowledge base itself supplies the rest: the newest article's date, the view counts and their viewers where the tool keeps them, the search test, and the archive check.

Where the tool does not keep a number, the team does, and the substitute takes a minute. No view counts: ask the desk who linked an article last week. Also read ten recent tickets of the top type to see whether any reply carried a link. No search log: the search test with the requester's words does the same job by hand, and the ten commonest phrases take five minutes to type. No register of owners and review dates: that absence is itself signs seven and eight, present, and the first fix is to build one. Do not let a missing number stop the check. Write "not recorded" on the sheet, mark the sign present if the substitute says so, and move on. A check with three estimated numbers beats no check. The estimates get replaced next quarter when the register exists.

The one number worth pulling that is not on the sheet is the view trend, if the tool keeps views over time. A knowledge base whose views fell by half in six months has a story that the ten signs will explain but not announce. Reading view counts honestly, separating readers from technicians and both from the author checking their own link, is a subject of its own in the self-service ROI series. It matters most when someone asks for a number to put on a slide. For the health check, a rising or falling line is enough. The ten signs say what is wrong; the trend says whether it is getting worse.

The Fix for Each Sign

Each sign has one fix, and the fixes are the earlier articles in this series, which is why the health check comes last. The table below puts the ten together with the number, the threshold and the fix. That way, a sign found in minute twelve has its remedy in the same row. The fixes are not equal in size. Some are an afternoon: retitling, assigning owners, archiving the dead. One is a week: the first twenty. One is a habit that takes a fortnight to set: the check on reuse. None of them is a new tool, and none of them is a documentation day.

Fix in order, not all at once, and the order is: ownership, then titles, then the twenty. Fix signs seven to nine first, because an article with a name on it gets fixed by its owner while you are working on the others. Also, the orphan audit retires the dead articles that would otherwise waste the next two steps. The method is in Ownership: Every Article Has a Name on It. Fix signs two and three next, because retitling the thirty articles the desk uses makes the existing knowledge base findable in an afternoon, as described in A Knowledge Base Structure That Stays Navigable. Fix signs four to six last, because writing new articles into an unowned, unfindable collection is pouring water into a bucket you have not yet looked at the bottom of. The writing week in The First Twenty Articles works on a clean collection and is wasted on a sprawl.

Signs one and ten are fixed by habit rather than by a task. Habits are set by the desk lead saying the same sentence for a fortnight. "Search before you solve; link the article in the reply; fix what you find wrong." Said at every stand-up for two weeks, with the Monday question "was an article opened from a ticket this week?" asked out loud, it sticks, as long as the articles the desk needs exist and are findable. That is why it comes after the others. The habit of checking an article each time it is used, and marking it stale rather than correcting it by hand, is the subject of Knowledge Base Maintenance Without a Documentation Day. That habit is the fix that keeps all the others fixed.

Scoring the Result, and What to Tell the Director

Count the signs present and read the score in three bands. Zero to two is healthy: keep the quarterly sweep going, fix the one or two signs in the next sweep, and run the check again next quarter. Three to five is ailing: the knowledge base is doing some of its jobs. The fix list is an afternoon of ownership and titles followed by a fortnight of habit. The writing week is included only if signs four and five are among those present. Six or more is sick: retire first, then assign, then write the twenty, and do not expect the numbers to move for a quarter. Theo's first run at Bellweather scored seven. The knowledge base had two hundred and sixty articles and was doing none of its three jobs, which is a thing two hundred and sixty articles can do.

The score is honest about one thing that the article count hides. A knowledge base can be large and dead or small and alive, and the number on the front page cannot tell you which. Resist reporting the article count as a health measure, however much it looks like one. It measures how much writing happened, and writing is the cheapest part of a knowledge base. Being opened is the expensive part, and the score measures that. If you need one number for a slide, use sign one's count, articles opened from tickets this month, and show it quarter by quarter. It goes up when the knowledge base is working and nothing else makes it go up.

Tell the director in three sentences, and resist the fourth. "The knowledge base has two hundred and sixty articles and was opened for two tickets last month, so it is not working yet. The reasons are that nobody owns the articles, the titles are in our words rather than the staff's, and the ten commonest tickets have no article. I will have owners and titles fixed by the end of the month and the ten articles written the week after, and I will give you the same ten numbers next quarter." That is the shape of a brief for someone who asked a plain question in a corridor. The longer form, for the director who wants the reasoning, is in Explaining Technical Things to Management. The director did not ask about the module. They asked whether the money was wasted, and the honest answer is "not yet".

The Thirty Minutes, Minute by Minute

Minutes zero to five: pull the volume report for the last ninety days, write down the top ten ticket types in the requester's words, and note the date of the newest knowledge base article. That gives you sign four and the raw material for signs two and five. Minutes five to twelve: type the requester's words for each of the ten types into the knowledge base search and record whether the first result answers it. That is ten searches, forty seconds each. That is sign two and sign five together, because a type with no useful result either has no article or has one under the wrong title. The sheet has a column for which.

Minutes twelve to eighteen: search the last thirty days of ticket replies and internal notes for links to the knowledge base. Count them and glance at who sent them and whether any has a correction typed underneath. That is sign one and half of sign ten. If the tool reports views, open the last month's and note whether any viewer was not a technician, which is sign three. Minutes eighteen to twenty-four: open five articles at random, write down who owns each if anyone can say, and whether it has a review date in the future. Then search for the names of the last three systems the organisation switched off. That gives you signs seven, eight and nine, in six minutes. The five random articles will tell you more about the collection than any summary.

Minutes twenty-four to twenty-eight: look at reopened tickets in the last month for a type whose replies linked the same article. That finishes sign ten. Ask the nearest two technicians "what are the five things only one person here knows how to do?" That is sign six and takes two minutes because they will have the list ready. Minutes twenty-eight to thirty: count the signs present, write the score, and pick the first three fixes in the order given above. Put the date on the sheet and the next check in the calendar for the first week of next quarter. Then go to the stand-up with an answer. It will not be the answer the director hoped for, but it will be a number, and numbers survive corridors.

A Short War Story: The Search That Had Stopped at Castlereagh Transport

Nia took over the desk at Castlereagh Transport, a haulage firm with three depots, from a lead who assured her in the handover that the knowledge base was "in good shape, about two hundred articles, the team uses it all the time". She believed him, because she had a supplier contract to renew and the knowledge base was not on fire. Six months later a new technician, Caleb, asked her why the knowledge base search returned nothing for anything. She discovered that the search index had stopped updating during a server migration the previous autumn. So every search had returned an empty page for eight months. The desk had noticed within a week, had assumed it was being fixed, and had quietly moved to pasting fixes into a private chat channel. Three hundred and some answers now lived there, unsearchable by anyone outside the four people in it. Caleb had taken three weeks longer than he should have to become useful, because the answers he needed were in a chat scroll he had not been added to. Two depot managers had stopped using the portal entirely because search "never found anything". The index was rebuilt in an afternoon. A thirty-minute check in Nia's first week would have found sign two at minute six. She now runs it in the first week of every quarter, whether or not anything is on fire. Never assume a knowledge base is healthy because nobody has complained. The people who found it broken did not complain; they left.

The Health Check Sheet

The sheet below is the ten signs, the number for each, the threshold, a box for present or not, and the fix, with a score and the first three fixes at the bottom. It fits on one page and is filled in during the thirty minutes rather than afterwards. Keep every completed sheet, because the quarterly comparison is where the check earns its place. The same ten numbers, four times a year, is a record of whether the knowledge base is getting better that nobody can argue with, including the director.

KNOWLEDGE BASE HEALTH CHECK — THIRTY MINUTES
Desk: ______________________   Date: ________   Run by: ______________
Articles in the knowledge base: ____   (for reference only; not a health measure)

NOBODY READS
 1. Replies and notes with a knowledge base link, last thirty days: ____
    Present if zero ................................................ [ ]
 2. Top ticket type: __________________  First search result answers it?  [ ] yes [ ] no
    Present if no .................................................. [ ]
 3. Viewers in the last month who were not technicians: ____  (or "not recorded")
    Present if none ................................................ [ ]

NOBODY WRITES
 4. Date of the newest article: ________
    Present if older than three months ............................. [ ]
 5. Top ten ticket types with no article (or none findable): ____ of 10
    Types missing: ______________________________________________
    Present if three or more ....................................... [ ]
 6. Five things only one person knows: written down ____ of 5
    The five: ___________________________________________________
    Present if fewer than two ...................................... [ ]

NOBODY OWNS
 7. Five random articles; owner named for ____ of 5
    Present if three or more unknown ............................... [ ]
 8. Articles with a future review date: ____ of ____  (none at all = present)
    Present if below half .......................................... [ ]
 9. Search for the last three retired systems: unarchived hits ____
    Present if any ................................................. [ ]

NOBODY TRUSTS
 10. Corrections typed under links ("ignore step", "has moved"): ____
     Reopens clustered on one article's ticket type: [ ] yes [ ] no
     Present if either ............................................. [ ]

SCORE: ____ of 10 signs present
  0-2 healthy: fix in the next sweep      3-5 ailing: an afternoon plus a fortnight
  6-10 sick: retire, assign, write the twenty; expect a quarter before numbers move

FIRST THREE FIXES, IN ORDER (ownership, then titles, then the twenty)
  1. _________________________________________________  by ________  who ________
  2. _________________________________________________  by ________  who ________
  3. _________________________________________________  by ________  who ________

One number for the slide: articles opened from tickets this month ____
Last quarter's score: ____   Next check: first week of ________

Theo ran the check at Bellweather between 09:05 and 09:33. He walked into the stand-up with a seven and a list of ten ticket types of which six had no article. He also had the news that the knowledge base had been opened for two tickets in the last month, both by the person who had just left. He told the director the three sentences that afternoon. Owners and titles took an afternoon the following week. The orphan audit retired a hundred and twelve of the two hundred and sixty. The desk wrote the six missing articles in the week after. The next quarter's check scored three. The one number for the slide went from two to forty-one. The director has not mentioned the module since, which Theo has chosen to read as satisfaction.

Frequently Asked Questions