Writing Reproducible Steps a Stranger Can Follow

Tuesday, 10:05, at Acme Manufacturing. Ticket 6204, from Bea in the design office: "The drawing tool crashes sometimes." Owen on first line connected to her machine on Monday, opened the drawing tool, and watched it open. He escalated to Farah with the note "crashes intermittently, pls look." Farah opened it twice on Monday afternoon, watched it open twice, and sent it back: "Cannot reproduce; close unless it recurs." It recurred three times on Monday, each time taking twenty minutes of Bea's work with it. The ticket has now done a full lap of the desk in a day and a half. The only thing anyone has learned is that the drawing tool opens when a technician is watching.

Reproducible steps, usually shortened to the repro, are a numbered list of actions that makes a problem happen again. The actions start from a state anyone can reach. They are written so that a person who was not there can do them and see the same thing. They are the part of a ticket note that describes the problem rather than the fix, and they are the part most often missing. This article is part of our Writing Ticket Notes That Survive You series. And it is about turning "it crashes sometimes" into something a stranger can make crash on purpose.

It covers why "cannot reproduce" is usually a note problem, then the six parts of a repro. Next come environment facts, and why "since when" is the most valuable of them. We cover exact steps from a known starting point, exact error wording, and expected versus actual. Then comes what changes it, which is where the cause hides. We finish with three vague complaints turned into repros, a short war story, and the repro block to copy.

"Cannot Reproduce" Is Usually a Note Problem

The problem was real. Bea saw it, three times, and lost work each time. What failed was not the drawing tool but the note, which carried the word "crashes" and nothing else across two handoffs. Owen did not write what Bea was doing when it crashed, which file she had open, whether a message appeared, or when it had started. Farah, reading "crashes intermittently," did the only thing the note allowed, which was to open the tool and wait. "Cannot reproduce" is the desk's polite way of saying "I did something else and it worked."

A thin repro has three possible endings and all of them are expensive. The ticket bounces between first and second line, each pass adding a day and a note that says less than the last. Or it is closed as not reproducible, recurs, and comes back as a new ticket with the history lost. Or the senior technician gives up on the ticket and goes to sit beside the requester until it happens. That works and costs an afternoon of the most expensive person on the desk. The handoff that carries enough to act on is the subject of the handoff that does not dump. The repro is the most important thing it carries when the problem is a fault.

The repro also matters after the fix, because it is the test. If you can make the problem happen, you can watch it stop happening, and "fixed" means something. Without a repro, "fixed" means "did not happen while I was watching," which is the same thing Farah found before anyone had fixed anything. A requester who replies "it's doing it again" to a resolved ticket reopens it in QuerySolv and in most other tools. The first thing the next technician should do is run the repro on the original ticket. If there is no repro, they start from "crashes sometimes," which is where Owen started.

The Six Parts of a Repro

A repro that a stranger can follow has six parts, and most tickets that fail to reproduce are missing three of them. The parts are the environment, the starting state, the steps, the expected result, the actual result, and what changes it. Each answers a question the next person will otherwise have to ask. And the table gives the question beside each part, with the answer as it should have appeared on 6204.

Read down the right-hand column and the cause is nearly visible: something on one machine, affecting large files opened from the network, since a restart. Farah could have had that in a quarter of an hour on Monday afternoon instead of a verdict of "cannot reproduce." None of the six parts is hard to collect. They are only hard to remember to collect, which is what the repro block at the end of this article is for.

Environment Facts: Machine, Account, Location, Since When

The environment is the set of facts about where the problem lives, and four of them are needed every time. The machine, named as the system names it, so "DES-WS-04" rather than "Bea's computer," because Bea has used three computers this year and the next reader needs to find the right one. The account that was signed in, which is not always the person reporting, since people share machines, use test accounts and report on behalf of colleagues. The location, both physical and network: which office, which floor, wired or wireless, in the building or working from home through the remote connection. And since when, as precisely as the requester can say.

"Since when" is the question most often skipped and most often the whole answer. A problem that started on Friday started because something changed on or just before Friday. And the desk usually knows what changed, because the desk did it. It might have been a restart, an update, a new printer driver, a policy change, a move to a new switch port. Ask "when did this last work?" and write the answer down even when it is vague. That is because "sometime last week" still rules out the changes made this week. Where the desk keeps a record of what it changed and when, as described in tracking changes without a change module, "since Friday" turns into a short list of suspects before anyone opens a log.

Add facts beyond the four only when they differ between where the problem happens and where it does not. The version shown in the application's About box matters if two machines have different ones and only one fails. The browser matters if the problem is in a web application and another browser behaves differently. The amount of free disk space matters if the failing machine is nearly full. The rule is not "write everything about the machine"; it is "write what might be different about this machine." Everything else is noise the next reader has to read past, and the next reader is tired.

Exact Steps From a Known Starting Point

Steps begin from a starting state that anyone can reach, and the starting state is written down. "Signed in as the requester, no applications open" is a starting state. "With the tool open" is not, because there are six ways to open the tool and the crash may live in one of them. Owen's one visit to Bea's machine opened the drawing tool from the desktop shortcut and then opened a drawing from the recent-files list. Bea opens drawings by double-clicking them in the shared folder, which starts the tool a different way with a different file path. That was the path that crashed. The step Owen skipped as obvious was the step that mattered.

Write one action per step, and name what is clicked or typed by the words on the screen, in quotes. That way, the next person can match them. "Click 'Open'" rather than "open the file," because the first can be followed by someone who has never seen the application. Include the data: which file, by name and location; which record, by its number; which customer, by an identifier that is not personal. Include the waits: "wait until the progress bar finishes" is a step, because a person who does not wait sees a different result. If a step involves a sensitive value such as a password or a real customer's details, say "enter the test customer number". In that case, keep the real value out of the ticket, for the reasons in what never to write in a ticket.

The test for the steps is to hand them to someone who was not there and watch where they stop to ask. Every question they ask is a missing step, and the answer goes into the list where the question arose. Do this once with a colleague on a real ticket and you will never again write "open the file" without saying which file and from where. The steps are finished when a stranger can follow them in silence. I have written repros that passed that test on the fourth attempt. And the first three were all missing the same kind of thing, which was the step I thought nobody could get wrong.

Exact Error Wording

Copy the error exactly as it appears, in quotes, with its code, its punctuation and its spelling mistakes. Say where it appeared: a dialog box, the title bar, a line in a log, a banner at the top of a web page. "Something about permissions" is not an error message; it is a memory of one. The exact text is what the next technician searches the knowledge base and the supplier's documentation for. And it is what distinguishes the error that means a locked file from the error that means a missing one. Those errors look identical to a requester. To the system they are not.

Take a screenshot and type the text as well, because a screenshot cannot be searched and the text cannot show the context. If the error vanishes before it can be captured, photograph the screen with a phone and transcribe it afterwards. An ugly photo of the right message is worth more than a tidy description of the wrong one. Where there is no message at all, say so in as many words: "no dialog; the window closes; nothing in the application's own log; one entry in the system event log at 10:41 reading '...'." A silent failure described as silent is a finding. A silent failure described as "it crashes" is a shrug. Writing the text so that search will find it later is the subject of notes for the stranger who finds this ticket next year. The rule for the repro is only that the text is verbatim and its location is named.

Expected Versus Actual

The expected result is what should have happened; the actual result is what did. Write both, as two short lines, because without the first the second has no meaning. "The report takes four minutes" is a fact. Whether it is a problem depends on whether the report usually takes ten seconds or ten minutes, and only the expected line says. "Expected: the drawing opens in the editor and stays open. Actual: the drawing appears for about two seconds, then the window closes with no message." Two lines, and the next reader knows exactly what they are looking for and exactly what they will see when they find it.

Numbers belong in both lines wherever the problem is about speed, size or quantity. "Slow" becomes "about ninety seconds to open, measured twice with a watch; the same file opens in eight seconds on the machine next to it." "Wrong total" becomes "the report shows 412 orders; the order list for the same day shows 418." "Keeps disconnecting" becomes "the session drops roughly every twenty minutes; three times between 09:00 and 10:10." A number can be compared after the fix. An adjective cannot.

The expected line occasionally reveals that there is no fault at all. The requester expected the system to do something it was never designed to do, or expected a feature that was removed on purpose, or expected behaviour from a different application. That is not a wasted repro; it is a ticket that has just become a request or a training conversation instead of an incident. And it should be handled as one rather than chased as a bug. Write what they expected, write what the system is meant to do, and let the two lines settle it.

What Changes It

The last part of the repro is the list of things that were varied and whether the result changed. And it is where the cause hides. Change one thing at a time and write down each attempt with its result, including the ones that made no difference. That is because "no difference" rules something out. The first two experiments are nearly always the same: a different user on the same machine, and the same user on a different machine. Between them they split the world into problems that travel with the person and problems that stay with the hardware. That split alone halves the investigation.

Write what you did not try as well as what you did. "Not yet tried on a wireless connection" saves the next person from assuming it was ruled out. That is the honest shape of most repros, which are finished by the second technician rather than the first. A repro that says "fails for Bea on DES-WS-04 with large files from the shared folder; works for Bea on DES-WS-05; not yet tried with the files copied locally" has done most of Farah's work for her before she has read the ticket. Farah's afternoon of watching the tool open would have been fifteen minutes of copying a file.

Three Vague Complaints Turned Into Repros

Every vague complaint is a repro with the fields empty, and the questions you ask are the fields. Here are three that arrive on every desk, with the questions that turn them into something a stranger can follow. How to ask those questions of a requester without making it feel like an interrogation is covered in translating the vague ticket into one you can work. This is what the answers look like once they are on the ticket.

"The printer doesn't work." Which printer, from which machine, what happens when you print, does anything at all come out, since when, and does it happen to anyone else? The answers at the accounts office became: environment, workstation ACC-WS-07, Dana's account, accounts office, wired, since Tuesday morning. Starting state, signed in with the month-end spreadsheet open. Steps: choose Print, select "ACC-Printer-2," click "Print." Expected: two pages within a minute. Actual: the job sits in the queue showing "Error - Printing," nothing prints, the printer's own panel shows ready. What changes it: Dana printing the same file from ACC-WS-08 prints fine; Marco printing from ACC-WS-07 gets the same error. The cause lives on ACC-WS-07, and the repro said so before a technician had left their chair.

"E-mail is slow." Slow doing what: sending, receiving, opening, searching? How slow, in seconds? On which machine, over which connection? Since when? The answers at the sales office became: opening any message with an attachment takes twenty to thirty seconds on SAL-WS-03 over the wired connection, measured four times. The same mailbox opened through the web client on the same machine opens the same messages in about two seconds. The problem has happened since the Thursday the mail client was updated. Expected: a message opens in a second or two. Actual: twenty to thirty seconds, with the client showing "Loading" the whole time. The cause is the mail client on that machine, not the server, not the network and not the mailbox. And the two-second web-client comparison is what proved it. Without the comparison it would have been a ticket about "the e-mail server."

"The drawing tool crashes sometimes." This is Bea's ticket, written as it should have been written on Monday. And it is below in full as a worked example of the repro block.

REPRO — TICKET 6204 — drawing tool closes when opening large drawings

ENVIRONMENT
  Machine: DES-WS-04        Account: Bea's own account
  Location: design office, second floor, wired
  Since when: Friday last week, after the weekend restart;
              worked normally before that per Bea
  How often: every time with the files below; about 3 times a day

STARTING STATE
  Signed in as Bea. No applications open. The shared drawings
  folder on FS-01 is reachable in the file browser.

STEPS
  1. Open the shared drawings folder in the file browser.
  2. Double-click the drawing file "Assembly-Line-3-rev-D".
     It is about 180 MB.
  3. Wait for the drawing tool to start and show the drawing.

EXPECTED
  The drawing opens in the editor and stays open.

ACTUAL
  The drawing appears for about two seconds, then the window
  closes. No dialog. Nothing in the tool's own log. One entry in
  the system event log at the same time: "Application terminated
  unexpectedly" with the tool's process name.

WHAT CHANGES IT
  Tried                                     Result
  Marco on DES-WS-04, same file             same crash
  Bea on DES-WS-05, same file               opens and stays open
  Bea on DES-WS-04, a 4 MB drawing          opens and stays open
  Same 180 MB file copied to the desktop
    and opened from there                   opens and stays open
  Starting the tool first, then File, Open  same crash
  Not yet tried: other machines restarted the same weekend

Read that and the cause is nearly named: something on DES-WS-04, since the weekend restart, that fails when a large file is opened across the network. Farah, reading it on Monday afternoon, would have checked what the restart installed on that machine. She would have found a network driver update that two other design machines had also received and that had not yet bitten them. Instead she opened the tool twice and watched it work. The repro took Owen about eight minutes to write with Bea at his elbow. The lap of the desk took a day and a half.

A Short War Story: The Scanner That Only Failed on Tuesdays at Northgate Retail

Northgate Retail's Harbor Street store raised three tickets in two months about the handheld stock scanners at the front tills dropping their connection. Each was visited by a technician on a Wednesday or Thursday, tested for twenty minutes, and closed as "cannot reproduce; scanner working on test." Two scanners were replaced along the way and an engineer called out once at a day's cost. The fourth ticket went to Ines, who did not visit at all. She sent the store manager a one-page log and asked for the time, till number and scanner number of every drop for a week. The log came back with nine entries, all on the Tuesday, all between 14:00 and 15:20, all on tills one and two. Those are the two nearest the loading bay. The Tuesday delivery arrived at 14:00 and the bay's roller door stood open for about an hour while it was unloaded. With the door open the two nearest scanners were roaming to the yard's wireless access point. That sat on a network that could not reach the till server. Ines wrote the repro with the day, the time, the door and the two tills in it. The network team moved the yard access point onto the right network in an afternoon. Two replaced scanners, one engineer visit and eight weeks of Tuesday queues had bought nothing. A repro with "since when" and "what changes it" in it had found the cause in a week.

The Repro Block

The block below is the six parts as a template, in the order the next reader needs them. Fill it in at first touch where you can and finish it as you learn more. The "what changes it" table is usually completed by the second person. That is fine, as long as each attempt is written down with its result. Name machines, accounts and files exactly as the systems show them, quote error text exactly, and put numbers where the problem is about speed or quantity. Leave the real values of anything sensitive out and say what kind of value it was. The block belongs in an internal note on the ticket. And the one-line title at the top should be the symptom in plain words, so that a search for the symptom finds it.

REPRO — TICKET ________ — ______________________________________
                           one line: the symptom in plain words

ENVIRONMENT
  Machine: ______________   Account signed in: ______________
  Location: ______________  Wired / wireless / remote: ________
  Since when: ___________________________________________________
  How often: ____________________________________________________
  Differs from working machines by: ____________________________

STARTING STATE
  _______________________________________________________________

STEPS
  1. ____________________________________________________________
  2. ____________________________________________________________
  3. ____________________________________________________________
  4. ____________________________________________________________
  File, record or input used: __________________________________

EXPECTED
  _______________________________________________________________

ACTUAL
  _______________________________________________________________
  Exact error text: "____________________________________________"
  Where it appeared: ____________________________________________
  Screenshot attached: [ ]   Log entry copied: [ ]

WHAT CHANGES IT
  Tried                                        Result
  Another user, same machine: ______________   ___________________
  Same user, another machine: ______________   ___________________
  Different file or input: _________________   ___________________
  Different network or location: ___________   ___________________
  Different time of day: ___________________   ___________________
  After a restart: _________________________   ___________________
  Not yet tried: ________________________________________________

REPRODUCED BY __________ on __________ at __:__   [ ] yes  [ ] no

Owen wrote his first full repro block on the ticket after 6204, with the template open in one window and the requester on the phone in the other. It took him eleven minutes. Farah fixed that ticket from the block alone without connecting to the machine. That was because the block said which of two settings differed between the failing machine and the working one. She sent it back with a note that read "this is what I need every time," which Owen printed and stuck to his monitor. It is still there. The drawing tool, for the record, was a network driver, and it was never seen crashing by anyone but Bea.

Frequently Asked Questions