What the surgical checklist trials actually found, and what that means for your company
The claim arrives in almost every management book: a nineteen-item checklist halved surgical deaths, therefore your company needs checklists. The study behind that claim is real and the number is quoted accurately. What almost never gets mentioned is the larger study that came afterwards, tried the same thing across an entire Canadian province, and found nothing. Both results are worth knowing, because together they say something much more useful than either one alone.
The study everyone quotes
In 2009 the New England Journal of Medicine published Haynes and colleagues on the World Health Organization’s Surgical Safety Checklist. Eight hospitals across eight cities, from Toronto and Seattle to Ifakara and Manila. The team collected data on 3,733 patients before the checklist was introduced and 3,955 afterwards. Inpatient death fell from 1.5% to 0.8%, and complications from 11.0% to 7.0%. The effect held when any single site was removed from the model, and it appeared in both high-income and lower-income hospitals.
Those are genuinely striking numbers and the paper has been cited thousands of times. The design, though, is a before-and-after comparison with no control group. Everyone in the operating theatre knew the checklist was being studied, surgical mortality was already falling year on year for unrelated reasons, and other improvements were happening at the same time. None of that makes the result wrong. It does mean the result cannot carry the causal weight that gets piled on it.
The study almost nobody quotes
Ontario then ran the experiment that the first study could not. The province mandated public reporting of checklist adherence from July 2010, which meant checklists arrived across the whole hospital system rather than in eight volunteer sites. Urbach and colleagues compared the three months before adoption with the three months after, across 101 hospitals, covering 109,341 procedures before and 106,370 after.
Adjusted risk of death went from 0.71% to 0.65%, an odds ratio of 0.91 with a confidence interval from 0.80 to 1.03, which does not reach significance at P=0.13. Complications went from 3.86% to 3.82%, P=0.29. Length of stay moved from 5.11 days to 5.07, a difference the authors described as statistically significant but small and not clinically relevant. Emergency department visits and readmissions did not improve either. The published conclusion is that implementation was not associated with significant reductions in operative mortality or complications.
This is roughly thirty times the patient volume of the study that made checklists famous, and it found no outcome benefit.
Reconciling the two
The obvious reading is that checklists do not work. A better reading comes from Urbach himself, who noted that a greater effect might occur with more intensive team training or better monitoring of compliance, and that as implemented the checklists did not improve outcomes. The phrase doing the work there is as implemented.
In the 2009 study the checklist arrived with a programme: training, observers, a team that had chosen to participate. In Ontario it arrived as a reporting requirement. Hospitals were asked to report adherence, and adherence is exactly what they produced. The artefact transferred perfectly and the behaviour did not transfer at all.
There is a sharper version of the critique in a 2019 BMJ debate where the authors of both studies argue it out directly. The sceptical case is that the individual steps on the checklist were already required in modern hospitals, and that none of them had been shown to reduce mortality on its own, which makes a halving of deaths from confirming them implausibly large. The one randomised trial available at that point, a stepped wedge design in two hospitals, found no statistically significant reduction in postoperative mortality.
So the honest summary is narrower and more practical than the management-book version. A checklist is a way of making a team do something it has agreed to do. Where the agreement and the training exist, writing the steps down helps. Where they do not, writing the steps down produces a signed form.
Where writing it down is not optional
For a UK employer this stops being a philosophical question at five employees. Regulation 3 of the Management of Health and Safety at Work Regulations 1999 requires every employer to make a suitable and sufficient assessment of risk, with no size threshold at all. Regulation 3(6) then adds that an employer with five or more employees must record the significant findings and any group of employees identified as being especially at risk. Regulation 5 applies the same threshold to recording your health and safety arrangements.
Below five employees you are still required to do the assessment. You are simply not required to write it down. HSE guidance is direct about this and recommends writing it anyway so you can review it when something changes.
The Approved Code of Practice adds a detail worth copying for any internal process document: the record should be retrievable for use in management reviews and by inspectors, and it can be combined with your other health and safety documents into a single document rather than maintained as a separate artefact. A record nobody can find is not a record.
What this means for meetings
The useful test falls out of the Ontario result. A checklist earns its place when it removes a conversation. It costs you something when it adds a ritual.
If a recurring meeting exists so that people can confirm that known steps happened, that meeting is a checklist wearing an hour of everyone’s time. Convert it and delete the meeting. If a meeting exists because a decision has to be made with judgement and incomplete information, no checklist replaces it, and turning it into an agenda template will only make it slower.
The failure mode to watch for is the one Ontario demonstrated at scale. Reporting compliance produces compliance. If the measure you track is whether the list was completed, you will reliably get completed lists, and you will learn nothing about whether the work got better. A process that generates evidence of activity rather than change in outcomes is the most expensive kind, because it looks like management.
How to tell a real one from theatre
- It is short enough to be used at the moment of action rather than filed afterwards. The WHO list is nineteen items covering three pause points in an operation, not a manual.
- It has a named owner who can change it. Lists that nobody is allowed to edit stop matching the work within months.
- It contains at least one item that can stop the process. A checklist with no possible failure state is a receipt.
- It is read out rather than ticked silently, where more than one person is involved. The communication is a large part of what the surgical version was doing.
- Somebody checks whether outcomes moved, not just whether the list was completed. This is the step Ontario skipped.
Two honest limits on all of this. The trials above are about surgery, where the steps are well defined, the failure is immediate and visible, and the team is in one room. Office and operational work rarely has any of those properties, so treating the surgical evidence as proof that checklists improve knowledge work is an analogy rather than a finding. And the legal thresholds described here are the position for employers in Great Britain under the 1999 Regulations; health and safety enforcement in Northern Ireland sits with its own separate regime, so check the local requirement rather than assuming this transfers.