Gawande's The Checklist Manifesto and reduction of surgical complications

Just finished Dr. Atul Gawande's The Checklist Manifesto: How to Get Things Right. It is so well written, moving and gripping that I would've gone through it in one sitting had the rest of life been kind enough not to butt in. In 2007 to 2008 a team of doctors and researchers, including Gawande, under the auspices of the World Health Organization performed a study that involved hospitals and surgical teams in eight developed and third world countries (USA, Canada, New Zealand, UK, India, Jordan, Philippines, Tanzania) to test the effectiveness of a simple 2-minute, 19-item surgical checklist the team had developed. In both his book and in a TED vid (starts at around 14:00) Gawande shares the incredible and almost unbelievable results they obtained: a drop in the number of deaths by 47% and in overall major complications by 36%.

Imagine being able to cut mortality rate in half! At very little cost and with no invasive procedures, drugs and side effects. Given how checklists have been effective in a number of areas including the airline and construction industries, it stands to reason that medicine could benefit from having checklists in place to make sure vital items/steps are not missed or incorrectly performed.

And now for the nitpick. First and quite clearly this was not a randomized, blinded clinical trial so certain biases may have affected the results. As Gawande himself pointed out, among other things, he was concerned the Hawthorne effect may have kicked in (but as he says the fact that observers were in place during the baseline data collection phase ought to have canceled out any effects resulting from that effect.) Secondly, and specifically with how Gawande presented the study findings, the double digit numbers can be misleading and can inflate the benefits of the intervention (the checklist). And this is because Gawande chose to impress us by providing the relative risk reduction and not the absolute numbers or percentages. For those figures I had to search for their published study. And so in their NEJM paper the results they obtained were summarized as follows:

The rate of death was 1.5% before the checklist was introduced and declined to 0.8% afterward (P=0.003). Inpatient complications occurred in 11.0% of patients at baseline and in 7.0% after introduction of the checklist (P = 0.001). 

Given the p values these results are statistically highly significant. Using the absolute risks above the absolute reduction in death is 1.5 - 0.8% = 0.7% and in overall complications ARR = 11% - 7% = 4%. In other words, 1 out of 143 inpatients was spared an untimely and needless death, and 1 in 25 spared major complications. These numbers I believe provide a much more realistic picture. Because death and complications are--fortunately--not that frequent (1.5% and 11%, respectively), quoting relative risk reductions has the tendency to mislead and to exaggerate the benefits (or risks).

Nonetheless, given that such a simple, inexpensive, apparently harm-free intervention does reduce major complications, particularly death, ought to be sufficient to spur the medical field to study checklists further, and implement, fine tune and improve the checklist that Gawande et al. have created.