Publication

• Title: Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients: A Randomized Clinical Trial
• Acronym: CHECKLIST-ICU
• Year: 2016
• Journal published in: JAMA
• Citation: Cavalcanti AB, Bozza FA, Machado FR, et al; Writing Group for the CHECKLIST-ICU Investigators and the Brazilian Research in Intensive Care Network (BRICNet). Effect of a quality improvement intervention with daily round checklists, goal setting, and clinician prompting on mortality of critically ill patients: a randomized clinical trial. JAMA. 2016;315(14):1480-1490.

Context & Rationale

• Background

  • ICU care requires consistent delivery of multiple evidence-based processes (ventilation strategy, sedation practice, device management, infection prevention, nutrition, thromboprophylaxis) under time pressure and high cognitive load.
  • Omission errors and failures to establish/communicate daily goals during rounds are well described, and may be amplified by hierarchical team dynamics.
  • Checklist-based approaches had been widely promoted (including in critical care) to standardise key processes and improve teamwork; however, most prior ICU checklist studies were before–after designs with substantial risk of bias and uncertain effects on hard clinical endpoints.
  • Brazilian ICUs had high mortality and marked variability in structure, processes, and safety culture, making them a plausible setting in which improved reliability of evidence-based care might translate into improved outcomes.
• Research Question/Hypothesis

  • Does a multifaceted quality-improvement programme (daily round checklist + explicit daily goals + clinician prompting) reduce in-hospital mortality (truncated at 60 days) among adult ICU patients with ICU stay >48 hours, compared with routine care?
  • Mechanistic premise: mortality benefit (if present) would be mediated by improved adherence to targeted care processes and improved safety culture/teamwork.
• Why This Matters

  • Checklist interventions are low-cost, scalable, and frequently adopted without rigorous evaluation; this trial tested a widely generalisable implementation strategy at scale using a cluster-randomised design.
  • By pairing checklist use with goal setting and “closed-loop” prompting, the intervention intentionally targeted both technical (process) and social (team dynamics/hierarchy) determinants of care reliability.
  • A null result would be equally practice-shaping: it would challenge assumptions that improving a bundle of common ICU processes via checklists necessarily improves mortality within pragmatic timeframes.

Design & Methods

• Research Question: In adult ICU patients with ICU stay >48 hours, does a multifaceted QI intervention using daily round checklists, explicit goal setting, and clinician prompting reduce in-hospital mortality (≤60 days) compared with routine care?
• Study Type: Two-phase (baseline observational then cluster-randomised), multicentre, pragmatic, investigator-initiated, parallel-group cluster trial across 118 adult ICUs in Brazil; randomisation stratified by baseline ICU mortality (observational phase). The protocol and statistical analysis plan were published in advance.¹²
• Population:
  • Cluster (ICU) eligibility: Adult ICUs across all Brazilian regions; required leadership willingness to implement multidisciplinary daily rounds (at least physician + nurse on working days); excluded paediatric ICUs, exclusively cardiac ICUs, step-down/semi-intensive units, ICUs whose leadership did not/would not implement rounds, and ICUs already using checklists during rounds.
  • Cluster run-in requirement for randomisation: Successful observational-phase data collection (≥30 patients within 6 months and Safety Attitudes Questionnaire response from >75% of ICU staff).
  • Patient inclusion: Consecutive adult patients (≥18 years) with ICU stay longer than 48 hours; per ICU, 40–60 consecutive eligible patients were enrolled in each phase.
  • Patient exclusions: Moribund patients with high probability of death anticipated between the 48th and 72nd hours of ICU stay; patients admitted for exclusive palliative care; suspected or confirmed brain death.
• Intervention:
  • Core clinical components (weekdays): Daily multidisciplinary rounds modified so that a nurse read a checklist aloud and the team responded; explicit daily goals were recorded on a standardised form and read aloud; the checklist targeted 11 care processes (VTE prevention, VAP prevention including head-of-bed elevation, CLABSI/UTI prevention via device removal, nutrition/analgesia, reduced sedation, readiness for extubation, sepsis/ARDS detection, antibiotic optimisation, and tidal volume reduction).
  • Closed-loop follow-through: Every afternoon, a nurse reviewed goals and used clinician prompting to contact the on-call physician about pending items.
  • Implementation strategy (“multifaceted”): Investigators meeting; initiation visit by a steering-committee intensivist to each ICU (training + participation in rounds); monthly audit & feedback reports with target goals; study website/resources; videos; active SMS reminders 1–3 times per week; targeted contact with medical/nursing directors if adherence was low.
  • Duration/exposure: Intervention intended to be applied for all eligible patients during their entire ICU stay; goal was application at least Monday–Friday.
• Comparison:
  • Routine care (including multidisciplinary visits where locally practised) with no pre-intervention training, no structured checklist/goals form, and no clinician prompting during the trial period.
  • As an inclusion agreement, control ICUs received the intervention after study completion.
• Blinding: Caregivers were not blinded (cluster-level behavioural intervention); research coordinators and the statistician were not blinded (adherence data collected only in intervention ICUs). Ventilator-associated pneumonia and central line–associated bloodstream infection diagnoses were adjudicated by a blinded committee using standardised definitions.
• Statistics: A total of 102 ICUs (≈50 patients/ICU; ≈5100 patients) were required to detect a 6% absolute reduction in in-hospital mortality (from 30% to 24%) with 90% power at the 5% significance level, assuming a coefficient of variation (K) of 0.25 and analysis adjusted for clustering; analyses followed intention-to-treat principles with mixed-effects models accounting for ICU clustering (primary: random-effects logistic regression adjusted for baseline ICU standardised mortality ratio and patient SAPS 3).
• Follow-Up Period: In-hospital mortality truncated at 60 days; ventilator-free days at 28 days; selected process and infection outcomes were assessed during ICU stay (many measured repeatedly from ICU day 2 to 17 by protocol definitions).

Key Results

This trial was not stopped early. Recruitment and follow-up were completed as planned (randomised phase: 118 ICUs; 6761 patients).

• Primary outcome: no evidence of mortality benefit (adjusted OR 1.02; 95% CI 0.82 to 1.26; P=0.88), despite high implementation fidelity and measurable culture/process changes.
• Process/culture effects: improvements were concentrated in selected processes with greater baseline “room to improve” (notably urinary catheter use: 62.8% vs 74.8% of patient-days; adjusted RR 0.86; P<0.001) and modest improvements in teamwork/safety climate.
• Subgroups:
  • Prespecified clinical subgroups: no evidence of effect modification across strata such as baseline ICU mortality, hospital regime (public vs private), admission type, SOFA score, sepsis at admission, or mechanical ventilation at admission (interaction P values reported as non-significant).
  • Exploratory safety-culture interactions: baseline safety climate (below median: OR 0.75; 95% CI 0.53 to 1.04 vs above median: OR 1.38; 95% CI 0.99 to 1.92; interaction P<0.01) and baseline perception of management (below median: OR 0.73; 95% CI 0.53 to 1.02 vs above median: OR 1.43; 95% CI 1.02 to 1.99; interaction P<0.01) suggested heterogeneity that is hypothesis-generating.

Internal Validity

• Randomisation and allocation:
  • Cluster randomisation at the ICU level (to minimise contamination), with random permuted blocks of 4 and stratification by median baseline in-hospital mortality from the observational phase.
  • Allocation concealment at sequence generation: the statistician received only ICU identification codes (not ICU identities) when preparing the allocation list.
• Dropout/exclusions and missing data:
  • ICUs were randomised only after meeting prespecified observational-phase data completeness (≥30 patients; SAQ response >75% staff), reducing risk of post-randomisation cluster attrition.
  • Vital status at hospital discharge was missing for 3 patients (handled with multiple imputation); primary analysis remained essentially unchanged.
  • Eligibility was defined by ICU stay >48 hours (a post-admission criterion); the trial enrolled consecutive eligible patients but did not report screening logs for all ICU admissions, which limits assessment of selection mechanisms.
• Performance/detection bias:
  • Blinding of clinicians was not feasible; co-interventions and documentation behaviour could therefore be influenced (Hawthorne/performance effects).
  • Primary outcome (mortality) is objective; VAP and CLABSI were adjudicated by a blinded committee (mitigating outcome misclassification for these endpoints).
• Protocol adherence and separation of the variable of interest:
  • Multidisciplinary rounds frequency (randomised phase): 92.8 days per 100 patient-weekdays (intervention) vs 61.5 (control).
  • Checklist application (intervention group): 90.6 days per 100 patient-weekdays.
  • Clinician prompting (intervention group): 89.1 days per 100 patient-weekdays.
  • Process separation demonstrated in multiple endpoints (e.g., urinary catheter use 62.8% vs 74.8% of patient-days; low tidal volume 67.5% vs 58.9% of ventilated patient-days).
• Baseline characteristics and illness severity:
  • At the ICU level, there were imbalances typical of pragmatic cluster trials (e.g., academic hospital status and ICU bed numbers differed between groups).
  • At the patient level, groups were broadly similar; mean SAPS 3 at admission was lower in the intervention group (51.2 ± 17.9) than in the control group (54.2 ± 17.5), a difference that would, if anything, bias towards benefit in intervention (yet no mortality reduction was observed).
• Heterogeneity and clustering:
  • Large number of clusters (59 per arm) strengthens robustness to between-ICU heterogeneity; analyses appropriately accounted for ICU clustering.
  • Observed intra-cluster correlation was reported (e.g., ICC 0.13 for ICU mortality), consistent with substantial between-ICU variability.
• Timing and dose:
  • Intervention applied primarily on weekdays; this may limit “dose” in ICUs with substantial weekend clinical decision-making drift.
  • Intervention period was constrained to ≤6 months (per funder/ICU leadership request), potentially limiting time for deeper organisational change to affect hard endpoints.
• Outcome assessment and statistical rigour:
  • Objective primary endpoint; prespecified mixed-effects modelling and intention-to-treat principles were used, aligning with cluster-randomised design.
  • Multiple secondary outcomes increase the false-positive risk; the manuscript reported sensitivity to multiplicity (e.g., Sidak correction materially reduced the set of “statistically significant” secondary findings).

Conclusion on Internal Validity: Overall, internal validity appears moderate to strong: allocation was concealed and cluster-adjusted analyses were prespecified, implementation fidelity was high, and the primary endpoint was objective; the main threats are lack of clinician blinding (performance effects) and limited transparency around the denominator of all ICU admissions given eligibility defined at >48 hours.

External Validity

• Population representativeness:
  • ICUs spanned Brazilian regions and included public/private settings, but participating sites were part of a research network and required baseline data-collection capability and willingness to engage with a structured QI programme.
  • ICUs already using checklists during rounds and those not performing multidisciplinary rounds were excluded, limiting direct generalisation to settings without baseline rounding infrastructure or with mature checklist systems already embedded.
• Patient-level applicability:
  • Eligibility restricted to ICU stay >48 hours; findings most directly apply to patients with prolonged ICU exposure and do not directly address effects on early ICU trajectories (rapid discharges or early deaths).
• Intervention transferability:
  • The intervention is low-technology and scalable (paper/electronic checklist + structured rounding behaviours + prompting), but it requires nursing empowerment, reliable multidisciplinary rounds, and local leadership engagement.
  • Effects are plausibly context dependent (baseline adherence, staffing ratios, safety culture, and organisational readiness), consistent with the observed exploratory interactions with baseline safety culture domains.

Conclusion on External Validity: Generalisability is moderate to adult ICUs capable of structured multidisciplinary rounds and willing to implement checklist-driven prompting; applicability is less certain in ICUs without stable rounding teams, in units with already high baseline adherence to checklist targets, or in health systems with different organisational constraints.

Strengths & Limitations

• Strengths:
  • Large pragmatic cluster-randomised evaluation across 118 ICUs (59 per arm), minimising contamination and increasing inferential stability across heterogeneous practice settings.
  • Prespecified protocol and statistical analysis plan; analytic approach matched the cluster design with mixed-effects models and adjustment for baseline ICU performance and patient severity.¹²
  • High implementation fidelity and direct measurement of both technical processes and safety culture (SAQ), allowing mechanistic interpretation beyond the primary endpoint.
  • Blinded adjudication for VAP and CLABSI mitigated outcome classification bias for key HAI outcomes.
• Limitations:
  • Unblinded clinicians and research staff (for adherence data) increase susceptibility to performance effects and measurement reactivity, especially for process outcomes.
  • Eligibility based on ICU stay >48 hours (and exclusion of patients expected to die between 48–72 hours) creates a selected patient cohort; comprehensive screening denominators were not reported.
  • Intervention duration capped at ≤6 months by stakeholder request, potentially too short for downstream mortality effects in complex ICUs.
  • Multiple secondary outcomes: nominal significance did not consistently persist after multiplicity correction; process improvements were uneven across checklist targets.

Interpretation & Why It Matters

• Mortality signal

  The intervention produced no detectable reduction in in-hospital mortality (32.9% vs 34.8%; adjusted OR 1.02; 95% CI 0.82 to 1.26), implying that improving a subset of common ICU processes and teamwork climate does not necessarily translate into short-horizon mortality benefit in heterogeneous, real-world ICUs.
• What improved (and why)

  Improvements concentrated in areas where omission is common and decisions are frequently deferred (device removal; lung-protective ventilation; sedation depth), supporting checklists as reliability tools for “everyday” ICU decisions rather than as standalone mortality-reducing therapies.
• Implementation lesson

  The “flattened hierarchy” design (nurse-led read-aloud + explicit goals + later prompting) measurably improved teamwork and safety climate, demonstrating that checklist interventions can shift culture even when hard outcomes are unchanged.

Controversies & Subsequent Evidence

• Selection and eligibility defined after 48 hours: Published correspondence highlighted that enrolling only patients surviving to >48 hours (and excluding those anticipated to die between 48–72 hours) can complicate interpretation and may limit representativeness; the authors replied with clarifications on design intent and bias control.³⁴
• Outcome choice vs expected mechanism: The intervention improved specific care processes and safety culture metrics but did not affect mortality; this mismatch reinforces that mortality may be an insensitive endpoint for broad, multi-target behavioural interventions unless baseline non-adherence is extreme, the “dose” is continuous (including weekends), or the targeted processes have large attributable mortality effects.
• Multiplicity and interpretation of secondary outcomes: Several secondary outcomes were nominally significant (e.g., low tidal volume, sedation depth, device utilisation, SAQ domains), but after stringent correction for multiple comparisons, urinary catheter use remained the clearest signal; this matters because process endpoints can otherwise be over-interpreted in complex QI trials.
• Hypothesis-generating heterogeneity by baseline culture: Exploratory interactions suggested the intervention might behave differently depending on baseline safety climate/perception of management; these results are not practice-changing but inform implementation science questions about where checklist programmes might be most effective.
• Subsequent synthesis: Systematic evaluations of ward-round/checklist interventions have generally found improvements in process measures and communication outcomes with low-to-moderate certainty, while effects on mortality and length of stay remain inconsistent across heterogeneous designs and settings.⁵⁶
• Alignment with infection-prevention implementation strategies: Contemporary HAI prevention guidance emphasises daily assessment of device necessity (central lines, urinary catheters, ventilation) and structured team processes to reliably implement bundles—areas directly targeted by CHECKLIST-ICU, even if mortality was unchanged.⁷⁸⁹

Summary

• In 118 Brazilian adult ICUs, a high-fidelity checklist + daily goals + clinician prompting intervention did not reduce in-hospital mortality (32.9% vs 34.8%; adjusted OR 1.02; 95% CI 0.82 to 1.26).
• The intervention did improve several targeted care processes, especially urinary catheter utilisation (62.8% vs 74.8% of patient-days; adjusted RR 0.86; P<0.001) and low tidal volume ventilation (67.5% vs 58.9%; adjusted RR 1.14; P=0.01).
• Teamwork and safety climate improved modestly (teamwork climate OR 1.30; safety climate OR 1.27), supporting the concept that checklist programmes can influence ICU culture.
• Device-associated infection rates and ventilator-free days were not improved (rate ratios and mean differences all non-significant).
• CHECKLIST-ICU is practice-shaping primarily as an implementation science trial: it supports checklists as reliability/culture tools but cautions against assuming mortality benefit from broad checklist adoption alone.

Overall Takeaway

CHECKLIST-ICU is a landmark implementation-science cluster trial showing that a well-supported, high-fidelity daily rounding checklist with explicit goals and clinician prompting can improve selected ICU processes and aspects of safety culture at scale, yet still fail to reduce mortality. It reoriented expectations for checklist programmes: they are best viewed as tools to improve reliability and team function, with clinical outcome benefits likely dependent on context, baseline adherence, and the mortality-attributable fraction of targeted processes.

Bibliography

• 1.Machado FR, Bozza FA, Salluh JI, et al. CHECKLIST-ICU trial: checklist during multidisciplinary daily visits and clinician prompting for reduction of mortality in intensive care units: study protocol for a cluster randomised trial. Implement Sci. 2015;10:8.
• 2.Damiani LP, Cavalcanti AB, Moreira FR, et al. A cluster-randomised trial of a multifaceted quality improvement intervention in Brazilian intensive care units (Checklist-ICU trial): statistical analysis plan. Crit Care Resusc. 2015;17(2):113-121.
• 3.Nunez RM, Garcia Perdomo HA, Munoz Chavez IC. Quality Improvement Intervention and Mortality of Critically Ill Patients. JAMA. 2016;316(8):879.
• 4.Cavalcanti AB, Machado FR, Berwanger O. Quality Improvement Intervention and Mortality of Critically Ill Patients—Reply. JAMA. 2016;316(8):879-880.
• 5.Wen X, et al. Systematic evaluation of the effect of bedside ward round checklist on clinical outcomes of critical care patients. J Healthc Eng. 2021;2021:8105516.
• 6.Erikson EJ, et al. The use of checklists in the intensive care unit: a scoping review. Crit Care. 2023;27:468.
• 7.Buetti N, Marschall J, Drees M, et al. Strategies to prevent central line-associated bloodstream infections in acute-care hospitals: 2022 update. Infect Control Hosp Epidemiol. 2022;43(5):553-569.
• 8.Klompas M, Branson R, Cawcutt K, et al. Strategies to prevent ventilator-associated pneumonia, ventilator-associated events, and nonventilator hospital-acquired pneumonia in acute-care hospitals: 2022 update. Infect Control Hosp Epidemiol. 2022;43(6):687-713.
• 9.Patel PK, Advani SD, Kofman A, et al. Strategies to prevent catheter-associated urinary tract infections in acute-care hospitals: 2022 update. Infect Control Hosp Epidemiol. 2023;44(8):1209-1231.