Original Article. Intra-operative monitoring many alarms with minor impact. Summary

Transcription

1 Original Article doi: /anae Intra-operative monitoring many alarms with minor impact F. R. de Man, 1 S. Greuters, 2 C. Boer, 3 D. P. Veerman 4 and S. A. Loer 5 1 Clinical Research Physician, 2 Staff Anaesthetist, 3 Associate Professor, 4 Staff Anaesthetist, 5 Professor of Anaesthesiology, Anaesthesiology Department, V.U. University Medical Centre/Institute for Cardiovascular Research, Amsterdam, the Netherlands Summary Alarms are key components of peri-operative monitoring devices, but a high false-alarm rate may lead to desensitisation and neglect. The objective of this study was to quantify the number of alarms and assess the value of these alarms during moderate-risk surgery. For this purpose, we analysed documentation of anaesthesia workstations during 38 surgical procedures. Alarms were classified on technical validity and clinical relevance. The median (IQR [range]) alarm density per procedure was 20.8 ( [ ]) alarms.h 1 (1 alarm every 2.9 min) and increased during induction and emergence of anaesthesia, with up to one alarm per 0.99 min during these periods (p < 0.001). Sixty-four per cent of all alarms were clinically irrelevant, whereas 5% of all alarms required immediate intervention. The positive predictive value of an alarm during induction and emergence was 20% (95% CI 16 24%) and 11% (95% CI 8 14%), respectively. This study shows that peri-operative alarms are frequently irrelevant, with a low predictive value for an emerging event requiring clinical intervention.... Correspondence to: F. R. de Man fr.deman@vumc.nl Accepted: 2 April 2013 This article is accompanied by an Editorial by Edworthy, pp of this issue. Alarms are key components of peri-operative monitoring devices in patients undergoing anaesthesia and surgery, and function as signals for life-threatening situations as well as device malfunction or imminent danger. Moreover, they may also provide information about non-critical incidents [1]. A high alarm rate and large number of false alarms may cause desensitisation, with consequent neglect of appropriate and critical alarms during device malfunction or life-threatening situations [2, 3]. Studies in paediatric and adult intensive care units demonstrated that up to 90% of the alarms were false [3, 4]. In particular, the positive predictive value (PPV) of monitoring alarms in these settings was only 16% and 27%, respectively [2, 3]. This low PPV reduces the effectiveness of monitors and may lead to delayed or absent responses to a correct alarm due to perceived unreliability [1, 5]. The limited number of studies focussing on alarm density and false-alarm rates in the intra-operative setting reported an alarm rate of 3 10 alarms per case and an alarm density up to one alarm every 4.5 min [6 8]. These studies, dating back over a decade, showed high numbers of technically false alarms. There have been significant advances in medical device technology [9], however, and this might make these data less applicable to currently used monitors. Recently, a density of 1.2 alarms.min 1 was reported during cardiac surgery, and of these, 80% of alarms had no therapeutic consequences [10]. This finding cannot be extrapolated to alarm rates and density during more minor surgery. We therefore investigated the The Association of Anaesthetists of Great Britain and Ireland

2 De Man et al. Intra-operative monitoring and alarms Anaesthesia 2013, 68, number of alarms during moderate-risk surgery and assessed the clinical relevance of these alarms. Methods This observational study was performed using video recording of the anaesthesia workspace during general surgical procedures. The institutional ethical review board approved the study but waived the need for consent from patients, as they were not filmed. Assent for video recording was obtained from the anaesthesia team, who were blinded to the aim of the study. Recording was performed during elective surgical procedures with moderate risk, including oncological, gynaecological, orthopaedic, pulmonary, gastrointestinal and ear, nose and throat. High-risk procedures such as cardiothoracic surgery and neurosurgery as well as emergency procedures were excluded. There were no other predefined inclusion or exclusion criteria. Patients were monitored by a combination of a patient monitor (Solar 9500; GE/Marquette, General Electric Company, Fairfield, CA, USA) and a Primus or Zeus anaesthesia workstation (Dr ager Medical GmbH, L ubeck, Germany). Monitors were equipped with default alarm settings that were frequently adjusted by the anaesthetic staff based on patient and procedural factors. During the study, the anaesthetist received no instructions regarding alterations to the alarm settings. Monitoring included an electrocardiogram (ECG), pulse oximetry and non-invasive blood pressure measurements, occasionally supplemented by monitoring of intra-arterial blood pressure (n = 8) and/or central venous blood pressure (n = 3). In 26 out of 38 procedures, infusion pumps were used. In five cases, patients were anaesthetised using regional anaesthesia without the use of mechanical ventilation. Data collection started upon arrival of the patient in the operating room. Video recordings were made using a Toshiba Camileo S30 (Toshiba Corporation, Tokyo, Japan) and a Samsung SMX-F50BP/EDC (Samsung Group, Seoul, South Korea) camcorder. The camera was secured on an intravenous pole opposite the patient monitor and anaesthesia workstation. The view did not include the patient. During video recording, clinical annotations were made by the researcher on a standard evaluation form. We have a local patient data management system in which the patient monitor and ventilator are connected to a network that digitally stores time-stamped data in dedicated full-disclosure files. The time-stamped alarm history was obtained from this data management system. Films were analysed using VLC Mediaplayer (VideoLAN; Free Software Foundation, Inc., Boston, MA, USA). For all alarms, we registered the alarm-producing device, the parameter related to the alarm, the cause of the alarm if this was identifiable and the effect and/or response of the anaesthesia team. The duration, defined as the time an alarm was audible, was assessed. When an alarm occurred again after being disabled, it was counted as a new event. In our setting, non-lifethreatening ventilator alarms are audible for 1 s, followed by visual continuation of the signal on the monitor. The audible signal is repeated every 30 s until the end of the alarm. Each individual audible signal was therefore recorded as a separate alarm of 1 s duration. For analysis, all procedures were divided into the induction, maintenance or emergence phase. Induction consisted of the time from the patient s arrival in the operating room until the end of anaesthetic induction. Maintenance lasted from the end of induction until the end of the surgical procedure. Emergence consisted of reversal of anaesthesia and extubation, in the period from the end of surgery until the patient left the operating room. Based on previous publications, alarms were classified in four categories according to clinical relevance and technical validity (Fig. 1) [3, 6 8, 10 13]. A measurement was considered technically correct if it accurately represented the patient s state and was not due to artefact such as interference from electrocautery or movement of the pulse oximetry probe. If the technical correctness could not be assessed, the alarm was excluded from analysis. The assessment of clinical relevance was based on subsequent interventions such as therapeutic interventions and adjustment of thresholds. Alarms were considered high-priority if the measurement was technically correct, clinically relevant and required an intervention. Low-priority alarms were technically correct and clinically helpful, requiring attention, but not immediate action. In case of a technically correct, but clinically irrelevant alarm, the signal was classed as irrelevant. In case of an incorrect measurement, the alarm was classified as false The Association of Anaesthetists of Great Britain and Ireland 805

3 De Man et al. Intra-operative monitoring and alarms Assessable alarms n = 1632 False n = 396 Technically true n = 1236 Clinically relevant n = 591 Irrelevant n = 645 High priority n = 75 Low priority n = 516 Figure 1 Classification of alarms by accuracy, relevance and priority. Data were analysed using GraphPad Prism 5 (GraphPad Software, Inc., La Jolla, CA, USA) and SPSS 18.0 (IBM, New York, NY, USA). True-positive alarms were classified as correct alarms of either high or low priority, whereas false-positive alarms were irrelevant or false alarms. Positive predictive value (PPV) was calculated as PPV = true positive/(true positive + false positive). All data were verified for normal distribution. Alarm density in each peri-operative phase was tested by repeated-measures ANOVA with Bonferroni post hoc comparison after log-transforming the data and adding a constant. The Kruskal Wallis test was used to analyse the difference in alarm density between different types of surgery and Mann Whitney U-test for the difference in alarm density between non-invasive and intra-arterial blood pressure. A p value < 0.05 was considered statistically significant. Results From December 2011 until February 2012, 51 patients were studied, with a mean (SD) age of 54.1 (18.2) years. There were 13 males and 38 females. Seventeen patients had an ASA classification of 1, 19 were ASA 2, and 15 were ASA 3. The participating anaesthesia teams consisted of 13 staff anaesthetists, 16 anaesthetic residents and 16 anaesthetic nurses. Nine video recordings were not available for analysis due to technical malfunction of the camera or failure to record the complete alarm history. During four procedures, all alarms were switched off for an unspecified reason throughout the procedure and were therefore excluded for the analysis. In the remaining 38 surgical procedures, there were 72.5 h of video recording, during which 1665 alarms were recorded. The median (IQR [range]) alarm density per procedure was 20.8 ( [ ]) alarms per hour (1 alarm every 2.9 min). The alarm density was lower in orthopaedic procedures vs oncological and gynaecological surgical procedures (Table 1). Haemodynamic, ventilator and infusion pump alarms occurred in 36%, 61% and 3% of the cases, respectively. The median (IQR [range]) number of alarms was 43 (28 55 [3 139]) per procedure, and alarms were switched off during 17% (11 30% [0 60%]) of the procedures (Table 2). The alarm density was significantly higher during emergence from anaesthesia than during maintenance (Fig. 2, Table 2). Table 1 Number of surgical procedures and alarm density according to type of surgery. Values are median (IQR [range]). Type of surgery Number of procedures Alarm density; number.h 1 Oncological 9 27 (19 35 [16 45]) Gynaecological 7 25 (20 78 [8 86]) Orthopaedic 7 9 (4 13 [4 17])* Pulmonary 5 21 (16 37 [12 42]) Gastrointestinal 5 21 (20 36 [20 44]) Ear, nose, throat 5 20 (12 46 [8 53]) *p < 0.05 vs oncological and gynaecological procedures The Association of Anaesthetists of Great Britain and Ireland

4 De Man et al. Intra-operative monitoring and alarms Anaesthesia 2013, 68, All 1665 alarms were assessed for technical validity and clinical relevance (Fig. 1, Table 3). In 33 cases, alarms were not assessable for technical validity and Table 2 Duration and frequency of alarms per procedure during 38 cases of moderate-risk surgery. Values are median (IQR [range]). Monitoring duration; min 104 ( [34 271]) Alarms; number 43 (28 55 [3 139]) Alarm density; number.h 1 21 (15 34 [4 86]) Alarm density during 33 (17 60 [0 142]) induction; number.h 1 Alarm density during 13 (7 25 [0 91]) maintenance; number.h 1 Alarm density during 60 ( [0 209]) emergence; number.h 1 Duration of disabled 18 (10 28 [ ]) alarms; min Duration of disabled alarms; % 17% (11 30% [0 60%]) Haemodynamic alarms; number 11 (5 21 [0 82]) Respiratory alarms; number 23 (12 41 [0 90]) Infusion pump alarms; number 1 (0 2 [0 18]) Alarm density (number.min 1 ) Induction Maintenance Emergence Figure 2 Alarm density during different peri-operative phases. Horizontal bar = median. *p < * excluded from further data analysis. In the total 1632 assessable alarms, 64% were clinically irrelevant, with 24% due to technical error. In the group of clinically relevant warnings, 5% of the alarms were high-priority alarms. During the induction and emergence of anaesthesia, 75 out of the 375 alarms and 52 out of the 473 alarms were clinically relevant, resulting in a low PPV for clinical relevance of 20% and 11%, respectively (Table 3). Alarms were separated according to the physiological variable based on the cause of the alarm (Table 4). Among the ventilator alarms, apnoea and minute volume warnings were most frequently recorded, of which 91% and 94%, respectively, were clinically irrelevant. Blood pressure alarms were often clinically relevant, but needed no immediate intervention in most cases and were therefore classified as low-priority alarms. The alarm density of intra-arterial blood pressure monitoring was 8 (6 11 [3 14]) alarms per hour, which was significantly higher than the alarm density of non-invasive blood pressure monitoring with 3 (1 7 [0 30]) alarms per hour (p = 0.023). The PPV of the intra-arterial blood pressure alarms was 63% (95% CI 56 70%), compared with a PPV of 84% (95% CI 56 70%) for non-invasive blood pressure monitor alarms. Discussion This observational study investigated the number and clinical relevance of alarms in the daily working routine of the anaesthetist. We have shown that during moderate-risk surgery, the majority of the alarms generated by the anaesthesia monitor and ventilator are irrelevant. There is a high alarm density with a low value, particularly during induction of and emergence from anaesthesia. Table 3 Number of alarms and positive predictive value according to the peri-operative phase. Results are number (proportion). Clinically relevant Clinically irrelevant High priority Low priority Irrelevant False Not assessable Total PPV (95% CI) Induction 14 (4%) 61 (16%) 121 (32%) 179 (47%) 6 (2%) % (16 24%) Maintenance 52 (6%) 412 (51%) 211 (26%) 109 (14%) 22 (3%) % (56 62%) Emergence 9 (2%) 43 (9%) 313 (65%) 108 (23%) 5 (1%) % (8 14%) Total 75 (5%) 516 (31%) 645 (39%) 396 (24%) 33 (2%) % (34 38%) PPV, Positive predictive value The Association of Anaesthetists of Great Britain and Ireland 807

5 De Man et al. Intra-operative monitoring and alarms Table 4 Number, value and frequency of alarms per parameter during 38 cases of moderate-risk surgery. Clinically relevant Clinically irrelevant Number of alarms Frequency; number.h 1 Duration; s High priority Low priority Irrelevant False Total Apnoea Non-invasive blood pressure Minute volume Intra-arterial blood pressure Expiratory CO 2 concentration Other ventilator Infusion pump Heart rate Oxygen saturation Premature ventricular complex Leakage from breathing circuit Airway pressure Inspiratory CO 2 concentration ECG ST segment Airway pressure upper limit exceeded Inspiratory pressure Tidal volume Central venous pressure Inspiratory O 2 concentration Constant pressure alert Respiratory rate Not assessable This study extends data from earlier studies suggesting that the majority of the alarms during anaesthetic care are clinically irrelevant, whereas the proportion of technically false alarms seems to have decreased. From the remaining clinically relevant alarms, only a small proportion indicates real patient risk [6 8, 10]. Interestingly, a recent study, also using video recordings, found an even higher alarm density when compared with our study (1.2 alarms.min 1 compared with 0.43 alarms.min 1 ). The authors studied alarms during cardiac surgery where 80% of all alarms they found were clinically irrelevant [10]. The difference in alarm density is likely to relate to the type of surgery and the amount of monitoring equipment. This is in accordance with our finding that there was a lower alarm density during surgical procedures under regional anaesthesia, where no ventilator was used. Alarm density during emergence from anaesthesia was significantly higher than during maintenance. This observation might be expected because of more haemodynamic and respiratory changes during emergence, but the PPV of alarms during this period is only 11%. We believe that this low reliability carries the risk that the anaesthetic team will adjust its behaviour based on the presumption that the alarm is false or irrelevant [5]. These critical phases of anaesthesia have recently been compared with take-off and landing during aviation [14]. As the authors clearly explain, the opportunity for serious harm of the patient increases with excessive noise and distraction during these phases. Furthermore, interruptions have been a contributory cause of aviation accidents [14]. This makes the high alarm density and low PPV of an alarm during these phases worrisome. False alarms and alarms with low clinical relevance are often cancelled without resulting in action. These alarms distract people, interfere with communication, may lead to treatment delay, can cause stress in patients and caregivers and increase the probability of a medical error [10, 12, 15 18]. Furthermore, they may lead to deliberate deactivation of audible alarm devices or setting overly wide alarm thresholds [16, 19 22]. In our study, alarms were frequently silenced, which in some cases resulted in disabled alarms during The Association of Anaesthetists of Great Britain and Ireland

6 De Man et al. Intra-operative monitoring and alarms Anaesthesia 2013, 68, a prolonged period of the anaesthetic induction. Disabling alarms may also cause a false sense of security in other caregivers unaware of the fact that the alarms have been disabled [23]. While technical features of monitoring systems are becoming more and more sophisticated [4, 13], future focus should also address context sensitivity of monitoring devices in the operating room. Adaptive alarm systems tailored to the needs of the users provide an interesting development for new warning systems. For instance, users are currently not able to silence a particular alarm without turning off all audible signals from the monitor [16]. Furthermore, our study clearly shows that the demands imposed on the alarm system change throughout the procedure. Increasing awareness and training of personnel might seem the easiest and cheapest approach to address this problem of alarms, but is less effective than tackling the problem at the origin by reengineering the devices themselves. In the current study, we focused on the alarm density and value of alarms in the daily working routine of an anaesthetist. There is therefore a large heterogeneity in the observed surgical procedures. We also included procedures under regional anaesthesia. The daily working routine of an anaesthetist consists of various procedures, and therefore, the inclusion criteria did not focus on one group of procedures. Our findings are not conclusive with respect to the relevance of specific alarms, and larger studies are warranted to resolve this. This study showed that although alarms are an essential part of peri-operative monitoring, they are frequently clinically irrelevant with a low PPV for clinical events. This low predictive value of alarms, especially during the induction and emergence from anaesthesia, may result in desensitisation of the anaesthesia team and an increased risk of ignoring a clinically relevant event. Future studies should therefore aim to improve the balance between the function of alarm systems and the predictive value, in order to improve the quality of peri-operative monitoring. Acknowledgements We want to thank all participating anaesthetists, residents and anaesthetic nurses for their cooperation with this study. This work was supported by the department of Anaesthesiology, VU University Medical Centre, Amsterdam, Netherlands, and funded by the Local production related allowance grant provided by Dutch health insurers. Competing interests No competing interests declared. References 1. Edworthy J, Hellier E. Alarms and human behaviour: implications for medical alarms. British Journal of Anaesthesia 2006; 97: Chambrin M-C, Ravaux P, Calvelo-Aros D, Jaborska A, Chopin C, Boniface B. Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis. Intensive Care Medicine 1999; 25: Lawless ST. Crying wolf: false alarms in a pediatric intensive care unit. Critical Care Medicine 1994; 22: Imhoff M, Kuhls S, Gather U, Fried R. Smart alarms from medical devices in the OR and ICU. Best Practice and Research Clinical Anaesthesiology 2009; 23: Bliss JP, Gilson RD, Deaton JE. Human probability matching behaviour in response to alarms of varying reliability. Ergonomics 1995; 38: Block FE Jr, Schaaf C. Auditory alarms during anesthesia monitoring with an integrated monitoring system. International Journal of Clinical Monitoring and Computing 1996; 13: Kestin IG, Miller BR, Lockhart CH. Auditory alarms during anesthesia monitoring. Anesthesiology 1988; 69: Seagull FJ, Sanderson PM. Anesthesia alarms in context: an observational study. Human Factors 2001; 43: Imhoff M, Kuhls S. Alarm algorithms in critical care monitoring. Anesthesia and Analgesia 2006; 102: Schmid F, Goepfert MS, Kuhnt D, et al. The wolf is crying in the operating room: patient monitor and anesthesia workstation alarming patterns during cardiac surgery. Anesthesia and Analgesia 2011; 112: G orges M, Markewitz BA, Westenskow DR. Improving alarm performance in the medical intensive care unit using delays and clinical context. Anesthesia and Analgesia 2009; 108: Siebig S, Kuhls S, Imhoff M, et al. Collection of annotated data in a clinical validation study for alarm algorithms in intensive care a methodologic framework. Journal of Critical Care 2010; 25: Borowski M, Gorges M, Fried R, Such O, Wrede C, Imhoff M. Medical device alarms. Biomedical Engineering/Biomedizinische Technik (1956) 2011; 56: Broom MA, Capek AL, Carachi P, Akeroyd MA, Hilditch G. Critical phase distractions in anaesthesia and the sterile cockpit concept. Anaesthesia 2011; 66: Edworthy J, Hellier E. Fewer but better auditory alarms will improve patient safety. Quality and Safety in Health Care 2005; 14: Block FE Jr. Optimization of alarms: a study on alarm limits, alarm sounds, and false alarms, intended to reduce annoyance. Journal of Clinical Monitoring and Computing 1999; 15: The Association of Anaesthetists of Great Britain and Ireland 809

7 De Man et al. Intra-operative monitoring and alarms 17. Hagenouw RR. Should we be alarmed by our alarms? Current opinion in Anesthesiology 2007; 20: Meredith C. Are there too many alarms in the intensive care unit? An overview of the problems. Journal of Advanced Nursing 1995; 21: Beatty PC, Beatty SF. Anaesthetists intentions to violate safety guidelines. Anaesthesia 2004; 59: McIntyre JW. Ergonomics: anaesthetists use of auditory alarms in the operating room. International Journal of Clinical Monitoring and Computing 1985; 2: Smith AF, Mort M, Goodwin D, Pope C. Making monitoring work : human-machine interaction and patient safety in anaesthesia. Anaesthesia 2003; 58: Koski EM. Clinicians opinions on alarm limits and urgency of therapeutic responses. International Journal of Clinical Monitoring and Computing 1995; 12: Raison JCA, Beaumont JO, Russell JAG, Osborn JJ, Gerbode F. Alarms in an intensive care unit: an interim compromise. Computers and Biomedical Research 1968; 1: The Association of Anaesthetists of Great Britain and Ireland