Evaluation of Effectiveness of a Computerized Notification System for Reporting Critical Values

Abstract

Failure to adequately communicate a critical laboratory value is a potential cause of adverse events. Accreditation requirements specify that clinical laboratories must undertake assessments and appropriate measures to improve the timeliness of critical value reporting and prompt receipt by the responsible caregiver. Documentation and communication processes must be regularly monitored and implemented under ongoing systems for quality monitoring. Critical value reporting is an important phase of the clinical laboratory testing process, and notifications of results outside the target time can indicate ineffectiveness of the process. In the present study, we report data obtained in a 12-month period of critical values analysis and describe a computerized communication system conducive to improving the quality of critical value reporting at a university hospital. Automated communication improves the timeliness of notification and avoids the potential errors for which accreditation programs require read-back of the result. The communication also improves the likelihood of reaching the physician on call and may provide important decision support.

Materials and Methods

After first advocated by Lundberg¹ more than 30 years ago, there has been wide agreement on how to define a critical value when a result becomes life-threatening unless some intervention is made by a physician and for which interventions are possible. The development of a critical values policy has become a quality practice in laboratory medicine procedures, and, in the United States, the requirements for reporting critical values are specifically described in the standards of accreditation agencies such as the Joint Commission, formerly the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), and the College of American Pathologists (CAP).² Since 2005, the JCAHO has released guidelines in which the reporting of laboratory critical values has become a National Patient Safety Goal.³ At an international level, the most widely accepted standard in the medical laboratory community, ISO EN 15189:2007, includes (in clause 5.8.7) the immediate notification of a critical value as a special requisite.⁴

In October 2004, the World Health Organization (WHO) launched the World Alliance for Patient Safety to improve the safety of care and facilitate the development of patient safety policies and practices in all WHO member states. Every year, the Alliance provides a number of programs covering systemic and technical aspects to improve patient safety worldwide. Recently, the fourth of 23 potential patient safety solution topics, entitled “communicating critical test results,” was selected by the WHO International Steering Committee. It is currently under development, and will be released in the near term.⁵

The reporting of critical values is an important phase of the clinical laboratory testing process, and laboratories are responsible for detecting life-threatening results, for reporting them to health care providers, and also for tracking and improving the timeliness of reporting and the receipt of results. To minimize communication errors, US accreditation programs require that the critical value must be read back by the health care worker, who must be contacted by phone.⁶ An indicator of quality of the process may be the critical value reporting rate; the failure to report these values, estimated at 0.1% to 10%, can indicate the operational efficiency of laboratories.⁷

The sizable number of critical results, the failure to provide notification within the target time, and the time required for phone calls may be considered tools to measure the quality of critical value reporting. In the present study, we analyzed the size and type of the critical values in hospital departments and for outpatients to understand the scope and opportunities for improvement in critical value reporting. To this end, we evaluated a computerized notification system, implemented in collaboration with the information technology (IT) department to improve timely notification and the rate of successful notifications. In real time, with 2 devices, the short message service (SMS) and an alert message on the desktop computer, the automated notification of critical values can be made to clinicians. The effectiveness of the process before and after the start of the computerized alerting system is also discussed.

Results

Study Setting

The Padua Hospital, Padua, Italy, known as the Azienda Ospedaliera, is a teaching hospital and research center of national and international relevance, extending health care to more than 400,000 people in the city and surrounding areas and providing high specialty care to the entire nation. There are a number of highly specialized surgical and medical research areas, including transplantation, cardiology, hepatology, gastroenterology, immunology, pediatrics, gynecology, oncology, traumatology, and orthopedics. The hospital deals with highly complex cases and large volumes of patients; in 2007, only 54% of admitted cases came from Padua and its surrounding area, whereas 46% of inpatients came from other parts of Italy, the south in particular. The clinical laboratory includes clinical chemistry, hematology specialties, and various others, such as molecular biology, immunology, proteomics, and emergency testing.

Analysis of Critical Values

The list of critical values carried over from our laboratory and used for the present study is shown in Table 1, Table 2, and Table 3. Critical values reported between January 1 and December 31, 2007, were considered to determine type and frequency of critical values among the hospital departments and outpatients. Multiple data sets of all critically high or low laboratory values, used for the statistical analyses, were sorted on the basis of analyte, department, patient, and time. The frequency of critical values was measured as an incidence rate (ie, the number of critical values occurring in a department divided by number of hospital days in the same unit), thus allowing comparisons to be made between different hospital departments.

Since 1994, our team has reported critical values for the most important biochemical and hematologic analytes and drugs and has amended the communication policy by identifying a member of the staff responsible for this aspect, a physician on call. The notification given is reported in a register (a quality document) together with the date, the time of the call, the patient’s identification, bar code, location, test result, the physician communicating the critical value, and the recipient of the information. The percentage of unsuccessful notifications was calculated by taking into account a time lapse of 1 hour from the detection of the critical value in question.

Phone Call System

Discussion

Computerized Notification System: Alerting System and SMS

During the study period of 1 year (2007), more than 6 million routine tests and about 1.7 million emergency tests were carried out in the laboratory. The total number of critical values found was 7,320, approximately 60% of which resulted from routine testing and 40% from emergency testing. The prevalence of critical values was 7.3/100 for routine testing and 16.7/100 for emergency testing. The frequency and cumulative frequency were calculated for every critical value (Tables 1 and 2). With the exception of hemoglobin, in routine and emergency testing, 90% of critical values were generated from the same analytes. The majority of tests expected to generate critical values were performed in the clinical chemistry section, but the parameters with the highest frequency of critical values (neutrophil count and activated partial thromboplastin time) were in the hematologic section.

Of all critical values, approximately 82% were found in inpatients. Evaluation of the critical values was made by measuring the incidence rate (ie, the number of critical values occurring in a department divided by the number of hospital days in the same unit) Figure 2. The frequency of critical values was evaluated for each department; the greatest number, with the respective analyte type, is shown in Figure 3.

Critical values for outpatients account for approximately 18% of the total, and the ratio of attendances with critical values to the total number of outpatient encounters was 0.4/100. Table 3 shows the frequency and cumulative frequency of critical values for the outpatients. A cumulative frequency of 90.48% was reached with 11 analytes (4 in hematology, 6 in biochemistry, and 1 drug). The parameter with the highest frequency was the international normalized ratio (INR), and digoxin was the most frequent analyte in proportion to the test volume. Critical INR values were for 242 patients; 22 (9.1%) patients had 2 critical INR values in different attendances; 12 (5.0%) patients had a critical INR value more than twice.

Trends for the critical values were analyzed over time. The greatest number was recorded in March (n = 873) and the smallest in September (n = 448); their frequency during 24 hours considered is shown in Figure 4. The communication process was analyzed before and after the computerized notification system was initiated, and an evaluation was made to assess the improvement achieved in the process using this technology. Critical values observed during the 2-month initial period using the new notification system were compared with those observed in 2 months during which time notification was made by telephone. The time required for the communication was also calculated. The average time for notification using the telephone call system was 30 minutes, considering the overall time (looking up the phone number and dialing it, finding the responsible clinician, relaying information, and reading back). More than 50% of notifications were unsuccessful (ie, notification made after an interval of >1 hour). The average time for computerized notifications was 11 minutes, considering the overall time for receipt of the alert and confirmation. The rate of unsuccessful notifications (computerized notification without confirmation within an interval of 1 hour) was 10.9%.

The rate of successful notifications, that is, communication within 1 hour, for the traditional phone process was compared with that for the computerized system for hospital departments Figure 5.

Critical values, formerly known as panic values, are abnormal laboratory results that constitute a life-threatening condition for the patient, but any values for which delays in reporting can result in adverse outcomes for patients represent a potential critical value. Although it is widely agreed that clinicians must be immediately informed of these values, the criteria for considering test results critical are controversial.

It has also been emphasized that critical values and critical tests are not necessarily the same thing. Critical tests are especially relevant to the emergency department (ED). In this setting, communication of critical values should be made without excessive communication with the medical team, highlighting only the critical values. In the Padua Hospital for ED patients, the times to test ordering and to test results are strictly monitored, and, when critical values are detected, an alert message requiring attention appears in the report. The SMS device was not applied because clinicians are usually very busy. According to all ED physicians, we consistently provide results promptly and highlight the critical values.

Data Collection and Statistical Analysis

There are not clear recommendations about this issue, nor is there consensus in the medical community about the formulation of a standard list or target time frames for critical values. Laboratories have therefore developed their own lists, which has led to great variability in the number of tests considered critical. No standards are available for pediatric or neonatal critical values. Any attempt to establish interpretive critical values reporting for inpatients hospitalized in different diagnostic departments is beset with difficulties, as is the management of outpatient critical values.

Frequency of Critical Values

Institutions and agencies for health care have stated that critical results are not only those from laboratory tests, but also those from diagnostic tests conducted in the departments of anatomic pathology, radiology, and cardiology.⁸ The term action alert has been proposed for critical pathology findings requiring special communication. This alternative term has been considered necessary because the nomenclature has regulatory and accreditation consequences.⁹ Significant findings in diagnostic departments (including laboratories and cardiology, radiology, and pathology departments) should be communicated by using health information systems.

The analysis of critical values made in the present study allowed us to compare values obtained in one hospital setting with those reported in the literature. The critical values list adopted in our laboratory takes into account only the analytes with a “red” level of criticality, which we indicate as truly life-threatening. This list and the cutoffs used by us are similar to those used at Massachusetts General Hospital, Boston.¹⁰ Also, the percentages of inpatients and outpatients are comparable (in the present study, the ED was included in the inpatient category). By comparing our critical values for chemistry and hematology with those outlined by Howanitz et al¹¹ in a 2002 Q-Probes program, our values seem stringent.

Despite this policy, however, we have a large number of critical values to communicate each day. The system involving the use of the telephone is known to be distracting if the call back is made by the people performing the tests, and it is time-consuming. Technological solutions with wireless devices present a strategy to improve communication of critical values. In the last 3 years, the workload in our laboratory has increased significantly, calling for a better solution.

References

Analysis of Inpatient Critical Values

As an example, in the nephrology department, we found the most frequent critical value was a low calcium level. Abnormal calcium levels are common in chronic kidney diseases and not only cause significant bone disease but also contribute to cardiovascular disease. In patients with hypocalcemia, the most common diagnoses were trauma, gastrointestinal disorders, and renal failure.¹³ Often, in the latter condition, hypocalcemia can be controlled with supplemental vitamin D. In the outpatient setting, the INR showed the highest test volume. This may be because outpatients receiving oral anticoagulant therapy need to be monitored for pharmacological effects. A meta-analysis of prospective research studies by Levine and colleagues¹⁴ showed that warfarin caused major bleeding in 0.8% to 4.1% of patients per year (average, 1.7%) and fatal bleeding in 0.2% to 2.3% of patients per year (average, 0.8%). In another meta-analysis, by Landefeld and Beyth,¹⁵ the average annual frequencies of major and fatal bleeding per year with warfarin therapy were 3% and 6%, respectively. In the outpatient setting, the risk of fatal bleeding with vitamin K antagonists has been calculated at 1%¹⁶ to 2%¹⁷ per year. Among outpatients taking warfarin for whom INR values have been proven critical, 5% of cases had more than 2 critical INR values, showing poor therapeutic control. Although other studies are needed using this database, we use it as a monitoring tool for patient safety.

Analysis of Outpatient Critical Values

Failure in communication, particularly in this type of situation, continues to be one of the most common factors contributing to the occurrence of adverse events.¹⁸ Opportunities in IT are and will continue to be available. The ability to interface the LIS and, for example, the radiology information system from the hospital information system with the electronic medical record is still a major challenge. For example, a voice-driven automated system, Veriphy (Vocada, Dallas, TX), developed for radiologic reporting, has recently been made available in the United States for the management of critical results in radiology, laboratory, pathology, and cardiology testing, promoting the communication required by the Joint Commission.¹⁹

Other professional organizations also promote improvements in communication: the CAP, in the 2008 survey of critical value reporting, showed that only 8.6% of 623 institutions communicate critical values using wireless technologies.²⁰ A national survey on critical values reporting in a cohort of Italian laboratories concluded that the importance of critical values reporting was poorly recognized and that internationally accredited practices for communication were not implemented.²¹

By working together in an interdepartmental team using IT, our laboratory found it possible to revise its reporting system. For the automated process, according to one’s own institution, the clinical laboratory must define a policy identifying the clinician responsible for receiving communication of critical values. At our hospital, the responsible clinicians are those receiving the SMS, ie, the physicians on call in the department who are available at all times for urgent situations. On the clinical wards, there is only 1 cellular phone, and it is used by the responsible clinician. Mobile phones linked to the information system provide real-time event notification for the physician on call, thus meeting the Joint Commission requirement that the laboratory is responsible for ensuring that the “responsible, licensed caregiver” is contacted. To confirm an alerting message on the desktop computer, the ward identifies the authorized provider for patient care, usually the ordering clinician. In case this person is not available, a resident or the nurse in charge may be appointed to receive the message. This reporting system was started for the inpatient setting but, by working with a group of general practitioners, will be extended to the outpatient setting.

Notification of Turnaround Time and Rate of Unsuccessful Notifications

In the LIS, outpatients are recorded in association with their own family doctor. To maximize efficiency of calls, there is a large list of general practitioners and their mobile phone numbers. The family physicians for outpatients are the responsible physicians available at all times. Unfortunately, delays in communication may still occur if a general practitioner is unavailable or his or her mobile phone number is missing from the list.

There is broad consensus that errors in communicating test results are frequent and potentially dangerous, and several groups have defined guidelines and useful strategies to improve communication processes.^22,23 Alerts are known to be a crucial part of a clinical decision-support system, and their value has been demonstrated in controlled trials. Kuperman et al²⁴ observed that, when clinicians were paged about “panic” laboratory values, the time to therapy decreased by 11% and the mean time to the resolution of an abnormality was 29% shorter. In another study, Kuperman et al²⁵ showed that an automatic alerting system reduced the time until the provision of appropriate treatment in patients with critical laboratory results. In our setting, computerized communication demonstrated a reduction in time notification and yielded further benefits: it eliminated the risk of errors occurring in phone notification and erroneous patient identification and test and value reporting, which occurs if the read-back step is not used.

In the present study, it was not possible to calculate errors in communication occurring before the introduction of the computerized notification system. However, data available in literature indicate an error rate of 3.5% for all telephone calls made from laboratories.²⁶ The use of IT is, therefore, of crucial importance in reducing the communication error rate. Computerized reporting does not call for read-back from the recipient, as it ensures reliable communication and interpretation of results. In addition, the use of SMS messages ensures that physicians on call are always reached; in the near future, it should be possible to immediately communicate options available for action to clinicians.²⁷

In using improved communication strategies, however, it is important to avoid overdefinition of critical values. The list should include tests that truly meet the criteria for life-threatening, and the choice of certain analytes should be tailored to each diagnostic discipline involved and endorsed by physicians. For example, the critical value for activated partial thromboplastin time differs depending on whether a patient is receiving heparin therapy. A low platelet count can be a critical value if it is an unexpected finding, but even in a hematologic or oncologic setting, it can become a critical value because it can contribute to decision making about the type of therapy to provide and may indicate that additional support, such as platelet transfusion, is required. The clinical usefulness of a critical value hinges on a careful evaluation of the cutoff limit, in consultation with clinicians. With these issues open, IT improves the timeliness of reporting. In the modern world of computerized laboratories and hospitals, the critical value reporting process should be revisited, and methods to improve reporting and communication strategies should be realized in a hospital setting as a multidisciplinary effort. Currently, computerized reporting of critical values meets accreditation and clinician requirements, has the potential to improve patient safety, and provides context-sensitive reporting. Better communication, guaranteeing the correct notification of important results such as critical values, must be achieved because it is key to improving patient outcomes.