Pilot Study of Linking Web-Based Supplemental Interpretive Information to Laboratory Test Reports

Abstract

Electronic medical records have the ability to link to reference material, providing clinicians with immediate access to information relevant to patient care. Adding relevant links to laboratory test results could add value while minimizing the volume of ancillary text presented.

We provided Web-based universal resource locator (URL) links with all results of 7 laboratory tests ordered at ARUP Laboratories (Salt Lake City, UT). URL links provided were modified 7 months later, and use between initial and subsequent URLs was tracked to establish frequency and duration of access to supplemental Web information.

Monthly Web-site hit rates for individual tests varied from 0.00% to 3.00% (median, 0.12%). Rare and specialty tests averaged higher hit rates. There was no decay in hit rate 9 months after URLs were removed from test reports.

We conclude that links to reference material are accessed by clinicians. The use of Web links months after links were no longer published raises an important issue of long-term maintenance and the resources required to support these features.

Materials and Methods

The literature on physician information needs suggests that clinical questions arise frequently at the point of care. In many cases, physicians do not pursue answers owing to time pressures involved in completing the patient visit; even when they pursue answers, they are not able to find them easily.^1–4 Although physicians are encouraged to use the most authoritative evidence available, in practice, most favor convenient, predigested resources.^1,2,4 Many such references are currently available, but in most settings, these references are not tightly integrated with clinical data.

In 1 study of family physicians, 8% of questions that arose in clinical practice related to ordering and interpreting diagnostic tests.⁵ To aid clinicians in diagnostic test interpretation, laboratories often add interpretive comments in the form of footnotes on laboratory test reports. However, the unstructured textual format in which paper charts and electronic medical records (EMRs) display laboratory results severely limits the nature and quantity of interpretive information that can be added to footnotes.

It would be ideal if laboratory results could be linked to interpretive guidance in more flexible formats, such as the hypertext markup language (html) of Web pages. This could provide easy and contextual access to general interpretive information and to clinical guidelines, review articles, diagnostic algorithms, in-depth commentary on test uses and limitations, and other data. It could potentially decrease postanalytic errors, which are one of the most common sources of total-process laboratory errors.⁶ For example, a clinician may need guidance regarding the ordering of follow-up tests in response to a set of complex screening tests; a link to professional guidelines or a testing algorithm based on those guidelines would facilitate selection of the correct test. Likewise, a test report for a test used to diagnose a rare disorder could provide a link to a review article, allowing clinicians to refresh their memory about additional laboratory testing, imaging, or other interventions that should be done at the time of diagnosis. Such resources could supplement or, in some cases, replace traditional interpretive comments on laboratory tests and could also potentially reduce clinician calls to the laboratory with questions about laboratory test performance and interpretation.

Electronic links to reference material and expert guidelines are now being incorporated into computerized health records and clinical decision-making systems.^7,8 A few EMR systems provide contextual links, called “infobuttons,” to external information resources such as drug databases or laboratory test manuals. Studies of infobuttons have shown that clinicians find them useful⁸ and that information specific to a particular patient’s situation is more useful than general information.⁹ In 1 study of infobuttons, 18% of clinician queries were related to laboratory results.¹⁰ Other than in this limited set of health care information systems, we are unaware of any studies regarding the broader referencing of World Wide Web pages within laboratory results.

Results

Establishing, Linking, and Tracking Traffic to Online Reference Material

Our study had 2 primary purposes: One was to determine whether clinicians would ever access Web sites referenced on laboratory result reports. Another was to assess how long after issuance of a laboratory result a clinician might attempt to access a referenced Web resource. These results will provide important information for laboratories that are developing Web resource maintenance and quality assurance processes.

Selection of Laboratory Test Reports to Link to Online Information

All research was performed at ARUP Laboratories, a national reference laboratory owned by the University of Utah (Salt Lake City) and closely affiliated with its Department of Pathology. ARUP provides a repertoire of approximately 2,500 tests for hospitals and regional laboratories around the country. ARUP recently created ARUP Consult (http://www.arupconsult.com), an online physician’s guide to laboratory test selection and interpretation.¹¹ This Web site was launched in September 2006 and currently receives approximately 20,000 unique visitors per month. It was developed by an editorial team at ARUP Laboratories, and the content is coauthored by faculty in the Department of Pathology, University of Utah School of Medicine. All content is formally reviewed and updated at least annually. The content is organized by disease; within each disease topic, the subsections include Clinical Background, Diagnosis, Laboratory Tests, and References. Many of the topics also include downloadable flowcharts to visually describe the recommended diagnostic algorithms Image 1. Access to ARUP Consult is free of charge and does not require registration.

We first identified all the tests on the ARUP Laboratories menu for which the most common indications for ordering were discussed in ARUP Consult. Of the 20 with the highest test volume, we randomly selected 7 to study: mitochondrial M2 antibody IgG, mumps virus antibody IgG, anticardiolipin IgG, Chlamydia trachomatis amplified, serum angiotensinconverting enzyme, free prostate-specific antigen, and calculus (stone) analysis.

Frequency of monthly hit rates for all tests. The hit rate for a given test is calculated as the number of hits in a month divided by the number of ordered tests. The monthly hit rate was calculated for each of the 7 tests for 16 months (n = 112 test-months); separate charts are not provided for each individual test. URL, uniform resource locator.

Analysis

Discussion

Hit Rates for Tests

Descriptive statistics were used to determine the mean and median number of hits per month. We estimated the percentage of hits per month by dividing the number of times the URL was accessed in a month by the number of tests ordered in that month. Monthly hit rates were evaluated for each individual test. Because of the small number of months analyzed for each individual test (n = 16 months) and the high variability in individual test hit rate, we analyzed monthly hit rates for all tests combined (n = 112 months). To calculate cumulative hit rates for individual tests, we took the total number of hits to both first- and second-phase URLs in the calendar year between January 1, 2008, and December 31, 2008, divided by the total number of tests ordered in that period. Note that there is always a time lag between placing an order and receiving the results (and possibly accessing the URL). This lag varies according to test turnaround time and clinician practice, and we did not attempt to estimate it. Because the total number of tests ordered does not vary widely from week to week, small differences in hit rates have little impact on interpretation. Because URLs were not unique to patients or clinicians, we were unable to link hits with tests ordered for specific clinicians. We used linear regression to determine significance of trends in hit rate over time.

Laboratory test reports contained only the first-phase URL between September 10, 2007, and March 4, 2008. The first-phase URLs were not accessible through searches using the ARUP Consult home page or commercial search engines. Thus, all hits associated with these URLs, even those occurring after March 4, 2008, were attributable to test reports issued in the initial time window. We plotted the number of hits per month after this point and used linear regression on hits from June through December 2008 to determine if there was a trend that could indicate decay in initial URL access over time.

Cumulative hit rates for URLs for 2008 were between 0.59% (for anticardiolipin IgG) and 0.02% (for mumps virus antibody IgG) with an average hit rate of 0.24% for the year 2008 Table 1. There tended to be higher hit rates for lower volume tests and for tests that were more likely to be ordered by specialists, but the number of tests included in this study was too small to determine the significance of these correlations (Table 1).

Monthly Web hit rates for tests varied from 0.00% to 3.00% per ordered test following a Poisson distribution Figure 1 (which shows distribution for all tests combined, distributions for individual tests are not presented separately). The median monthly hit rate was 0.12%. There was substantial month-to-month variation for individual tests and total monthly hits Figure 2. The hit rate was zero in 29 (25.9%) of the 112 months in which we gathered data, and each of the 7 tests had at least 1 month with zero hits. There was a significant increase in overall hits during the last year of the study period (P = .01) (Figure 2).

There was a trend toward increasing access to second-phase URL links and combined hits to first- and second-phase URLs over time. However, the number of hits per month was extremely variable for each individual test and for the total hits for all tests (Figure 2). After the implementation of second-phase URLs on test reports, the rate at which clinicians accessed the first-phase URLs did not decline during a 9-month period. Linear regression showed no significant difference in hit rate for first-phase URL links from June to December 2008 (slope = 0.01; P = .89).

Web links provided as supplemental information on laboratory test reports were accessed by clinicians. Clinicians began accessing the Web sites almost immediately after the URLs were incorporated into test result reports. Online information provided for lower volume tests and tests ordered in specialty situations may be accessed more frequently. Hit rates were low but increased over time, even though the number of tests ordered did not change substantially from month to month. These observations are consistent with other studies that have shown linked electronic resource utilization increases over time as users become more familiar with resources.⁹

The month-to-month variability in URL hits was high. Since the average number of hits per test per month was low, it is not surprising that there were several months with no hits for individual tests. Access to some tests dramatically increased during short periods and then decreased to a relatively low baseline (Figure 2). This variation could be due to a high level of stochastic variation, or it could be indicative of patterns of clinical information use.

Trends in Access to URLs After Discontinuation of First-Phase URLs

In some cases, links were accessed many months after the initial test report was issued. We were surprised that we did not see a decrease in hit rates to initial URLs even 9 months after test reports were issued. This could be because test results are revisited over time, or it could be that the clinicians’ Web browsers stored the first-phase URLs and that they continued using these URLs to access information for new tests. We maintained all clinical information and links throughout the testing period, even after we had stopped reporting first-phase URLs on test reports, so there was no disincentive to access the first-phase URLs. Regardless of the method used to access URLs, it will be important for laboratories that provide links to Web information to have quality control for this information many months after the links have been provided. The importance of long-term quality control may become even greater as implementation of interoperability standards for EMRs increases access to laboratory results and associated URLs. The presence of bad URL links may reflect negatively on a laboratory’s reputation beyond a local scope of the laboratory’s direct clients. This may be a significant limitation to providing Web links in laboratory reports, especially when links are to external sites not maintained by the testing laboratory.

There are several limitations to our study: The number of test results that provided Web links was small; it is possible that a larger set of test results or a different set of tests and Web links would have a different use pattern. In addition, our method required clinicians to physically enter the URL into a Web browser. This caused a higher barrier to access than the ideal “clickable” URLs available in some EMRs. Hence, our hit-rate estimates may be artificially low. On the other hand, many current EMRs reformat the information on well-formatted pathology reports stripping URL links and other information from the original context,¹² so our method of URL presentation may reflect the current standard of practice. Likewise, as the Web links were provided in the results report, we assume that the ordering clinician or any other provider viewing the results was accessing the Web pages. However, owing to the anonymous nature of tracking the Web page hits, we cannot identify who was accessing the Web links. These URLs could also have been shared with other clinicians and even patients. Clinicians and laboratories sharing these URLs could cause a large number of hits from 1 or 2 test result reports. It is not difficult to think of one resident at a teaching hospital prompting a half dozen treatment team members to visit a Web site or to imagine the same individual visiting the site multiple times during the course of a few days while treating 1 patient.

A wide variety of clinical resources are currently available over the Web. Ideally, laboratories should link results only to high-quality resources.^8,13 Determining quality is not necessarily straightforward, however. For example, some Web-based resources may be inconsistently curated and/or updated. Another challenge is that some resources require subscriptions; in addition to the monetary costs, this requirement can create technical complications in implementation vs use of free public resources. Regardless of the assumed quality of the information source, it is the laboratory’s responsibility to ensure that data given or linked to by its reports is accurate and up-to-date.

Internet links to external reference material have been incorporated into clinical information systems for several years. They have been used in a variety of settings. However, this is the first report, to our knowledge, of linking from laboratory test results outside the context of any specific EMR system. Links to Web-based information are potentially a powerful tool to allow clinicians to access specific information without forcing them to sort through excessive text. These links may become more common as more laboratory results are incorporated into clinical information systems, as groups of tests are interpreted together in panels with interpretations that are not necessarily intuitive, and as the interpretation of laboratory results becomes more specialized. We have shown that, if provided, quality Web-based information will be accessed. Likewise, substantial laboratory effort and resources may be necessary to ensure quality and maintenance of Web material even months after test results are reported.

References