The results, estimates, and confidence intervals are expressed in seconds. Fig. 4 shows the mean sign-off times for the three systems. Both physician provider and non-physician provider study subjects were timed, on a report-by-report basis, to sign off each result as either critical or non-critical. The analysis of variance mixed model showed significant differences among the three systems (p-value < 0.001), and a significant (p-value < 0.001) order effect, for both physician and non-physician providers. The results across systems for the physician provider and non-physician provider, respectively, per report acknowledgement in seconds (95% confidence intervals) were for EM 1.78 (1.40–2.26) and 1.99 (1.72–2.30), for EO 2.69 (2.12–3.42) and 2.78 (2.40–3.21), and for R.A.P.I.D. 0.83 (0.70–0.98) and 1.58 (1.43–1.76). The order effect showed a significant decreasing time from the first to the third system tested.
EM’s superiority relative to EO may relate to the assignment of a yellow colour to non-critical, but abnormal, results in EM. This feature might decrease the set of results that a subject would exam- ine to partition between the critical and non-critical designations.
The improved sign-off times of the physician providers with the new display system compared to those of the non-physician providers is likely because the new display system, for the physician providers, had only a single acknowledgement button. However the significantly improved times for the non-physician providers using R.A.P.I.D. were based on identical acknowledgement protocols that differed only in the display.
The 30 non-physician providers frequently reassign with EM, and similarly with R.A.P.I.D., results as critical that the Meditech system designates as non-critical (Fig. 5A). The reassignment as critical by each non-physician provider for each of the 95 non-critical results is indicated by a ‘‘tick”. The 30 non-physician providers also reassign with EM, and again similarly with R.A.P.I. D., results as non-critical that the Meditech system designates as critical. The assignment of non-critical by each of the 30 non- physician providers for each of the 5 critical results is indicated by a ‘‘tick” (Fig. 5B). Non-physician provider reassignment of the 97 results as critical that the Orchard system designates as non- critical is indicated in Fig. 5C. Non-physician providers reassign with EO the 3 results as non-critical that the Orchard System designates as critical. The reassignment pattern for these results is very much the same with R.A.P.I.D. (Fig. 5D).
The aggregate results of non-critical/critical reassignment reveal that among the 2850 results in EM that were designated non- critical by the system, 15.2% were reassigned as critical by the 30 non-physician providers. For EO, of the 2910 results that the system designated non-critical, 18.4% were reassigned as critical. For R.A.P.I. D., of the 5760 combined results that the respective systems desig- nated as non-critical, 7.83% were reassigned as critical. These percent- age differences are significant at p < 0.0005, and the reassignment rate for R.A.P.I.D. is significantly lower than for both EM and EO. For the 150 results designated critical in EM and the 90 critical results in EO the rate of reassignment was 14.7% for EM, 5.6% for EO and 5.8% for the 240 combined reports in R.A.P.I.D. (Fig. 5E). These reassign- ment rates are significantly different at p = 0.003, with R.A.P.I.D. at a significantly lower rate than EM at p = 0.02.
The concordance (‘‘mirror image”) of reassignment by non- physician providers on a question-by-question basis was evident for both system-defined critical and non-critical results between both EM and R.A.P.I.D. as well as between EO and R.A.P.I.D. Such concordance is consistent with a highly non-random (i.e., decision- based) process.