The availability of annotated arrhythmia and ST databases has permitted different automated arrhythmia detection
algorithms to be tested on the same data. This recommended practice provides a protocol for a reproducible test
with realistic clinical requirements, and emphasizes record-by-record presentation of results that reflect an
algorithm's ability to detect events of clinical significance. Beat-by-beat comparisons are used to measure
performance in QRS (see 2.7), ventricular ectopic beat (VEB), and supraventricular ectopic beat (SVEB) detection.
Run-by-run comparisons are used to measure an algorithm's ability to detect consecutive VEBs and SVEBs.
Detection of ventricular flutter, atrial flutter, ventricular fibrillation, and atrial fibrillation are addressed. The evaluation of heart-rate variability measurement algorithms and ST segment measurement algorithms are also examined.
Although this document seeks to establish clinically relevant measures of performance for the comparison of
algorithms, it must be recognized that certain clinical concerns cannot be addressed within the context of this
recommended practice. Available databases do not yet contain a representative sample of nonventricular
arrhythmias, paced patients or artifacts typical of a very significant portion of ECG signals originating in the clinical setting. In addition, these databases have a limited bandwidth and should be used with caution when testing algorithms designed for full ECG diagnostic bandwidth devices. Therefore, the clinical implications of a test are necessarily limited by the size, scope, and characteristics of the databases used for testing. Performance measures derived from such testing should be regarded as uncertain indicators of performance in clinical settings.
This recommended practice has been developed for testing algorithms, not entire systems. It is not a performance
standard, but rather a set of recommendations for testing cardiac rhythm and ST measurement and reporting the
results of those tests. The intent of this recommended practice is that automated testing methods be reproducible.
This recommended practice applies to algorithms implemented in devices or systems which use automated methods to analyze the ECG.
This document applies both to human-operated, stand-alone devices which use automated methods to analyze the
recorded ECG, and to so-called real-time event recorders, which use automated methods to select abnormal events
for recording.
Testing methodologies other than beat-by beat techniques, specified rhythm analysis, and ST segment analysis are
outside the scope of this document. The evaluation of systems which rely on intensive interaction by a skilled user is also outside the scope of this document. However, if beat-by-beat evaluations are performed, the results of such testing should conform to this recommended practice.