Challenge Open Access
Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021
Matthew Reyna , Nadi Sadr , Erick Andres Perez Alday , Chengyu Liu , Salman Seyedi , Amit Shah , Gari D. Clifford
Published: Jan. 31, 2021. Version: 1.0.1 <View latest version>
2020 and 2021 Challenges are complete (Jan. 26, 2022, midnight)
January 26, 2022: Both the 2020 Challenge and the 2021 Challenge, which extended the 2020 Challenge, are now complete. The CinC articles for both Challenges are available on the CinC website here and here. The final scores can be found here. Please cite Perez Alday EA, Gu A, J Shah A, Robichaux C, Ian Wong AK, Liu C, Liu F, Bahrami Rad A, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD* Reyna MA*. Classification of 12-lead ECGs: The PhysioNet/Computing in Cardiology Challenge 2020. Physiol. Meas. 2021 Jan 1;41(12):124003. doi: 10.1088/1361-6579/abc960 to refer to the 2020 Challenge. Please also cite Reyna MA, Sadr N, Perez Alday EA, Gu A, Shah AJ, Robichaux C, Rad AB, Elola A, Seyedi S, Ansari S, Ghanbari H, Li Q, Sharma A, Clifford GD. Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021. Computing in Cardiology 2021; 48: 1-4 and Reyna MA, Sadr N, Perez Alday EA, Gu Annie, Shah AJ, Robichaux C, Rad AB, Elola A, Seyedi S, Ansari S, Ghanbari H, Li Q, Sharma A, Clifford GD. Issues in the automated classification of multilead ECGs using heterogeneous labels and populations. Preprint. 2022 to refer to the 2021 Challenge. Finally, please also cite the standard PhysioNet citation. You can find followup articles to the 2020 Challenge and the 2021 Challenge in the Journal of Physiological Measurement Focus Issue on Classification of Multilead ECGs.
The PhysioNet/Computing in Cardiology Challenge entries are being evaluated (Oct. 25, 2021, 1 a.m.)
October 25, 2021: We are currently evaluating entries on the 2021 Challenge test data in support of the Physiological Measurement focus issue on multilead ECG classification. The deadline to submit your code and a preprint is 1 December 2021, and the deadline to submit your article is 11 January 2022. See this forum announcement for details.
More news
Winners of the PhysioNet/Computing in Cardiology Challenge 2021 announced (Sept. 20, 2021, 1 a.m.)
September 20, 2021: The winners of the 2021 Challenge were announced on 15 September 2021 at CinC in Brno, Czech Republic. Congratulations, teams! See this page for the results and the full announcement for the final steps in this year’s Challenge, including details about the focus issue (deadline: 11 January 2022).
The challenges have been renamed to the George B. Moody PhysioNet Challenge in honor of George Moody (Sept. 15, 2021, 1 a.m.)
September 15, 2021: In honor of the contributions of George Moody to PhysioNet and Computing in Cardiology, the Board of CinC voted to rename the Challenges to the George B. Moody PhysioNet Challenge.
Preparing CinC papers for the PhysioNet/Computing in Cardiology Challenge 2021 (July 21, 2021, 1 a.m.)
July 21, 2021: As you prepare your CinC papers, please follow the CinC preparation and submission instructions and use either our LaTeX (Overleaf or download) or Word templates, which include important instructions, advice, and references. Please see here for more information, including our draft paper and important citation information.
PhysioNet/Computing in Cardiology Challenge 2021 accepted abstracts (June 23, 2021, 1 a.m.)
June 23, 2021: CinC has released its abstract decisions for the Challenge track of the conference. Congratulations to those with accepted abstracts. Those without an accepted abstract can still compete for a wildcard entry as outlined here.
The official phase of the PhysioNet/Computing in Cardiology Challenge 2021 has reopened (May 1, 2021, 1 a.m.)
May 1, 2021: The official phase of the Challenge reopens today. Due to your engagement, we have enormously expanded the training data, modified the lead combination, and modified the example code and scoring function. Please see our announcement on the Challenge forum for more details. We will update and clarify these changes in response to your questions in the coming days.
CinC deadline for the PhysioNet/Computing in Cardiology Challenge 2021 extended (April 19, 2021, 1 a.m.)
April 19, 2021: CinC has extended its abstract submission deadline to April 24, 2021. Please submit your abstract if you have not done so already. Like last year, CinC will host a hybrid conference with both in-person and remote attendance. Please see our announcement on the Challenge forum for more details.
PhysioNet/Computing in Cardiology Challenge 2021 submissions are due soon (April 13, 2021, 1 a.m.)
April 13, 2021: Only two days left to submit an abstract to CinC! Please find the abstract submission announcement and the instructions announcement on the Challenge forum. Please see the leaderboard for the final scores of the unofficial phase, and please submit your abstract today!
Leaderboard for the PhysioNet/Computing in Cardiology Challenge 2021 is available (Feb. 24, 2021, midnight)
February 24, 2021: The leaderboard is now live! Please see the announcement on the Challenge forum. Please see the timing and priority of entries section here regarding the number of submissions allowed per day, so please submit early!
Accepting submissions for the PhysioNet/Computing in Cardiology Challenge 2021 (Jan. 30, 2021, midnight)
January 30, 2021: We are now accepting submissions for the 2021 Challenge! See below for details. Please register your team (even if you registered last year), check the submission instructions, and submit your code when ready. As always, please join the Challenge forum to discuss this year’s Challenge.
The PhysioNet/Computing in Cardiology Challenge 2021 is now open (Dec. 24, 2020, midnight)
December 24, 2020: The NIH-funded 2021 Challenge is now open! See below for details. Please read this website for details and share questions and comments on Challenge forum. This year’s Challenge is generously co-sponsored by Google, MathWorks, and the Gordon and Betty Moore Foundation.
Perez Alday EA, Gu A, Shah AJ, Robichaux C, Wong AI, Liu C, Liu F, Rad AB, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD, Reyna MA. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas. 2020 Nov 11. http://doi.org/10.1088/1361-6579/abc960.
Additionally, when using this resource, please cite:
(show more options)
Reyna, M., Sadr, N., Perez Alday, E. A., Liu, C., Seyedi, S., Shah, A., & Clifford, G. D. (2021). Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021 (version 1.0.1). PhysioNet. https://doi.org/10.13026/13he-1932.
Please include the standard citation for PhysioNet:
(show more options)
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220.
Abstract
The electrocardiogram (ECG) is a non-invasive representation of the electrical activity of the heart. Although the twelve-lead ECG is the standard diagnostic screening system for many cardiological issues, the limited accessibility of twelve-lead ECG devices provides a rationale for smaller, lower-cost, and easier to use devices. While single-lead ECGs are limiting, reduced-lead ECG systems hold promise, with evidence that subsets of the standard twelve leads can capture useful information and can even be comparable to twelve-lead ECGs in some limited contexts. In 2017 we challenged the public to classify AF from a single-lead ECG, and in 2020 we challenged the public to diagnose a much larger number of cardiac problems using twelve-lead recordings. However, there is limited evidence to demonstrate the utility of reduced-lead ECGs for capturing a wide range of diagnostic information.
In this year's Challenge, we ask the following question: 'Will two do?' This year's Challenge builds on last year's Challenge, which asked participants to classify cardiac abnormalities from twelve-lead ECGs. We are asking you to build an algorithm that can classify cardiac abnormalities from either twelve-lead, six-lead, three-lead, and two-lead ECGs. We will test each algorithm on databases of twelve-lead, six-lead, three-lead, and two-lead ECGs, and the differences in performances of the algorithms on these databases will reveal the utility of reduced-lead ECGs in comparison to standard twelve-lead EGCs.
Objective
The goal of the 2021 Challenge is to identify clinical diagnoses from twelve-lead, six-lead (I, II, III, aVL, aVR, and aVF), three-lead (I, II, and V2) and two-lead (II and V5) ECG recordings.
We ask participants to design and implement a working, open-source algorithm that, based only on the provided twelve-lead ECG recordings and routine demographic data, can automatically identify any cardiac abnormalities present in the recording. We will award prizes for the top performing twelve-lead algorithm and the top performing two-lead and six-lead algorithms.
For more details and updates about the objectives of this year’s Challenge, please see https://physionetchallenges.org/2021/. This webpage will be updated after the end of the Challenge.
Participation
We invite participants from academia, industry, and elsewhere to participate in the Challenge. Like previous years, the Challenge has both an unofficial phase and an official phase than run over the course of several months, culminating with Computing in Cardiology on 12-15 September 2021. For more details and updates about participating in this year’s Challenge, including important rules and deadlines, please see https://physionetchallenges.org/2021/.
Data Description
We have obtained twelve-lead ECG recordings and labels from diverse sources for the public training and hidden test sets for the Challenge. We have posted multiple databases of twelve-lead ECG recordings and labels as public training sets and sequester databases of twelve-lead, six-lead, and three-lead, and two-lead ECG recordings as private test sets. For more details about the data for this year’s Challenge, please see https://physionetchallenges.org/2021/.
Evaluation
To better capture the importance of correctly identifying cardiac abnormalities, we defined an evaluation metric to score participant algorithms by assigning different weights to different classes and classification errors. For more details about the evaluation metric for this year’s Challenge, please see https://physionetchallenges.org/2021/.
Conflicts of Interest
The authors have no conflicts of interest to declare.
Access
Access Policy:
Anyone can access the files, as long as they conform to the terms of the specified license.
License (for files):
Creative Commons Attribution 4.0 International Public License
Discovery
DOI (version 1.0.1):
https://doi.org/10.13026/13he-1932
DOI (latest version):
https://doi.org/10.13026/gt86-a263
Topics:
challenge
cardiac abnormalities
multilead ecgs
classification
competition
Project Website:
https://physionetchallenges.org/2021/
Corresponding Author
Files
Total uncompressed size: 14.6 KB.
Access the files
-
Download the files using your terminal:
wget -r -N -c -np https://physionet.org/files/challenge-2021/1.0.1/
-
Download the files using AWS command line tools:
aws s3 sync --no-sign-request s3://physionet-open/challenge-2021/1.0.1/ DESTINATION
Name | Size | Modified |
---|---|---|
LICENSE.txt (download) | 14.5 KB | 2021-01-30 |
SHA256SUMS.txt (download) | 77 B | 2021-01-31 |