Brain Segmentation Performance in Head CT from MISTIE III Trial
Muschelli, J., Srirambhatla, R., & Walborn, N. (2026). Brain Segmentation Performance in Head CT from MISTIE III Trial (version 1.0). PhysioNet. RRID:SCR_007345. https://doi.org/10.13026/a3rk-n089
Pollard, T., Moody, B. E., Lehman, L., Gow, B., Fernandes, C., Xie, C., Johnson, A., Mark, R. G., & Heldt, T. (2026). PhysioNet as a global platform for biomedical research. Nature Health. https://doi.org/10.1038/s44360-026-00096-z. Available from: https://pmc.ncbi.nlm.nih.gov/articles/PMC13641116/
Abstract
This resource supports evaluation of automated brain extraction (skull-stripping) in head CT from the MISTIE III intracerebral haemorrhage trial. It has two components. (1) Full-cohort visual QC: for 5,062 scans from 499 participants, PNG overlays (axial, coronal, sagittal) of the brain mask from each of seven methods (CTBET-UNet, 2,403 scans; see Methods), with pass/fail ratings for volumetric, registration and deep-learning failures in CSV files. (2) Manual reference set: for 10 purposively selected examinations from 10 participants, defaced CT volumes, expert full-brain masks, and the masks from the seven automated methods. The 10-scan set was created because other manual datasets used in the accompanying evaluation had been used to train some algorithms. It supports reference-based validation and illustrative benchmarking. It is not for training or for estimating performance in the full MISTIE III population. NIfTI volumes and masks are released for the 10 scans only.
Background
Accurate separation of brain from skull, scalp, and other non-brain tissue is an important preprocessing step for quantitative CT analysis, image registration, and image-based machine learning. Brain extraction in acute haemorrhagic stroke can be difficult because haemorrhage, mass effect, surgical changes, image artifacts, angiographic contrast, variable fields of view, and differences in CT reconstruction can violate assumptions used by general-purpose algorithms. In particular, partial-volume effects can blur the boundary between bone and brain, and scans that extend into the neck or shoulders can cause failures in methods that expect a more restricted head field of view [@muschelli2015].
The 10 reference scans were selected from the larger Minimally Invasive Surgery plus rt-PA for Intracerebral Hemorrhage Evacuation (MISTIE III) phase 3 trial, which enrolled adults with spontaneous, non-traumatic, supratentorial intracerebral haemorrhage of at least 30 mL [@hanley2019]. They were manually segmented by a stroke-imaging reader to provide a human reference for assessing automated methods on clinically challenging CT data. We share the resource to enable transparent, repeatable comparisons of brain-extraction approaches and to help researchers identify cases in which automated masks may be unsuitable for downstream analyses. The released CTs were defaced before release to reduce the risk of facial re-identification.
Methods
Source data and selection
The release has two components. Full-cohort QC (5,062 scans, 499 participants): PNG overlays of each method’s brain mask and pass/fail ratings in CSV files. Manual reference set (10 scans, 10 participants): defaced CT volumes, expert whole-brain masks, face masks, and automated masks for the same scans. The 10 scans are a subset of the 5,062. CT volumes and masks for the other 5,052 scans are not distributed, so the PNGs are the only record of those masks.
The source MISTIE III imaging cohort comprised 5,062 CT scans from 499 participants acquired at 78 clinical sites. CT was obtained on systems from GE, Siemens, Philips, Toshiba, NeuroLogica, MPTronic, Hitachi, and Mobius; across the full cohort, tube voltage ranged from 80 to 140 kVp, pixel spacing from 0.24 to 0.98 mm, and slice thickness from 0.41 to 10.0 mm. These cohort-level values describe the parent study, not necessarily this selected subset. Scanner model, reconstruction kernel, tube settings, contrast dose, and acquisition time for each released examination are not distributed with this resource.
Analyses on the full MISTIE III cohort. All seven methods were run on the tilt-corrected NIfTI volumes, except CTBET-UNet, which was run only on the 2,403 scans that meet its input requirements (512x512 image). For every method-scan pair we generated a PNG overlay showing the axial, coronal, and sagittal planes, and every overlay was manually scored.
Analyses on manually segmented reference scans. The resource contains one head CT examination from each of 10 MISTIE III participants. The examinations were purposively selected to illustrate brain-extraction failure modes, including post-craniotomy imaging, scans that include the neck, CT angiography (CTA), and strong motion artifacts. The accompanying quality-control tables identify 3 artifact cases, 4 post-craniotomy cases, and 2 CTA cases; characteristics may overlap within an examination.
The original two-dimensional DICOM series were converted to three-dimensional NIfTI volumes with dcm2niix, which corrects gantry tilt and accommodates variable slice thickness [@li2016]. Only tilt-corrected images were used. The released source images are compressed NIfTI volumes (.nii.gz) in scanner anatomical coordinates. In this subset, in-plane matrices are 512 by 512 or 512 by 587–606 voxels, with 30–74 axial slices; in-plane voxel sizes range from 0.395 to 0.586 mm, and slice spacing ranges from 3.0 to 5.0 mm. Source CT volumes are stored in nifti_anonymized/.
CTBET-UNet requires in-plane slices of exactly 512 × 512 pixels and orthogonal orientation. Gantry-tilt correction with dcm2niix enlarges the volume and violates this requirement, so CTBET-UNet was run only on the scans with no gantry tilt. The remaining tilt-corrected scans were not processed, so no masks or PNGs exist for them. These results are not directly comparable to those of the other six methods unless subset to the same scans.
De-identification
Before release, facial anatomy was removed from each CT volume. Face masks used in this process are provided in face_mask/. Automated brain-extraction algorithms were run on the original CT images before defacing; they were not re-run on the released defaced volumes. Consequently, a small discrepancy between an automated mask and the released CT boundary may occur near the removed facial region.
Manual reference segmentation
A reader with experience segmenting thousands of scans manually delineated the full brain for each examination. The resulting reference masks are in brain_mask_manual/, with one NIfTI file per CT and matching filenames across directories. The brain label is 1; background is 0, except that one file also contains a -2000 fill value outside the image region. The release does not preserve the annotation software, detailed labeling protocol, timing of annotation, or an inter-rater/repeat-read assessment; users should not interpret the masks as a quantified measure of inter-rater reliability.
Automated masks and quality assessment
For each examination, binary brain masks were generated using CTBET, Robust-CTBET, HD-CTBET, Brainchop/MindGrab, SynthStrip, CTBET-Docker, and CTBET-UNet. The methods were selected because they were publicly available, usable from a command line without retraining, and described as applicable to head CT. CTBET-Docker was executed from its underlying code rather than its Docker container. These outputs are supplied in the corresponding brain_mask_* directories.
Each automated mask was visually assessed with a PNG orthographic display showing axial, coronal, and sagittal views using a custom Python quality-control interface. For each method and examination, the rater assigned independent binary pass/fail assessment for volumetric consistency, registration accuracy, and deep-learning stability. Volumetric failures included gyrus-sized internal voids, cortical bone retention, or extraneous non-brain voxels; registration failures included misalignment, incomplete cranial-vault coverage, or non-brain soft-tissue inclusion; and deep-learning failures included severe fragmentation, incomplete prediction, or near-zero masks. Assessments were not blinded to method identity. Per-method records are available in csv/; they document whether a mask was marked for exclusion from volumetric, registration, or deep-learning analyses. These visual assessments are quality-control annotations, not replacement manual segmentations.
Automated masks and CT volumes for the other 5,052 scans are not distributed. The 10 manual-reference scans are a subset of the 5,062 and can be linked to their PNGs and CSV rows by filename.
Intended use
The manual masks may be used as reference labels for evaluating overlap, boundary agreement, volume error, and robustness of CT brain-extraction pipelines. Because this is a 10-examination subset with heterogeneous imaging challenges, performance estimates should be reported with the case-level results and should not be generalized to all MISTIE III participants or to broader clinical populations without additional validation. The PNGs are to reference how the overall cohort was assessed via visual inspection for use and the CSVs show the scoring done, which can be used to replicate the values from the results from our analysis.
Data Description
The following folders contain gzipped NIfTI images (extension .nii.gz).
nifti_anonymizedbrain_mask_brainchopbrain_mask_ctbet_unetbrain_mask_ctbetdockerbrain_mask_hdctbetbrain_mask_original_ctbetbrain_mask_robust_ctbetbrain_mask_synthstripbrain_mask_manualface_mask
Those with the prefix brain_mask are binary masks of the brain. The folder face_mask is a binary mask of the face that was removed from the original CT scans and the output scans are in nifti_anonymized.
The images used or assessment are included as zip folders of PNGs in pngs/. The folder csv has a CSV of each method's quality assessment of successful or failure for specific tasks (registration, volumetrics, or deep learning). The scan_list.csv file list the files and relevant scanning information, the artifact.csv list the scans with artifacts, craniotomy.csv indicate scans with a craniotomy, cta.csv for CTA (CT angiograms), and exclusions_combined.csv combine the artifact/cta/craniotomy to one file. The csv/success_rates.R shows how to calculate success rates for each method we compared and the type of failures we saw.
Usage Notes
The manual masks can serve as an independent, case-level reference set for evaluating CT skull-stripping or full-brain segmentation algorithms, especially on scans with haemorrhage, craniotomy, motion artifact, CTA contrast, or extended neck coverage. Useful analyses include spatial-overlap and surface-distance metrics, intracranial-volume agreement, error visualization, and stress testing of preprocessing pipelines before use in a larger cohort. The matched automated masks and task-specific QC annotations also support reproducible method comparisons, error analysis, and development of automated QC or failure-detection models.
Limitations and precautions
The manual reference set is purposively selected 10 scans from 10 participants, not a random or representative sample of the MISTIE III cohort. It is enriched for challenging acquisition and clinical features, so aggregate performance estimates should be accompanied by case-level results and should not be generalized to healthy populations, other diseases, or the full MISTIE III population. The cohort consists of patients with spontaneous intracerebral haemorrhage; algorithm performance may differ for other indications or acquisition protocols.
All source images were defaced before release, whereas the automated masks were generated from the original, non-defaced CT images. Minor mismatch near the removed facial region is therefore possible. The release has a single manual segmentation per case and does not include an inter-rater or repeat-read assessment, annotation software, or a detailed labeling protocol. One manual-mask file contains a -2000 fill value outside the image region; analyses should use label value 1 as brain and treat all other values as non-brain. Finally, the QC ratings are unblinded, binary task-specific assessments and should not be interpreted as continuous measures of segmentation quality.
All visual QC was performed by a single, unblinded rater, and neither inter-rater nor intra-rater agreement was quantified. Rater-specific thresholds, such as what counts as a “gyrus-sized” void or “moderate” bone retention, may have influenced the failure rates.
Full-cohort annotations are single rater, unblinded and binary.
Responsible use and licensing
These images and derived masks must not be used in any attempt to identify or re-identify participants, including by linking them with other data sources or by evaluating facial-recognition or identity-inference methods. Although the CT volumes have been defaced, defacing does not eliminate all privacy risk. Users must follow the terms governing access to the resource, applicable institutional approvals, and all relevant privacy and data-protection requirements.
The resource should not be used to develop, validate, deploy, or support commercial methods without first discussing the proposed use with the data stewards and confirming that the applicable license, consent, and data-use permissions allow it. The distributed files do not themselves grant commercial-use rights.
Release Notes
Version 1.0.0
This is the initial release of this data set and contains 10 CT scans from a larger cohort of scans taken from the minimally invasive surgery with thrombolysis in intracerebral haemorrhage evacuation (MISTIE III) stroke clinical trial. These scans have hand segmentations of the brain region by an expert in stroke imaging.
Automatic brain extraction/skull stripping was done using a number of methods:
- CTBET [@muschelli2015, @ctbet]
- Robust CTBET [@ctskullstrip]
- HD-CTBET [@hdctbet]
- Brainchop/Mindgrab [@brainchop, @masoud2023]
- SynthStrip [@hoopes2022, @synthstrip]
- CTBET-Docker [@ctbetb]
- CTBET-UNet [@ctbetc, @akkus2020]
Each method resulted in a binary brain mask. This brain mask was visually assessed using a PNG of an orthographic view (axial, coronal, and sagittal views) to assess the quality of brain extraction.
These CT scans were defaced prior to release. The original CT scan (with the face on) was used for each of the methods. We did not re-run the brain extraction from these methods using the de-faced images, so results may vary slightly from the masks given here.
Please see the brain extraction benchmark repository for more information [@brainextract].
Ethics
Re-identification of patients is strictly prohibited.
Data collected from human subjects
Each hospital obtained local institutional review board or ethics committee approval. Eligible patients were aged 18 years or older with a spontaneous, non-traumatic, supratentorial intracerebral haemorrhage of 30 mL or more due to cerebral small-vessel disease, with a Glasgow Coma Scale (GCS) score of 14 or less or National Institutes of Health Stroke Scale (NIHSS) score of 6 or higher, an mRS score of 0 or 1 before the bleed, and an intracerebral haemorrhage that remained the same size (growth <5 mL) for at least 6 h after diagnostic CT. We did not enrol patients with expressed care limitations or those deemed to have life-threatening mass effect requiring surgery. The full list of inclusion and exclusion criteria for the MISTIE III trial is in the appendix [@hanley2019].
Local principal investigators determined eligibility and stability of haematoma growth and written, informed consent was then obtained from all patients or their legal representatives. The study protocol [@hanley2015] and clinical trial data [@nationallibrarymedicine2019] are available online.
Acknowledgements
We thank the MISTIE III trial investigators and participants for providing the dataset used in this analysis. We acknowledge the contribution of all clinical sites and research coordinators involved in data collection.
Conflicts of Interest
The authors were involved in the MISTIE III trial and received grant money from the NIH to run the trial and the rtPA was donated by the vendor. There are no additional conflicts of interest to release this data.
References
@muschelli2015Muschelli J, Ullman NL, Mould WA, Vespa P, Hanley DF, Crainiceanu CM. Validated automatic brain extraction of head CT images. Neuroimage. 2015 Jul 1;114:379-85.@hanley2019Hanley DF, Thompson RE, Rosenblum M, Yenokyan G, Lane K, McBee N, Mayo SW, Bistran-Hall AJ, Gandhi D, Mould WA, Ullman N. Efficacy and safety of minimally invasive surgery with thrombolysis in intracerebral haemorrhage evacuation (MISTIE III): a randomised, controlled, open-label, blinded endpoint phase 3 trial. The Lancet. 2019 Mar 9;393(10175):1021-32.@li2016Li X, Morgan PS, Ashburner J, Smith J, Rorden C. The first step for neuroimaging data analysis: DICOM to NIfTI conversion. Journal of neuroscience methods. 2016 May 1;264:47-56.@ctbetMuschelli, J. (n.d.). CT_BET: Paper on brain extraction for CT imaging [Computer software]. GitHub. https://github.com/muschellij2/CT_BET@ctskullstripMuschelli, J. (n.d.). CT_Skull_Strip_robust.R [Source code]. ichseg. GitHub. https://github.com/muschellij2/ichseg/blob/master/R/CT_Skull_Strip_robust.R@hdctbetCAAI. (n.d.). HD-CTBET [Computer software]. GitHub. https://github.com/CAAI/HD-CTBET@brainchopMasoud, M., & Plis, S. (2024). Brainchop: In-browser 3D MRI rendering and segmentation [Computer software]. GitHub. https://github.com/neuroneural/brainchop@masoud2023Masoud M, Hu F, Plis S. Brainchop: In-browser MRI volumetric segmentation and rendering. Journal of Open Source Software. 2023 Mar 28;8(83):5098.@hoopes2022Hoopes A, Mora JS, Dalca AV, Fischl B, Hoffmann M. SynthStrip: skull-stripping for any brain image. NeuroImage. 2022 Oct 15;260:119474.@synthstripHoopes, A., Mora, J. S., Dalca, A. V., Fischl, B., & Hoffmann, M. (n.d.). SynthStrip: Skull-stripping for any brain image. FreeSurfer. https://surfer.nmr.mgh.harvard.edu/docs/synthstrip/@ctbetbLLG Neuroresearch. (n.d.). CTbet-Docker [Computer software]. GitHub. https://github.com/llgneuroresearch/CTbet-Docker@ctbetcAkkus, Z. (n.d.). CT_BET: Robust brain extraction tool for CT head images [Computer software]. GitHub. https://github.com/aqqush/CT_BET@akkus2020Akkus Z, Kostandy P, Philbrick KA, Erickson BJ. Robust brain extraction tool for CT head images. Neurocomputing. 2020 Jun 7;392:189-95.@brainextractSrirambhatla, R., Walborn, N., Hanley, D., & Muschelli, J. (n.d.). Benchmarking automated CT brain extraction methods across 5,000 scans [Computer software]. GitHub. https://github.com/rsriram2/brain-extraction-benchmark@hanley2015Hanley, D. F., Awad, I. A., & Zuccarello, M. (2015). MISTIE III protocol: Statistical analysis plan. http://braininjuryoutcomes.com/images/MISTIE3/MISTIE_III_Protocol_SAP.pdf@nationallibrarymedicine2019National Library of Medicine. (2019). Minimally invasive surgery plus rt-PA for intracerebral hemorrhage evacuation (MISTIE III) (NCT01827046). ClinicalTrials.gov. https://clinicaltrials.gov/study/NCT01827046
Files
- sign the data use agreement for the project