InReDD-Dataset-PAN924-Ext-BoneLoss-769

@article{PhysioNet-inredd-pan924-ext-boneloss-769-1.0.0,
  author = {{Alves Vieira de Matos}, Thiago and Bauman, João Donato and Ramos, Maria Julia and Vitareli, Thais and {Uehara Martins}, Caio and Castro, Antonio and Tirapelli, Camila and {Alaniz Macedo}, Alessandra},
  title = {{InReDD-Dataset-PAN924-Ext-BoneLoss-769}},
  journal = {{PhysioNet}},
  year = {2026},
  month = sep,
  note = {Version 1.0.0},
  doi = {10.13026/s6wh-nq14},
  url = {https://doi.org/10.13026/s6wh-nq14}
}

@article{pollard_physionet_2026, title = {{PhysioNet} as a global platform for biomedical research}, volume = {1}, issn = {3005-0693}, url = {https://doi.org/10.1038/s44360-026-00096-z}, doi = {10.1038/s44360-026-00096-z}, abstract = {Twenty-five years after its first public repository was established, the PhysioNet resource for biomedical and clinical data has enabled tens of thousands of research papers. Further efforts are ongoing to advance secure data sharing, community engagement and scalable research infrastructure.}, number = {8}, journal = {Nature Health}, author = {Pollard, Tom and Moody, Benjamin E. and Lehman, Li-wei H. and Gow, Brian J. and Fernandes, Chrystinne and Xie, Chen and Johnson, Alistair and Mark, Roger G. and Heldt, Thomas}, month = aug, year = {2026}, pages = {792--795},}

Abstract

InReDD-Dataset-PAN924-Ext-BoneLoss-769 is a restricted-access extension of InReDD-Dataset-PAN924 containing 769 panoramic dental radiographs and 21,047 geometric annotations in a single COCO-compatible JSON file. It provides 18,792 tooth-crown polygons, 769 alveolar-ridge polygons, 760 mandibular reference polygons and 726 maxillary reference polygons, together with per-image sex. The annotations were produced by an expert-in-the-loop workflow in which a senior dental undergraduate delineated cementoenamel-junction and alveolar-crest references under specialist supervision, pre-existing detection and instance-segmentation models generated crown candidates geometrically from those references, two master's students in dentistry divided the manual refinement, and a specialist professor adjudicated uncertain cases and approved the final ground truth. Manual review removed 1,487 of 20,296 candidates, principally crowns on impacted or unerupted teeth, duplicate candidates, severely compromised crowns and tooth-detection failures; a later pre-release audit removed 17 further records. In a blinded re-annotation exercise covering 52 crowns in three radiographs chosen across bone-loss severity, the two annotators identified the same crowns and obtained a mean Dice similarity coefficient of 0.89 and a mean intersection over union of 0.81. Separate downstream YOLO11x-seg benchmarks reached mask mAP@.5:.95 of 70.39% for crowns and 89.63% for the alveolar ridge on a fixed patient-wise test set. The resource supports crown instance segmentation, cementoenamel junction to alveolar crest geometric analysis, and methodological research on periodontal bone-loss assessment.

Background

Panoramic radiography is a routine part of dental diagnosis and treatment planning, and periodontal bone loss is one of the findings clinicians assess on it. Measuring that loss geometrically requires two landmarks: the cementoenamel junction, where the crown meets the root, and the alveolar crest. Public dental datasets commonly annotate whole teeth or diagnostic categories, which supports tooth detection and numbering but does not provide the crown boundary and the arch-level reference lines that a bone-loss measurement needs.

The parent resource, InReDD-Dataset-PAN924, contains 924 anonymized panoramic radiographs drawn from the clinical archive of the School of Dentistry of Ribeirao Preto, University of Sao Paulo, and acquired on a single Morita Veraviewepocs unit [1]. Its cohort spans 14 to 81 years of age, with a median of 35 years and an approximate female to male distribution of 60% to 40% [1]. Its labels comprise mouth and tooth bounding boxes for every image, tooth instance masks with FDI positions for a subset, and a per-image mouth category assigned through a three-radiologist workflow in which one radiologist labeled the study, a second reviewed it independently, and a third resolved disagreements [1,2].

This extension adds the geometry that bone-loss assessment requires: an individual polygon for each tooth crown, a continuous alveolar-crest reference, and separate maxillary and mandibular cementoenamel-junction references. It covers the 769 parent radiographs that were eligible for crown annotation and completed both annotation phases described in the Methods.

Two earlier reports document the work that produced these annotations. A conference paper introduced the hybrid geometric workflow that derives crowns from expert reference lines combined with pre-existing tooth masks, together with preliminary segmentation benchmarks [3]. A later dataset descriptor documented the completed manual filtering, the named annotator roles, the agreement and sensitivity analyses, and updated patient-wise held-out benchmarks [4]. The values reported in the two papers correspond to different stages of the work and are not interchangeable; the figures given here match the released file.

The resource is intended for methodological development in crown instance segmentation and in cementoenamel junction to alveolar crest geometry. It does not by itself establish diagnostic accuracy. It originates from one clinical archive and one imaging device, so it does not support claims of cross-device or cross-institution generalization.

Methods

The annotations were produced in two phases. In Phase 1 an expert traced the cementoenamel-junction and alveolar-crest reference lines that the rest of the work depends on. In Phase 2 those references were combined with pre-existing tooth-segmentation models to generate one crown candidate per tooth, and every candidate was then reviewed by hand and approved by a specialist. This section describes the source images and their eligibility, each phase in turn, and the quality control applied to the result.

Parent images and eligibility

The radiographs are a subset of InReDD-Dataset-PAN924 v1.0.0 [1]. Before the parent release, studies were screened for diagnostic quality, and radiographs with deciduous dentition, orthodontic appliances, bone lesions, fractures, poor contrast or motion artifacts were excluded [1,2]. Those criteria shaped the 924-image parent cohort; we do not reuse them as an explanation for the radiographs absent from this extension.

Complete edentulism was not determined by any image-derived numerical metric. It was taken from the expert-assigned parent mouth category, in which Ed denotes absence of teeth in both arches, Me an edentulous maxilla, Mne an edentulous mandible, and De a dentate mouth. Radiographs labeled Ed were ineligible because they cannot contain crown instances. Single-arch edentulism did not exclude a radiograph when the opposing arch held eligible crowns and a usable reference line, so partially edentulous cases are present in the release.

The parent release contains 924 radiographs and this extension contains 769. Two conditions reduce the set: radiographs recorded as completely edentulous are ineligible, and further radiographs did not complete both annotation phases. The project did not keep an image-level disposition record assigning each of the 155 absent radiographs to one of those reasons, so we report the two conditions and no numerical breakdown.

Phase 1: expert CEJ and alveolar-crest references

The two phases used different people. In Phase 1, Joao D. Bauman, then a senior dental undergraduate, produced the reference annotations in LabelMe [5] under the clinical supervision of Camila Tirapelli, a specialist professor. Supervision meant defining the clinical protocol, training the annotator, and reviewing the resulting references.

The protocol traced a continuous reference polygon along the cementoenamel junction and alveolar crest of each arch, rather than a freehand contour around each tooth. These were then organized programmatically into the three released reference categories: max for the maxillary reference, mand for the mandibular reference, and alveolar_ridge for the arch-wide alveolar-crest reference.

A single annotator produced all three reference categories, so no inter-rater agreement statistic exists for them. We report that as a limitation rather than substitute another measurement for it.

Nine radiographs carry no mand annotation and 43 carry no max annotation. These absences track the parent mouth category: 35 of the 43 without a maxillary reference are labeled Me, 5 of the 9 without a mandibular reference are labeled Mne, and the remaining cases retain only a handful of teeth in the arch concerned. A missing reference category is a fact about annotation availability and should not on its own be read as a diagnosis of edentulism.

Phase 2a: automated crown-candidate generation

The crown candidates were generated by a two-stage pipeline of pre-existing components: a detector that localizes the dentomaxillofacial region, and an instance-segmentation model that separates individual teeth inside it. Neither was trained on, tuned to, or evaluated against the crown annotations released here, so no crown label influenced the candidate that preceded it. Instance segmentation was used rather than semantic segmentation so that teeth in contact remain distinct objects, which the per-tooth crown operation requires.

The pipeline was originally implemented in Detectron2, following earlier work by this group on dentomaxillofacial region detection and tooth instance segmentation [6,7]. That earlier work drew on a separate set of 935 panoramic radiographs from the same clinical archive and the same Morita Veraviewepocs device, with tooth instance masks manually produced by radiologists for a 605-image subset comprising 14,582 tooth polygons [6,7]. The models distributed with this project, under weights_deposit/, are that same pair of architectures, updated and retrained on torchvision [8] so that the workflow runs on a current and more readily reproducible platform. They are released together with the inference configuration so that the candidate-generation step can be re-executed and audited.

Both were fine-tuned from COCO-pretrained weights on the same source collection, using stochastic gradient descent at a learning rate of 0.005 with momentum 0.9 and weight decay 0.0005, over twelve epochs. The detector used 802 dentomaxillofacial region boxes for training and 133 for validation, reaching a bounding-box mAP@.5:.95 of 0.922; the tooth segmenter used the FDI-labeled tooth-mask subset, 104 radiographs for training with 61 for validation and 31 for testing and no radiograph shared between splits, reaching a mask mAP@.5:.95 of 0.768.

The weight files are weights_deposit/mouth_detection_fasterrcnn_resnet50fpn.pth, SHA-256 2f916cbbbb6ada0a83d07f60d535338f9a4d9234391746df2bba8f503699d87b, and weights_deposit/tooth_segmentation_maskrcnn_resnet50fpn.pth, SHA-256 769de568497ebff9cff0b80eb2c874ff5023078c8d4e7a49153c0522d45fc3a1. The operating settings above and the geometric constants below are recorded in weights_deposit/inference_config.json.

The candidates were then produced by the sequence below. No photometric preprocessing was applied to the source radiographs: brightness and contrast were left untouched, and the only geometric operation before inference was the region-of-interest crop in step 1.

1. The detector localized the dentomaxillofacial region of interest and the radiograph was cropped to that box.
2. The instance-segmentation model predicted one mask per tooth inside the crop.
3. Predicted probability masks were binarized at 0.50 and restored to the native radiograph dimensions by bilinear interpolation.
4. Contours were extracted and simplified with the Ramer-Douglas-Peucker algorithm [11], at a tolerance of 0.005 times the contour perimeter.
5. Each tooth instance was assigned to the maxillary or mandibular arch by the position of its centroid, and the crown candidate was computed as the intersection of the tooth mask with the cutting mask derived from the corresponding expert reference: crown = tooth AND cut.
6. The resulting crown polygons were scaled by a factor of 1.04 to compensate for boundary undercoverage before manual review.

This step produced 20,296 crown candidates across the 769 radiographs.

Phase 2b: manual crown refinement and specialist adjudication

Maria J. Ramos and Thais Vitareli, both master's students in dentistry, reviewed and manually refined the crown candidates. The workload over the 769 radiographs was divided between them, so each radiograph was refined by one of the two; the full set was not annotated twice. They corrected polygon boundaries and rejected candidates that did not meet the clinical protocol.

Camila Tirapelli defined the clinical criteria, trained the annotators, resolved the cases they escalated as uncertain, and approved the final ground truth. She was the sole adjudicator for this dataset, and she did not independently redraw every crown. The project kept no per-crown intervention log, so the proportion of annotations she edited or arbitrated cannot be reconstructed, and we do not state a figure that was never measured. Antonio J. R. de Castro took no part in adjudicating these annotations; his contribution was the separate perceptual evaluation described below.

Manual review removed 1,487 of the 20,296 candidates, or 7.3%, for the following reasons:

• crowns on impacted or non-fully-erupted teeth, most often third molars substantially overlapped by the mandibular ramus, which do not sit within the alveolar ridge and would distort a bone-loss measurement
• duplicate candidates covering a tooth already represented
• severely compromised or fractured crowns with no usable boundary
• tooth-detection or inference failures.

Third-molar position was not by itself a removal criterion, and erupted third molars with an intact crown were retained. An audit against the parent tooth-level clinical labels bears this out. Of the 487 teeth in these 769 radiographs recorded as impacted or still forming, 477, or 97.9%, have no released crown; of the 19,368 teeth not so recorded, only 666, or 3.4%, lack one. What separates a removed candidate from a retained one is impaction, not tooth position.

The project did not record machine-readable reason codes for individual rejections, so the 1,487 removals cannot be disaggregated by reason after the fact, and no count exists of boundaries redrawn for poor quality. We state both gaps rather than estimate around them.

Quality control and validation

The two production annotators independently re-annotated a common subset of 52 crowns in three radiographs, drawing from scratch rather than editing the automatic candidates. The three radiographs were chosen purposively to span bone-loss severity: one with few remaining teeth and severe loss, one intermediate, and one with a complete dentition and mild loss. The exercise was blinded in that each annotator worked in a separate Label Studio project and could not see the other's work. It was not a random sample, and it was not a validation by an external specialist. The annotators identified the same crowns in all three radiographs, giving 100% detection agreement, a mean Dice similarity coefficient of 0.89 with a 95% confidence interval half-width of 0.02, and a mean intersection over union of 0.81.

Because a single annotator produced the max, mand and alveolar_ridge categories, no inter-rater statistic is available for them. In its place we report how much the released crowns depend on the position of the reference line, which is the only human-variable input to the geometric crown operation. Crown pixels lying within 2, 5 and 10 native pixels of the nearest arch reference were removed, and the reduced mask was compared with the released one. Across 18,785 crowns in 768 radiographs, one radiograph being excluded because it has neither arch reference, mean Dice was 0.977, 0.965 and 0.933, and mean intersection over union was 0.955, 0.933 and 0.876, for the three displacements respectively. The analysis is one-sided: it erodes the crown and does not model the opposite, crown-extending displacement. It is an exploratory robustness check, not a measured bidirectional perturbation, and not a substitute for an inter-rater statistic.

The distributed JSON was checked programmatically. All 769 image identifiers and all 21,047 annotation identifiers are unique; no annotation refers to a missing image; every annotation carries a non-empty ring of at least three coordinate pairs, a strictly positive stored area and strictly positive bounding-box dimensions; no coordinate or bounding box falls outside its image canvas; and the image list corresponds exactly to the 769 JPEG files distributed, with no unlisted file. The same review removed 17 further records: five crown fragments under two pixels wide, confirmed on clinical inspection as erroneous residual annotations, and twelve duplicate crown records that overlapped an already-annotated crown at a bounding-box intersection over union above 0.7. The released crown count is therefore 18,792.

As a utility benchmark, and not as part of the annotation process, single-class YOLO11x-seg models were trained separately for crowns and for the alveolar ridge after the labels were finalized [12]. Neither model produced any released annotation. A fixed patient-wise test set of 77 radiographs was held out and never used for training or model selection, and whole patients were kept within a single partition throughout. On that test set, mask mAP@.5:.95 was 70.39% for crowns and 89.63% for the alveolar ridge. The runs used Ultralytics 8.4.75 with Python 3.10.20 and PyTorch 2.6.0+cu124. The full benchmark protocol is reported in the dataset descriptor [4].

In a separate exercise, Joao D. Bauman, Camila Tirapelli and Antonio J. R. de Castro independently rated the perceived accuracy of model-generated periodontal bone-loss outputs. Across 222 ordinal ratings, 95.1% placed the outputs above 70% perceived accuracy, with a Fleiss kappa of 0.32. This was a small within-group plausibility exercise on model outputs. It was not blinded external validation, it was not annotation adjudication, and it does not establish diagnostic accuracy.

Several limitations bear on how this resource should be used. The crown agreement subset is small and was purposively rather than randomly selected. No inter-rater metric exists for the three reference categories. No specialist from outside the project reviewed the annotations. All radiographs come from one clinical archive and one imaging device, so the resource does not support claims of cross-device or cross-institution generalization.

Data Description

File organization

InReDD-Dataset-PAN924-Ext-BoneLoss-769/
├── README.txt         title and overview of the package
├── images/            769 JPEG panoramic radiographs
├── weights_deposit/   crown-candidate pipeline models and inference configuration
└── labels.json        one COCO-compatible annotation file covering all images

All coordinates are expressed in the native pixel space of each radiograph, so no rescaling is required when an image is loaded at its stored resolution.

Model artifacts

weights_deposit/ contains the two models of the crown-candidate pipeline, retrained on torchvision, together with the settings under which that pipeline is applied. They are included so that the automated step of the annotation workflow can be reproduced and audited. They contain model weights only, with no patient data, stored in safetensors format.

These models were not trained on the released crown annotations. See Methods for the full application sequence.

Summary

Each radiograph carries between 4 and 36 annotations (median 30, mean 27.4) and between 1 and 33 crown instances (median 27, mean 24.4).

Sex is recorded for all 769 radiographs: 519 (67.5%) are female and 250 (32.5%) are male. This differs from the approximate 60% to 40% distribution of the parent cohort quoted in the Background, because the extension is a subset selected on crown eligibility rather than on demographics. Age is not distributed with this extension; the parent record reports the age range of the full cohort [1].

Images

Each entry of the COCO images array contains:

Image dimensions are preserved as acquired: 761 radiographs are 2903 x 1536 pixels, six are 1452 x 768, one is 2926 x 1536 and one is 2660 x 1536.

Annotations

Each entry of the COCO annotations array contains:

Crown annotations carry two additional dental-specific fields:

associated_tooth_id is a convenience mapping derived geometrically, by associating each released crown with the tooth boxes of the parent clinical labels. It was not annotated independently. It is available for 18,784 of the 18,792 crowns, or 99.96%, and is null for the remaining eight. The three reference categories do not carry either field.

Annotation identifiers are unique but not contiguous, because records removed during the pre-release audit were not renumbered. This is valid COCO and preserves traceability of identifiers across versions.

Categories

The three reference categories describe line-like anatomical boundaries, but they are stored as COCO polygons, not as a polyline type. mand and max occur at most once per image. Their absence means that the corresponding reference is not part of the release for that radiograph; see Methods for how those absences relate to single-arch edentulism in the parent labels.

Usage Notes

The annotations follow the COCO object-detection and instance-segmentation conventions [13], so the file loads directly in tooling that reads COCO. An annotation is linked to its radiograph through image_id.

The resource has been used to develop and benchmark crown and alveolar-ridge instance segmentation, and to derive tooth-level cementoenamel junction to alveolar crest measurements for periodontal bone-loss assessment [3,4].

Beyond that, the crown polygons combined with the two arch references support work on geometric landmark propagation, on the effect of reference-line placement on derived measurements, and on crown-level tasks such as numbering or condition classification when joined to the parent tooth labels [1].

A reference implementation of the annotation workflow, covering the geometric crown-isolation pipeline and the benchmark training routine, is available as open-source code [14]. The models of the crown-candidate pipeline, retrained on torchvision, ship with this project under weights_deposit/ together with their inference configuration.

FiftyOne is convenient for exploring the annotations programmatically, querying by category, computing descriptive statistics, and comparing predictions with the ground truth side by side [15]. Label Studio can be used to inspect or extend the released annotations through a graphical interface [16]. Both are suggestions for downstream use. The reference annotations themselves were created in LabelMe [5].

Known limitations

The radiographs come from a single clinical archive and a single panoramic imaging device, so models trained on this resource should be validated externally before any claim of generalization. The crown inter-annotator agreement exercise covers a small purposive subset, and no inter-rater statistic exists for the three reference categories. The annotations support geometric and methodological work; they are not a diagnostic reference standard for periodontal disease staging.

Access and license

This is a restricted-access resource. Use requires a credentialed PhysioNet account, completion of the required training, acceptance of the PhysioNet Restricted Health Data License and Data Use Agreement, and approval of a project request. The clinical images and annotations are not openly downloadable.

Release Notes

Version 1.0.0 is the initial release of the crown and bone-loss annotation extension derived from InReDD-Dataset-PAN924 v1.0.0. It contains 769 panoramic radiographs and 21,047 annotations across the four categories described in the Content Description, together with per-image sex. The release also bundles, under weights_deposit/, the two models of the crown-candidate pipeline, retrained on torchvision, with their inference configuration.

Ethics

The acquisition of the radiographs and the annotation protocol were approved by the institutional Research Ethics Committee through Plataforma Brasil, under protocol CAAE 51238021.2.0000.5419. The radiographs are routine clinical studies from the archive of the School of Dentistry of Ribeirao Preto, University of Sao Paulo, and were anonymized before any research use or distribution; no direct identifiers are present in the images or in the annotation file.

The resource is shared to support methodological research on dental image analysis and on the geometric assessment of periodontal bone loss. The foreseeable benefit is reproducible development and comparison of crown segmentation and bone-loss measurement methods on expert-referenced data. The principal risk of misuse is treating the annotations as a diagnostic reference standard, or deploying models trained on them clinically without external validation; the Usage Notes state both limits explicitly. Access is restricted under a data use agreement to keep use within the research purpose described here.

Acknowledgements

This work was supported by CNPq grant 133030/2025-3 and FAPESP grant 2024/15912-0. The authors thank the University of Sao Paulo, the Department of Computer Science, and the School of Dentistry of Ribeirao Preto for the research infrastructure that made this work possible, and the InReDD research group for access to the parent collection.

Conflicts of Interest

The authors have no conflicts of interest to declare.

References

1. Uehara Martins C, Tirapelli C, Gaeta-Araujo H, Baranauskas JA, Zancan B, Carneiro J, Macedo AA. InReDD-Dataset-PAN924 (version 1.0.0). PhysioNet. 2025. doi:10.13026/r5nt-we67
2. Costa ED, Gaeta-Araujo H, Carneiro JA, Zancan BAG, Baranauskas JA, Macedo AA, Tirapelli C. Development of a dental digital data set for research in artificial intelligence: the importance of labeling performed by radiologists. Oral Surg Oral Med Oral Pathol Oral Radiol. 2024;138(1):205-213. doi:10.1016/j.oooo.2023.12.006
3. de Matos TAV, Uehara Martins C, Bauman JD, Tirapelli C, Macedo AA. From crown delineation to CEJ-alveolar ridge assessment: a framework for multi-task panoramic radiographic segmentation. In: 2026 IEEE 39th International Symposium on Computer-Based Medical Systems (CBMS). 2026. doi:10.1109/CBMS69103.2026.00247
4. e Matos TAV, Uehara Martins C, Bauman JD, Ramos MJ, de Castro AJR, Vitareli T, Tirapelli C, Macedo AA. Descriptor: Periodontal Bone-Loss Assessment of Panoramic Radiograph Dataset (BoneLoss-PAN769). IEEE Data Descriptions. Accepted for publication, 2026. Associated dataset record: doi:10.5281/zenodo.21114193
5. Wada K. LabelMe: image polygonal annotation with Python. Computer software. Available from: https://github.com/wkentaro/labelme
6. TorchVision maintainers and contributors. TorchVision: PyTorch's computer vision library. Computer software. Available from: https://github.com/pytorch/vision
7. Carneiro JA, Zancan BAG, Tirapelli C, Capel CP, Costa ED, Gaeta-Araujo H, Baranauskas JA, Macedo AA. Transformative deep learning modular segmentation for dental panoramic radiographs. In: 2025 IEEE 38th International Symposium on Computer-Based Medical Systems (CBMS). 2025:270-275. doi:10.1109/CBMS65348.2025.00061
8. Carneiro JA. Algoritmo para segmentacao dentaria em radiografias panoramicas [Enhanced tooth segmentation algorithm for panoramic radiographs]. Master's dissertation. University of Sao Paulo; 2023. doi:10.11606/D.59.2023.tde-20022024-073306
9. Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;28:91-99.
10. He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV). 2017:2980-2988. doi:10.1109/ICCV.2017.322
11. Douglas DH, Peucker TK. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica. 1973;10(2):112-122. doi:10.3138/FM57-6770-U75U-7727
12. Jocher G, Qiu J. Ultralytics YOLO11. Computer software. 2024. Available from: https://github.com/ultralytics/ultralytics
13. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL. Microsoft COCO: common objects in context. In: Computer Vision, ECCV 2014. 2014:740-755. doi:10.1007/978-3-319-10602-1_48
14. de Matos TAV. boneloss-inference: v1.0.0, initial release accompanying BoneLoss-PAN769. Computer software. Zenodo. 2026. doi:10.5281/zenodo.20089016
15. Voxel51. FiftyOne. Computer software. Available from: https://github.com/voxel51/fiftyone
16. HumanSignal. Label Studio. Computer software. Available from: https://github.com/HumanSignal/label-studio