Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information

@article{PhysioNet-b2ai-voice-4.0.0,
  author = {Bensoussan, Yael and Sigaras, Alexandros and Rameau, Anais and Elemento, Olivier and Powell, Maria and Dorr, David and Payne, Philip and Ravitsky, Vardit and Bélisle-Pipon, Jean-Christophe and Bahr, Ruth and Watts, Stephanie and Bolser, Donald and Siu, Jennifer and Lerner-Ellis, Jordan and Rudzicz, Frank and Boyer, Micah and Abdel-Aty, Yassmeen and {Ahmed Syed}, Toufeeq and Anibal, James and Aradi, Stephen and Armosh, Kirollos and Martinez, Ana Sophia and Awan, Shaheen and Bedrick, Steven and Beltran, Helena and Bernier, Alexander and Berrios, Moroni and Bevers, Isaac and Blatter, Alden and Brito, Rahul and Brown, Amy and Brown, Johnathan and Casalino, Selina and Costello, John and Dalal, Abhijeet and {De Santiago}, Iris and Diaz-Ocampo, Enrique and Doerksen, Emily and Ebraheem, Mohamed and Eiseman, Ellie and Elmahdy, Mahmoud and Evangelista, Emily and Fletcher, Kenneth and Gallois, Hortense and Garrett, Gaelyn and Gelbard, Alexander and Ghaffar, Omar and Ghavanini, Amer and Goldenberg, Anna and Hanna, Karim and Hersh, William and Jain, Jennifer and Jayachandran, Lochana and Jenney, Kaley and Jenkins, Kathy and Jenkins, Phillip and Jo, Stacy and Johnson, Alistair and Kalia, Ayush and Kalia, Megha and Khawa, Zoha and Kobayashi, Kenji and Kostelnik, Cindy and Krause, Alisa and Krussel, Andrea and Lapadula, Elisa and Leo, Genelle and Levinsky, Justin and Loewith, Chloe and Mah, Linda and Maharaj, Vrishni and Maramraju, Sudhiksha and Miao, Siyu and Michaels, LeAnn and Mifsud, Matthew and Mikhael, Marian and Moothedan, Elijah and Nafii, Yosef and Neal, Tempestt and Newberry, Karlee and Ng, Evan and Nickel, Christopher and Peltier, Amanda and Persaud, Jacqueline and Pharr, Trevor and Pontell, Matthew and Potter, Jaiden and Premi-Bortolotto, Claire and Rafatjou, Parnaz and Rahman, JM and Rajkumar, Gayathiri and Rajkumari, Starry and Ramos, John and {de Riesthal}, Michael and Rohde, Sarah and Rossi, Jillian and Russell, Laurie and {Salvi Cruz}, Samantha and Samuel, Joyce and Shah, Suketu and Shawkat, Ahmed and Silberholz, Elizabeth and Stark, John and Su, Lala and Sudhakar, Shrramana Ganesh and Sutherland, Duncan and {Swarna Mukhi}, Venkata and Tang, Jeffrey and Taylor, Luka and Toghranegar, Jamie and Tu, Julie and Urbano, Megan and Vinson, Kimberly and Wilke, Jordan and Wilson, Claire and Zanin, Madeleine and Zeng, Xijie and Zesiewicz, Theresa and Zhao, Robin and Zisimopoulos, Pantelis and Ghosh, Satrajit},
  title = {{Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information}},
  journal = {{PhysioNet}},
  year = {2026},
  month = oct,
  note = {Version 4.0.0},
  doi = {10.13026/hh2a-kn84},
  url = {https://doi.org/10.13026/hh2a-kn84}
}

@article{pollard_physionet_2026, title = {{PhysioNet} as a global platform for biomedical research}, volume = {1}, issn = {3005-0693}, url = {https://doi.org/10.1038/s44360-026-00096-z}, doi = {10.1038/s44360-026-00096-z}, abstract = {Twenty-five years after its first public repository was established, the PhysioNet resource for biomedical and clinical data has enabled tens of thousands of research papers. Further efforts are ongoing to advance secure data sharing, community engagement and scalable research infrastructure.}, number = {8}, journal = {Nature Health}, author = {Pollard, Tom and Moody, Benjamin E. and Lehman, Li-wei H. and Gow, Brian J. and Fernandes, Chrystinne and Xie, Chen and Johnson, Alistair and Mark, Roger G. and Heldt, Thomas}, month = aug, year = {2026}, pages = {792--795},}

Abstract

The human voice contains complex acoustic markers which have been linked to important health conditions including dementia, mood disorders, and cancer. When viewed as a biomarker, voice is a promising characteristic to measure as it is simple to collect, cost-effective, and has broad clinical utility. Recent advances in artificial intelligence have provided techniques to extract previously unknown prognostically useful information from dense data elements such as images. The Bridge2AI-Voice project seeks to create an ethically sourced flagship dataset to enable future research in artificial intelligence and support critical insights into the use of voice as a biomarker of health. Here we present Bridge2AI-Voice, a comprehensive collection of data derived from voice recordings with corresponding clinical information.

Bridge2AI-Voice v4.0.0 contains derived audio features for 48,226 recordings (189.7 hours of audio) from 1,370 participants at five sites in the United States and Canada. Participants were selected based on known conditions which manifest within the voice waveform including voice disorders, neurological disorders, mood disorders, and respiratory disorders. The release contains data considered low risk, including derivations such as spectrograms but not the original voice recordings. Detailed demographic, clinical, and validated questionnaire data are also made available.

Background

The production of human voice involves the complex interaction among respiration, phonation, resonation, and articulation. The respiratory system provides the air flow and pressure to initiate and maintain vocal fold vibration. The vocal folds generate the sound source which is then modified within the vocal tract by the oral and nasal cavities and the articulators involved in speech production. Each of these processes is influenced by the speaker’s ability to adjust and shape these interacting systems.

Although many use the terms voice and speech interchangeably, it is important to understand the distinction between the different terms used to describe human sounds:

• Voice: In the voice research field, refers to sound production and is the phonatory aspect of speech. In other words, it is the sound produced by the larynx and the resonators. For example, voice can be assessed by asking someone to do a prolonged vowel sound like /e/.
• Speech: Speech is the result of the voice being modified by the articulators and is produced with intonation and prosody. For example, a patient having a stroke can have abnormal speech production due to difficulty with articulating words but have a normal voice. For this project, the term Voice as a Biomarker of Health will include speech in its definition.

For voice to emerge as a biomarker of health, there is a pressing need for large, high quality, multi-institutional and diverse voice database linked to other health biomarkers from various data of different modality (demographics, imaging, genomics, risk factors, etc.) to fuel voice AI research and answer tangible clinical questions. Such an endeavor is only achievable through multi-institutional collaborations between voice experts and AI engineers, supported by bioethicists and social scientists to ensure the creation of ethically sourced voice databases representing our populations.

Based on the existing literature and ongoing research in different fields of voice research, our group identified 5 disease cohort categories for which voice changes have been associated to specific diseases with well-recognized unmet needs. These categories were:

1. Voice Disorders: Laryngeal disorders are the most studied pathologies linked to vocal changes. Benign and malignant lesions can affect the shape, mass, density, and tension of the vocal folds resulting in changes in vibratory function resulting in changes in phonation.
2. Neurological and Neurodegenerative Disorders: Voice and speech are altered in many neurological and neurodegenerative conditions. Acute strokes can present with slurred speech (Dysarthria) or expressive deficits speech (Aphasia). Voice and speech changes can be the presenting symptoms of many neurodegenerative conditions, such as Parkinson’s and ALS with changes such as slowed, low frequency, monotonous speech as well as vocal tremor.
3. Mood and Psychiatric Disorders: Changes in voice have been linked to depression, and other mood disorders. Individuals with depression have been found to have decreased fundamental frequency (f0) as well as a monotonous speech, while individuals with anxiety disorders have a significant increase in F0. Regrettably, much of the literature examining the intersection of voice and speech changes in psychiatric conditions have used small datasets with limited demographic diversity reporting, lack of standardized data collection protocol precluding meta-analysis and possible confounders, all limiting external validity and clinical usability.
4. Respiratory disorders: Respiratory sounds, including breath, cough and voice have long been used for diagnostic purposes. For instance, pediatric croup can be suspected based on the presence of barking cough, stridor and dysphonia. With advances in acoustic recording and analysis in the second half on the twentieth century, increasing interest has emerged in the use of respiratory sounds for disease screening and therapeutic monitoring, especially with cough sounds.
5. Pediatric Voice and Speech Disorders: The literature is sparser in terms of pediatric voice and speech analysis partly due to ethical concerns and challenges in data acquisition for this cohort. However, many studies have investigated the use of machine learning models for voice and speech analysis for detection of Autism and Speech Delays in the pediatric population.

The protocols used for data collection in this study have been extensively described [1].

Methods

Patients presenting at specialty clinics and institutions were considered for enrollment. Patients were selected based on membership to five predetermined groups (Respiratory disorders, Voice disorders, Neurological disorders, Mood disorders, Pediatric); the pediatric group is released as a separate dataset. Patients presenting at the given clinic were screened for inclusion and exclusion criteria prior to their visit by the project investigators. If eligible for enrollment, patient consent was sought for the data collection initiative and to share the acquired research data. Once consented, a standardized protocol for data collection was adopted. This protocol involved the collection of demographic information, health questionnaires, targeted questionnaires inquiring about known confounders for voice, disease specific information, and voice recording tasks such as sustained phonation of a vowel sound. Data collection was conducted using a custom application on a tablet with a headset used for data collection when possible. For most participants a single session was sufficient to collect all relevant data. However, a subset of participants required multiple sessions to complete the data collection. As a result, there may be more than one session per participant in the current dataset. 263 participants in this release were enrolled as controls (is_control_participant in enrollment/eligibility.tsv). Data were exported from REDCap [2], organized into a BIDS-like folder structure with tab-delimited phenotype tables, and de-identified and bundled using b2aiprep, an open-source library developed by our team [3]. Acoustic features were computed with senselab [4].

Raw audio was preprocessed by converting to monaural and resampling to 16 kHz, then low-pass filtered (zero-phase Butterworth filter, 7.9 kHz cutoff). From this standardized audio, we extracted the following derived data:

• Spectrograms: time-frequency representations computed using the short-time Fourier transform with a 25 ms window, 10 ms hop length and a 400-point FFT. Spectrograms were further down-sampled by a factor of two in the time domain after derivation.
• Mel spectrograms: computed with the same parameters (25 ms window, 10 ms hop length, 400-point FFT) and 60 Mel bins, then down-sampled in time in the same way.
• Mel-frequency cepstral coefficients (MFCC): 60 MFCCs computed from the Mel spectrograms, then down-sampled in time in the same way.
• Articulatory features: using the Speech Articulatory Coding (SPARC) package [5], we generated the kinematic traces of vocal tract articulators and source features as well as measures of loudness, periodicity, and pitch, at 50 frames per second.
• Acoustic features extracted using openSMILE [6], capturing temporal dynamics and acoustic characteristics.
• Phonetic and prosodic features computed using Praat and Parselmouth [7, 8], providing measures of fundamental frequency, formants, and voice quality.
• Phonetic posteriorgrams (PPGs): a time-varying categorical distribution over acoustic units of speech (e.g., phonemes) at 100 frames per second, generated with the ppgs package [9].
• Transcriptions: generated with OpenAI's Whisper large-v3-turbo model [10].

Each recording was also reviewed by an automated pipeline [4] that locates the requested task within the recording (the task extent), measures audio quality, transcribes the recording with two speech recognizers (CrisperWhisper [11] and Qwen3-ASR [12]) and aligns their output word by word into a consensus transcript, and screens the transcript for personally identifying speech. To improve data quality, recordings the pipeline marked for review or discard were excluded from this release, as were recordings excluded by earlier manual review. The review's measurements of the released recordings, and the consensus transcripts of the recordings with content features, are provided (see Data Description).

Participants are identified by a six-digit pseudonym that is the same across versions of this dataset. Sessions are numbered per participant. The following de-identification steps were taken in the process of preparing the dataset:

• HIPAA Safe Harbor identifiers were removed. While not all relevant to this dataset, these identifiers include: names, geographic locators, date information (at resolution finer than years), phone/fax numbers, email addresses, IP addresses, Social Security Numbers, medical record numbers, health plan beneficiary numbers, device identifiers, license numbers, account numbers, vehicle identifiers, website URLs, full face photos, biometric identifiers, and any unique identifiers.
• Voice-derived features are released, but highly identifying features such as speaker embeddings (vectors that characterize a speaker's voice) are not; the recording review reports only similarity scores between recordings, not the embeddings themselves. To make speaker identification more difficult, the time axis of the spectrogram, Mel spectrogram and MFCC features was also down-sampled by keeping every second frame.
• No calendar dates or clock times are released; timing is given as intervals, such as days since the last anxious, depressive or manic episode or since a traumatic event, or seconds since the first session, or as the local hour of day. Ages of 90 and above are reported as "90 and above".
• Geographic information is limited to country and state or province of residence. Collection sites are given as pseudonyms (site_A to site_E).
• Free-text answers were released only after review, with identifying details replaced by [REDACTED].
• Some answer options chosen by very few participants were combined with a broader option.
• Recordings of free-speech and open-response tasks need further review before features that capture what was said can be shared (content features: spectrograms, Mel spectrograms, MFCCs, PPGs, articulatory traces and transcripts). For these recordings, this release provides pitch, loudness, periodicity, the static acoustic features and the audio-quality measures; content features are not provided at this time.

Data Description

The dataset contains derived audio features and recording-review measures (under features) and phenotypic information acquired during data collection (under phenotype), including per-recording metadata.

Features

Binary files are made available as Parquet, an open-source column-oriented data file format. Each of the feature parquet files is formatted similarly. Each row contains a unique identifier for the participant (participant_id), the session number within that participant (session_id; sessions are numbered 01, 02, … for each participant, so a session is identified by participant_id and session_id together), the task performed (task_name), the number of time frames for that feature (n_frames), and the tensor data for the feature.

• torchaudio_spectrogram.parquet (n=44,691) contains log-power spectrograms (dB, floored at 80 dB below each recording's maximum) of dimension 201xT generated with the short-time Fourier transform using a 25 ms window, 10 ms hop length and a 400-point FFT. Every second frame was then dropped, giving an effective hop of 20 ms (50 frames per second).
• torchaudio_mel_spectrogram.parquet (n=44,691) contains Mel spectrograms (power) of dimension 60xT generated with a 25 ms window, 10 ms hop length, a 400-point FFT and 60 Mel bins. Every second frame was then dropped, giving an effective hop of 20 ms (50 frames per second).
• torchaudio_mfcc.parquet (n=44,691) contains 60 Mel-frequency cepstral coefficients of dimension 60xT computed from Mel spectrograms with a 25 ms window, 10 ms hop length, a 400-point FFT and 60 Mel bins. Every second frame was then dropped, giving an effective hop of 20 ms (50 frames per second).
• torchaudio_pitch.parquet (n=48,226) contains the detected pitch (fundamental frequency) over time, of dimension T at 100 frames per second, with a minimum and maximum pitch of 80 and 500 Hz.
• sparc_ema.parquet (n=44,682) contains the estimated electromagnetic articulography (EMA) from a deep learning model [5], of dimension Tx12 at 50 frames per second, where the 12 channels are the X/Y positions of six articulators: tongue dorsum (TD), tongue body (TB), tongue tip (TT), lower incisor (LI), upper lip (UL) and lower lip (LL).
• sparc_loudness.parquet (n=48,217) contains the estimated loudness, the average absolute amplitude of the z-scored audio waveform in 20 ms windows, of dimension T at 50 frames per second.
• sparc_periodicity.parquet (n=48,217) contains the estimated periodicity (confidence of pitch presence), of dimension T at 50 frames per second.
• sparc_pitch.parquet (n=48,217) contains the fundamental frequency (F0) estimated with a different algorithm (CREPE [13]), with a range of 50–550 Hz, of dimension T at 50 frames per second.
• ppgs.parquet (n=44,691) contains phonetic posteriorgram [9] probabilities across 40 categories (39 phonemes and silence), of dimension 40xT at 100 frames per second.

The dataset covers 48,226 recordings, totalling 189.7 hours, of which an estimated 161.6 hours fall within the task extent. Pitch, loudness, periodicity, the static features and the audio-quality files cover all of them. Content features (spectrograms, Mel spectrograms, MFCCs, PPGs, EMA and transcripts) are provided only for recordings outside the free-speech and open-response tasks (164.2 hours, 138.7 hours within the task extent). Additionally, some files, whether due to length or other issues, could not generate certain features and so are not included in those feature files. As a result, the number of rows differs between files.

The features folder also contains the following files:

• static_features.tsv (n=48,226, one row per recording) contains features derived from openSMILE [6], Praat [7], parselmouth [8] and torchaudio [14, 15], and a transcription column with an automatic transcript of the recording made with Whisper large-v3-turbo [10]. The transcription is empty for free-speech and open-response recordings.
• audio_quality_metrics.tsv (n=48,226, one row per recording) contains five audio quality measures per recording (proportion clipped, proportion silent, peak SNR, spectral-gating SNR and amplitude-modulation depth), each with a pass/fail check.
• audio_quality_annotations.tsv (n=48,226, one row per recording) contains the measurements of the automated recording review: the estimated task extent, level, noise, signal-to-noise and speech-quality estimates, clipping and dropouts, speech-recognition agreement, speaker-consistency scores, and the review pipeline's version. The task extent (task_start_s to task_end_s, with task_duration_s and the fraction of the recording it covers) is the part of the recording the review estimated to hold the requested task. It is provided as an aid for analysis, for example to trim silence and other sound outside the task, but because it is estimated automatically it may leave out part of the task where the alignment is wrong.
• consensus_transcripts.parquet (705,619 rows, one per word, from 43,330 recordings) contains the consensus transcripts of the recordings with content features. Each row gives the word, its start and end time in the recording, whether it is a non-speech tag such as [UH] or [COUGH], how the two recognizers' words aligned (agreement, variant or insertion), and each recognizer's own word and timing. Transcripts cover the whole recording; task_start_s and task_end_s in audio_quality_annotations.tsv can be used to keep only the words within the task. Recordings in which the recognizers found no words have no rows. The rows of each recording are consecutive and in consensus order (word_index 0, 1, 2, …); in 876 recordings (2.0%) this order does not always follow the word start times, so sort by start_s where time order matters.

audio_quality_annotations.tsv and consensus_transcripts.parquet carry recording_id, which also appears in phenotype/task/recording.tsv.

Phenotype

The phenotype subfolder contains organized information collected from the participant or other individual during their encounter:

.
├── confounders
│   ├── confounders.json
│   └── confounders.tsv
├── demographics
│   ├── demographics.json
│   └── demographics.tsv
├── diagnosis
│   ├── airway_stenosis.json
│   ├── airway_stenosis.tsv
│   ├── amyotrophic_lateral_sclerosis.json
│   ├── amyotrophic_lateral_sclerosis.tsv
│   ├── anxiety.json
│   ├── anxiety.tsv
│   ├── ataxia.json
│   ├── ataxia.tsv
│   ├── benign_lesions.json
│   ├── benign_lesions.tsv
│   ├── bipolar_disorder.json
│   ├── bipolar_disorder.tsv
│   ├── cognitive_impairment.json
│   ├── cognitive_impairment.tsv
│   ├── control.json
│   ├── control.tsv
│   ├── copd_and_asthma.json
│   ├── copd_and_asthma.tsv
│   ├── depression.json
│   ├── depression.tsv
│   ├── essential_tremor.json
│   ├── essential_tremor.tsv
│   ├── glottic_insufficiency.json
│   ├── glottic_insufficiency.tsv
│   ├── huntingtons_disease.json
│   ├── huntingtons_disease.tsv
│   ├── laryngeal_cancer.json
│   ├── laryngeal_cancer.tsv
│   ├── laryngeal_dystonia.json
│   ├── laryngeal_dystonia.tsv
│   ├── laryngitis.json
│   ├── laryngitis.tsv
│   ├── muscle_tension_dysphonia.json
│   ├── muscle_tension_dysphonia.tsv
│   ├── parkinsons_disease.json
│   ├── parkinsons_disease.tsv
│   ├── precancerous_lesions.json
│   ├── precancerous_lesions.tsv
│   ├── unexplained_chronic_cough.json
│   ├── unexplained_chronic_cough.tsv
│   ├── unilateral_vocal_fold_paralysis.json
│   └── unilateral_vocal_fold_paralysis.tsv
├── enrollment
│   ├── eligibility.json
│   ├── eligibility.tsv
│   ├── enrollment_form.json
│   ├── enrollment_form.tsv
│   ├── participant.json
│   └── participant.tsv
├── questionnaire
│   ├── adhd_adult.json
│   ├── adhd_adult.tsv
│   ├── custom_affect_scale.json
│   ├── custom_affect_scale.tsv
│   ├── dsm5_adult.json
│   ├── dsm5_adult.tsv
│   ├── dyspnea_index.json
│   ├── dyspnea_index.tsv
│   ├── gad7_anxiety.json
│   ├── gad7_anxiety.tsv
│   ├── leicester_cough_questionnaire.json
│   ├── leicester_cough_questionnaire.tsv
│   ├── panas.json
│   ├── panas.tsv
│   ├── phq9.json
│   ├── phq9.tsv
│   ├── psychiatric_history.json
│   ├── psychiatric_history.tsv
│   ├── ptsd_adult.json
│   ├── ptsd_adult.tsv
│   ├── vhi10.json
│   ├── vhi10.tsv
│   ├── voice_perception.json
│   └── voice_perception.tsv
└── task
    ├── acoustic_task.json
    ├── acoustic_task.tsv
    ├── harvard_sentences.json
    ├── harvard_sentences.tsv
    ├── productive_vocabulary.json
    ├── productive_vocabulary.tsv
    ├── random_item_generation.json
    ├── random_item_generation.tsv
    ├── recording.json
    ├── recording.tsv
    ├── session.json
    ├── session.tsv
    ├── stroop.json
    ├── stroop.tsv
    ├── winograd.json
    └── winograd.tsv

The enrollment folder holds participant (site, whether any data was collected remotely, selected language), eligibility and enrollment-form tables; diagnosis holds one table per diagnosis form, including the clinician-confirmed (gold-standard) diagnosis; questionnaire holds the validated questionnaires; and task holds sessions, acoustic tasks, recordings and the answers recorded during some speech tasks. The recording.tsv file holds per-recording metadata, including the task, instructions, stimulus, language and audio format.

Phenotype data files only contain rows for a participant if at least one column is not missing. Note that as participants may have repeated visits in order to collect data, there may be more than one row per participant in the data files. Furthermore, there is no requirement the participant provide the same response for each visit. As a result, participant information for the same data element may vary across the data file.

data_explorer.html, at the top level of the dataset, summarizes every phenotype field: its description, how often it is blank, and the distribution of its values. It opens in any web browser.

Data dictionaries

Every data file has a data dictionary with the same file stem and a JSON suffix (e.g. torchaudio_spectrogram.json). For the tab-separated feature files and consensus_transcripts, keys are column names and values give a description (and units or answer options where they apply). For the feature parquet files, the dictionary lists each field with its type and description, and gives the tensor's shape, units, frame rate, extraction parameters and software version. For phenotype tables, the dictionary gives a description of the table, naming its source form, and for each column its description, the question as worded on the form, its value type and its response options (choices).

The code used to preprocess the raw audio waveforms into the feature files and to merge the source data into the phenotype files has been made open source in the b2aiprep library [3].

Usage Notes

If using Python, the parquet dataset can be loaded in with the HuggingFace datasets library as follows:

from datasets import Dataset
ds = Dataset.from_parquet("features/torchaudio_spectrogram.parquet")

Loading a file this way writes a cached copy of it to the HuggingFace cache directory; for the spectrogram file this needs about 12 GB of disk space.

The Mel spectrogram is stored as power and can be plotted in decibels after conversion. The spectrogram in torchaudio_spectrogram.parquet is already in decibels and can be plotted without conversion.

from datasets import Dataset
import numpy as np
import librosa
import matplotlib.pyplot as plt

ds = Dataset.from_parquet("features/torchaudio_mel_spectrogram.parquet")
mel_db = librosa.power_to_db(np.asarray(ds[0]["mel_spectrogram"]))
plt.figure(figsize=(10, 4))
plt.imshow(mel_db, aspect="auto", origin="lower")
plt.title("Mel spectrogram")
plt.xlabel("Frame (20 ms)")
plt.ylabel("Mel bin")
plt.colorbar(label="dB")
plt.show()

A phenotype file can be loaded with any statistical analysis tool. Participant and session identifiers contain leading zeros, so read them as text; otherwise they are converted to numbers and no longer match the identifiers in the feature files. In the phenotype tables the session column is named session_id or ends in _session_id (for example demographics_session_id or recording_session_id); diagnosis and enrollment tables have no session column. For example, with the pandas library in Python:

import pandas as pd

demographics = pd.read_csv(
    "phenotype/demographics/demographics.tsv",
    sep="\t",
    dtype={"participant_id": str, "demographics_session_id": str},
)

Each feature row matches one row of recording.tsv on participant, session and task, so per-recording metadata can be joined to the features:

static = pd.read_csv(
    "features/static_features.tsv", sep="\t", dtype={"participant_id": str, "session_id": str}
)
recordings = pd.read_csv(
    "phenotype/task/recording.tsv", sep="\t", dtype={"participant_id": str, "recording_session_id": str}
)
merged = static.merge(
    recordings,
    left_on=["participant_id", "session_id", "task_name"],
    right_on=["participant_id", "recording_session_id", "task_name"],
)

The recording-review measures and consensus transcripts carry recording_id, so they join to each other and to recording.tsv on it. For example, to keep only the transcript words within each recording's task:

annotations = pd.read_csv(
    "features/audio_quality_annotations.tsv", sep="\t", dtype={"participant_id": str, "session_id": str}
)
words = pd.read_parquet("features/consensus_transcripts.parquet")
words = words.merge(annotations[["recording_id", "task_start_s", "task_end_s"]], on="recording_id")
in_task = words[(words["start_s"] >= words["task_start_s"]) & (words["end_s"] <= words["task_end_s"])]

Release Notes

b2ai-voice v4.0.0: A major update with data for 556 new participants, for 1,370 in total. New diagnosis tables for ataxia, essential tremor and Huntington's disease were added, task descriptions were updated, and every recording went through a new automated review of task content, audio quality and identifying speech. This version also changes identifiers, file layout and some column names, so code written for v3.1 needs updating:

• Participants: 1,370 in total: 814 from v3.1, with the same participant_id, and 556 new. 19 v3.1 participants are not in this version, because of changes to their eligibility, consent or data-sharing status, or because none of their recordings passed the new quality review. Ages are now whole numbers (v3.1 wrote them with one decimal place).
• Recordings: 48,226 (v3.1: 32,535). To improve data quality, recordings flagged by the new automated recording review were excluded, including 5,752 recordings that were released in v3.1.
• Spanish-language data: 23 participants completed the protocol in Spanish (783 recordings). recording.tsv now gives each recording's language, and its instructions and stimulus in that language.
• New feature files: features/audio_quality_annotations.tsv (recording-review measures, including the estimated task extent) and features/consensus_transcripts.parquet (word-level consensus transcripts from two speech recognizers).
• New phenotype tables: diagnosis/ataxia.tsv, diagnosis/essential_tremor.tsv and diagnosis/huntingtons_disease.tsv.
• Data explorer: data_explorer.html, at the top level, summarizes every phenotype field in a web browser.
• Free-text answers: 51 fields that needed review before release, mostly free text such as "other" specifications and clinician comments, are released for the first time. Each value was reviewed, and identifying details were replaced by [REDACTED]. More such fields may be released in a later version, pending ethics review.
• Controlled-access fields: some fields, for example medication and substance-use details, are available only in the controlled-access release (see the link at the top of this page).
• Sessions: session_id is now the session number within the participant (01, 02, …) instead of an 8-character identifier, so sessions can no longer be matched to v3.1 by session_id. session.tsv adds session_index (the session's position, equal to the number in session_id), session_local_hour and session_seconds_since_first_session, and no longer has session_is_control_participant. Whether a participant was enrolled as a control is recorded by is_control_participant in enrollment/eligibility.tsv.
• Recording metadata: the metadata folder (metadata.parquet) was removed. Per-recording metadata is now in phenotype/task/recording.tsv, which grows from 11 to 31 columns and adds the task name, language, instructions, stimulus, speech type, recording order and local hour, audio sample rate and channel count, and device and browser information. recording_id is unchanged, so v3.1 recordings can be matched to v4.0 by joining the two versions' recording.tsv files on recording_id. The v3.1 recording.tsv has no task_name; its recording_name gives the task name once lower-cased, with spaces as hyphens and parentheses removed.
• Task names: the task name of 11 recordings changed: rainbowpassage → rainbow-passage (4), high-to-low → glides-high-to-low (3), and 4 maximum-phonation-time-v2 recordings were renumbered (-v2-3, -v2-4 and -v2-5 → -v2-2).
• Phenotype layout: productive_vocabulary moved from questionnaire to task. Phenotype data dictionaries now have the form {table: {description, data_elements: {column: …}}}, adding a description of each table.
• Renamed columns: CAPE-V ratings in the voice diagnosis tables now end in _severity and _consistency instead of _1 and _2 (for example diagnosis_bl_degree_b_severity). In unilateral_vocal_fold_paralysis, only the _2 ratings were renamed, to diagnosis_uvfp_degree_*_consistency, along with diagnosis_degree_comments_other → diagnosis_uvfp_degree_comments_other; the severity ratings keep their v3.1 names (diagnosis_degree_b, diagnosis_degree_l, …). diagnois_bl_gold_standard_diagnosis → diagnosis_bl_gold_standard_diagnosis. In control.tsv, diagnosis_c_ac → clinical_global_impression_voice_affecting_condition. In confounders.tsv, average_number_of_uses_each_day → nicotine_uses_per_day_lifetime_average, number_of_uses_per_day_now → nicotine_uses_per_day_current and currently_pregnant → pregnancy_trimester.
• New columns: site and remote-collection flag in participant.tsv; state or province, Canadian household income and transportation in demographics.tsv; smoking and alcohol detail in confounders.tsv; the type, interview and questionnaire details of the depression and bipolar gold-standard diagnoses; days since the last anxious, depressive or manic episode, and since a traumatic event; and self-reported essential tremor and ataxia in eligibility.tsv.
• Features: all features were recomputed for this release, including for recordings released in v3.1, so values for the same recording can differ slightly from v3.1. Feature dictionaries now give units and frame rates, and the PPG dictionary cites the correct reference.

b2ai-voice v3.1.0: Minor update with no new participants, but some additional data released for certain participants. Fixed broken parquet files from previous release. Adds files related to audio quality metrics and releases per audio metadata. Updated validated diagnosis information across conditions with back-filled information. Some phenotypic files were rearranged and certain gold standard variable names were renamed.

b2ai-voice v3.0.0: A major update with new data for an additional 391 participants. The single phenotype data file has been separated into more user-friendly and intuitive individual files. Additional features were provided from the Speech Articulatory Coding (sparc) package as well as Phonetic Posteriorgrams from the ppgs package. The files have been reorganized.

b2ai-voice v2.0.1: Corrections in the authorship list.

b2ai-voice v2.0: This release provides data for an additional 136 new participants. Spectrograms were reprocessed to fix some minor issues identified in the previous release. All spectrograms and Mel-frequency cepstral coefficients from free speech related files have been removed.

b2ai-voice v1.1: This release added Mel-frequency cepstral coefficients (MFCCs).

b2ai-voice v1.0: This was the first release of the Bridge2AI voice as a biomarker of health dataset [16].

Ethics

Data collection and sharing was approved by the University of South Florida Institutional Review Board.

Acknowledgements

This release would not be possible without the graceful contribution of data from all the participants of the study.

This project was funded by NIH project number 3OT2OD032720-01S1: Bridge2AI: Voice as a Biomarker of Health - Building an ethically sourced, bioaccoustic database to understand disease like never before. We would also like to thank the NIH for their continued support of the project.

Conflicts of Interest

None to declare.

References

1. Rameau, A., Ghosh, S., Sigaras, A., Elemento, O., Belisle-Pipon, J.-C., Ravitsky, V., Powell, M., Johnson, A., Dorr, D., Payne, P., Boyer, M., Watts, S., Bahr, R., Rudzicz, F., Lerner-Ellis, J., Awan, S., Bolser, D., & Bensoussan, Y. (2024). Developing multi-disorder voice protocols: A team science approach involving clinical expertise, bioethics, standards, and DEI. Proc. Interspeech 2024, 1445–1449. https://doi.org/10.21437/Interspeech.2024-1926
2. Bensoussan, Y., Ghosh, S. S., Rameau, A., Boyer, M., Bahr, R., Watts, S., Rudzicz, F., Bolser, D., Lerner-Ellis, J., Awan, S., Powell, M. E., Belisle-Pipon, J.-C., Ravitsky, V., Johnson, A., Zisimopoulos, P., Tang, J., Sigaras, A., Elemento, O., Dorr, D., … Bridge2AI-Voice. (2025). Bridge2AI Voice REDCap (v3.23.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.14989503
3. Johnson, A., Bevers, I., Ng, E., Wilke, J., Brito, R., Bedrick, S., Catania, F., & Ghosh, S. (2026). Bridge2AI Data Processing Library (Version 4.0.0) [Computer software]. https://github.com/sensein/b2aiprep
4. Catania, F., Wilke, J., & Ghosh, S. (2026). senselab (Version 1.4.0-alpha.0) [Computer software]. https://github.com/sensein/senselab
5. Cho, C. J., Wu, P., Prabhune, T. S., Agarwal, D., & Anumanchipalli, G. K. (2024). Coding speech through vocal tract kinematics. IEEE Journal of Selected Topics in Signal Processing, 18(8), 1427–1440. https://doi.org/10.1109/JSTSP.2024.3497655
6. Eyben, F., Wöllmer, M., & Schuller, B. (2010). openSMILE: The Munich versatile and fast open-source audio feature extractor. Proceedings of the 18th ACM International Conference on Multimedia (MM '10), Florence, Italy, 1459–1462. https://doi.org/10.1145/1873951.1874246
7. Boersma, P., & van Heuven, V. (2001). Speak and unSpeak with Praat. Glot International, 5(9/10), 341–347.
8. Jadoul, Y., Thompson, B., & de Boer, B. (2018). Introducing Parselmouth: A Python interface to Praat. Journal of Phonetics, 71, 1–15. https://doi.org/10.1016/j.wocn.2018.07.001
9. Churchwell, C., Morrison, M., & Pardo, B. (2024). High-fidelity neural phonetic posteriorgrams. 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 823–827. https://doi.org/10.1109/ICASSPW62465.2024.10669905
10. Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 28492–28518. https://proceedings.mlr.press/v202/radford23a.html
11. Zusag, M., Wagner, L., & Thallinger, B. (2024). CrisperWhisper: Accurate timestamps on verbatim speech transcriptions. Proc. Interspeech 2024, 1265–1269. https://doi.org/10.21437/Interspeech.2024-731
12. Shi, X., Wang, X., Guo, Z., Wang, Y., Zhang, P., Zhang, X., Guo, Z., Hao, H., Xi, Y., Yang, B., Xu, J., Zhou, J., & Lin, J. (2026). Qwen3-ASR technical report. arXiv:2601.21337.
13. Kim, J. W., Salamon, J., Li, P., & Bello, J. P. (2018). CREPE: A convolutional representation for pitch estimation. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 161–165. https://doi.org/10.1109/ICASSP.2018.8461329
14. Hwang, J., Hira, M., Chen, C., Zhang, X., Ni, Z., Sun, G., Ma, P., Huang, R., Pratap, V., Zhang, Y., Kumar, A., Yu, C.-Y., Zhu, C., Liu, C., Kahn, J., Ravanelli, M., Sun, P., Watanabe, S., Shi, Y., Tao, Y., Scheibler, R., Cornell, S., Kim, S., & Petridis, S. (2023). TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch. 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 1–9. https://doi.org/10.1109/ASRU57964.2023.10389648
15. Yang, Y.-Y., Hira, M., Ni, Z., Chourdia, A., Astafurov, A., Chen, C., Yeh, C.-F., Puhrsch, C., Pollack, D., Genzel, D., Greenberg, D., Yang, E. Z., Lian, J., Mahadeokar, J., Hwang, J., Chen, J., Goldsborough, P., Roy, P., Narenthiran, S., Watanabe, S., Chintala, S., Quenneville-Bélair, V., & Shi, Y. (2022). TorchAudio: Building blocks for audio and speech processing. 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6982–6986. https://doi.org/10.1109/ICASSP43922.2022.9747236
16. Johnson, A., Bélisle-Pipon, J.-C., Dorr, D., Ghosh, S., Payne, P., Powell, M., Rameau, A., Ravitsky, V., Sigaras, A., Elemento, O., & Bensoussan, Y. (2024). Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information (version 1.0). Health Data Nexus. https://doi.org/10.57764/qb6h-em84