Resources
4 results for "pretraining"
Model
Credentialed Access
EntityBERT: BERT-based Models Pretrained on MIMIC-III with or without Entity-centric Masking Strategy for the Clinical Domain
Transformer-based neural language models have led to breakthroughs for a variety of natural language processing (NLP) tasks. However, most models are pretrained on general domain data. We propose a methodology to produce a model focused on the clini…
Model
Credentialed Access
Clinical BERT Models Trained on Pseudo Re-identified MIMIC-III Notes
This project contains the weights of 7 different BERT models trained over a surrogate re-identified set of MIMIC-III (v1.4) notes. Each model is suitable for direct compatibility with the HuggingFace framework; users can easily load and apply the mo…
Model
Credentialed Access
Clinical-T5: Large Language Models Built Using MIMIC Clinical Text
Recent advances in scaling large language models (LLMs) has resulted in significant improvements over a number of natural language processing benchmarks. There has been some work to pretrain these language models over clinical text. These works demo…
Database
Credentialed Access
MIMIC-Ext-SynNotes: Large-Scale Synthetic Clinical Notes via Rephrasing MIMIC-III/IV with LLMs
MIMIC-III and MIMIC-IV are foundational resources for clinical NLP, but their scale is modest relative to the data volumes needed to train language models. Building on recent advances in synthetic data generation through LLM-based rephrasing, we con…