Minimal sample for training language models with Hugging Face Transformers.
This repository provides minimal sample code to train and fine-tune language models using Hugging Face Transformers and PyTorch. It includes training scripts, data preprocessing, checkpointing, and settings for distributed training (DeepSpeed/ZeRO), suitable as a lightweight starting point for LM fine-tuning or small-scale pretraining.
0