Hugging Face Transformers is an open-source library that provides a unified API for downloading, using, and fine-tuning state-of-the-art pretrained models across natural language processing, computer vision, audio processing, and multimodal tasks. Since version 5 (released 26 January 2026) the library is PyTorch-only: v5 removed the TensorFlow and JAX (Flax) code paths to focus on torch, and it now ships minor releases weekly (v5.17 as of September 2026). With access to more than a million Transformers-compatible checkpoints among the roughly 3 million models on the Hugging Face Hub, Transformers has become the central distribution mechanism for the machine learning research community.

The library’s pipeline API offers a high-level interface for common tasks including text generation, sentiment analysis, named entity recognition, question answering, summarization, translation, image classification, object detection, speech recognition, and many more. For researchers and engineers who need more control, the library provides model-specific classes, tokenizers, feature extractors, and training utilities. The Trainer API and integration with the Accelerate library simplify distributed training across multiple GPUs and nodes. PEFT (Parameter-Efficient Fine-Tuning) integration enables fine-tuning large models with LoRA, QLoRA, and other adapter methods using a fraction of the memory required for full fine-tuning.

Transformers has fundamentally changed how the ML community shares and consumes models. Rather than reimplementing architectures from papers, researchers publish model weights on the Hugging Face Hub with Transformers-compatible code, enabling one-line model loading. The library is used in production at thousands of companies and is the starting point for most applied NLP and increasingly computer vision and audio projects.

Key Capabilities

  • 1M+ Transformers Checkpoints - Instant access to models for text, vision, audio, and multimodal tasks via the Hugging Face Hub
  • Unified Pipeline API - High-level interface for inference across 30+ task types with automatic model and tokenizer selection
  • PyTorch-Native - A single PyTorch backend since v5 (TensorFlow and JAX support were removed in January 2026), with integrations for vLLM, SGLang, TGI, and training frameworks such as Axolotl, Unsloth, and DeepSpeed
  • Fine-Tuning Tools - Trainer API, PEFT/LoRA integration, and Accelerate for efficient distributed fine-tuning on custom datasets

Cloud Equivalents

Hugging Face Transformers is the open-source alternative to model access in AWS Bedrock, Azure OpenAI Service, and Google Vertex AI Model Garden (Vertex AI was rebranded Gemini Enterprise Agent Platform in April 2026 - see Google Vertex AI for the full story). Cloud model services provide API-based access to curated models with managed infrastructure, while Transformers provides direct access to a vastly larger model ecosystem with full customization and fine-tuning capabilities.

Origins and History

The Transformers library was created by Thomas Wolf, Julien Chaumond, and Clement Delangue at Hugging Face, initially released in 2018 as pytorch-pretrained-bert. It was renamed to transformers in 2019 as it expanded beyond BERT to support GPT-2, XLNet, and eventually hundreds of architectures. The library is licensed under the Apache License 2.0. Hugging Face, Inc. was founded in 2016 and has grown into the central platform for open-source ML, raising over $395 million in funding by 2023.

Sources

  1. https://huggingface.co/docs/transformers
  2. https://github.com/huggingface/transformers
  3. Transformers v5 release notes, 26 January 2026. https://github.com/huggingface/transformers/releases/tag/v5.0.0
  4. Transformers v5 migration guide (“Removal of TensorFlow and Jax”). https://github.com/huggingface/transformers/blob/main/MIGRATION_GUIDE_V5.md