BERT (Bidirectional Encoder Representations from Transformers) is a transformer-based language model developed by Google that pre-trains deep bidirectional representations by jointly conditioning on both left and right context in all layers.
Sources:
- Devlin et al.: BERT: Pre-training of Deep Bidirectional Transformers
- Google AI Blog: Open Sourcing BERT