Back to Catalog

Mastering Generative AI: Language Models with Transformers

Learn on

edX logo
IntermediateCourse

Build job-ready skills for language modeling in just 3 weeks. Plus, valuable practical experience and a credential. Familiarity with Python and PyTorch.

Language

  • English

Topic

  • Artificial Intelligence

Enrollment Count

  • 440

Skills You Will Learn

  • BERT, PyTorch Functions, Positional Encoding andMasking, Language Transformation, Generative Pre-trained Tranformers (GPT)

Offered By

  • IBMSkillsNetwork

Estimated Effort

  • 3 Weeks

Platform

  • edX

Last Update

  • September 4, 2025
About this Course
This Generative AI Language Modeling with Transformers course teaches you the skills you need for your AI career.  
 
You will learn: 

 Job-ready skills in 3 weeks, plus you’ll get practical experience employers look for on a resume and an industry-recognized credential 
  • Understand the concept of attention mechanisms in transformers, including their role in capturing contextual information 
  • Explore language modeling with the decoder-based GPT and encoder-based BERT 
  • Implement positional encoding, masking, attention mechanism, document classification, and create LLMs like GPT and BERT 
  • Apply transformer-based models and PyTorch functions for text classification, language translation, and modeling 
 
Course Overview 

Generative AI is a continuously evolving technology, and transformer language models are high in demand. According to Gartner, by 2026, 75% of businesses will use generative AI to create synthetic customer data. This Generative AI Language Modeling with Transformers course builds job-ready skills that will fuel your AI career in just 3 weeks.   
(Source: Gartner) 
 
In this course, you’ll get an overview of how to use transformer-based models for natural language processing (NLP). You’ll also learn to apply transformer-based models for text classification, focusing on the encoder component.  
 
You’ll learn about positional encoding, word embedding, and attention mechanisms in language transformers and their role in capturing contextual information and dependencies.  
 
Additionally, you will be introduced to multihead attention and gain insights on decoder-based language modeling with generative pre-trained transformers (GPT) for language translation, training the models, and implementing them in PyTorch.  
 
Further, you’ll explore encoder-based models with bidirectional encoder representations from transformers (BERT) and train using masked language modeling (MLM) and next sentence prediction (NSP). You will apply transformers for translation by gaining insight into the transformer architecture and performing its PyTorch implementation. 
 
Additionally, you’ll get valuable hands-on practice in online labs for attention mechanisms, positional encoding, decoder GPT-like models, and pretraining BERT models,  
 
If you’re keen to boost your resume and extend your generative AI language modeling skills for transformers, ENROLL today and build job-ready skills in 10–15 hours. 


Course Syllabus

Module 1: Fundamental Concepts of Transformer Architecture 
  • Reading: Module Introduction and Learning Objectives 
  • Video: Positional Encoding 
  • Video: Attention Mechanism 
  • Video: Self-attention Mechanism 
  • Video: From Attention to Transformers 
  • Hands-on Lab: Attention Mechanism and Positional Encoding 
  • Video: Transformers for Classification: Encoder 
  • Hands-on Lab: Applying Transformers for Classification 
  • Reading: Summary and Highlights: Fundamental Concepts of Transformer Architecture 
  • Practice Quiz: Fundamental Concepts of Transformer Architecture 
  • Graded Quiz: Fundamental Concepts of Transformer Architecture 
Module 2: Advanced Concepts of Transformer Architecture 
  • Reading: Module Introduction and Learning Objectives 
  • Video: Language Modeling with the Decoders and GPT-like Models 
  • Video: Training Decoder Models 
  • Video: Decoder Models-PyTorch Implementation-Causal LM 
  • Video: Decoder Models: PyTorch Implementation Using Training and Inference 
  • Hands-on Lab: Decoder GPT-like Models 
  • Video: Encoder Models with BERT: Pretraining Using MLM 
  • Video: Encoder Models with BERT: Pretraining Using NSP 
  • Video: Data Preparation for BERT with PyTorch 
  • Video: Pretraining BERT Models with PyTorch 
  • Hands-on Lab: Pretraining BERT Models 
  • Hands-on Lab: Data Preparation for BERT 
  • Video: Transformer Architecture for Language Translation 
  • Video: Transformer Architecture for Translation: PyTorch Implementation 
  • Lab: Transformers for Translation 
  • Reading: Summary and Highlights: Advanced Concepts of Transformer Architecture 
  • Practice Quiz : Advanced Concepts of Transformer Architecture 
  • Graded Quiz : Advanced Concepts of Transformer Architecture 
 Module 3: Course Cheat Sheet, Glossary and Wrap-up 
  • Reading: Cheat Sheet: Language Modeling with Transformers 
  • Reading: Course Glossary:  Language Modeling with Transformers

Recommended Skills Prior to Taking this Course

Basic knowledge of generative AI and working knowledge of machine learning with Python and PyTorch, and neural networks is required.  

Instructors

Joseph Santarcangelo

Senior Data Scientist at IBM

Joseph has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.

Read more

IBM Skills Network

IBM Skills Network Team

At IBM Skills Network, we know how crucial it is for businesses, professionals, and students to build hands-on, job-ready skills quickly to stay competitive. Our courses are designed by experts who work at the forefront of technological innovation. With years of experience in fields like AI, software development, cybersecurity, data science, business management, and more, our instructors bring real-world insights and practical, hands-on learning to every module. Whether you're upskilling yourself or your team, we will equip you with the practical experience and future focused technical and business knowledge you need to succeed in today’s ever-evolving world.

Read more