In the Generative Advanced Fine Tuning for LLMs course you will gain hands-on expertise in fine-tuning large language models (LLMs) and learn how to optimize AI systems for real-world applications. This course will equip you with cutting-edge skills that are highly sought after by top employers.

You will learn:

In-demand Generative AI engineering skills in fine-tuning LLMs employers are actively looking for in just 2 weeks
Instruction-tuning and reward modeling with Hugging Face, plus LLMs as policies and RLHF
Direct preference optimization (DPO) with partition function and Hugging Face and how to create an optimal solution to a DPO problem
How to use proximal policy optimization (PPO) with Hugging Face to create a scoring function and perform dataset tokenization

Course Overview

Fine-tuning your LLM is crucial for aligning it with specific business needs, enhancing accuracy, and optimizing its performance. In turn, this gives businesses precise, actionable insights that drive efficiency and innovation. This course gives aspiring Generative AI engineers valuable fine-tuning skills employers are actively seeking.

During this intermediate-level course, you’ll explore different approaches to fine-tuning and casual LLMs with human feedback and direct preference. You’ll look at LLMs as policies for probability distributions for generating responses and the concepts of instruction-tuning with Hugging Face. You’ll learn to calculate rewards using human feedback and reward modeling with Hugging Face. Plus, you’ll explore reinforcement learning from human feedback (RLHF), proximal policy optimization (PPO) and PPO Trainer, and optimal solutions for direct preference optimization (DPO) problems.

As you learn, you’ll get valuable hands-on experience in online labs where you’ll work on reward modeling, PPO, and DPO.

If you’re looking to add in-demand capabilities in fine-tuning LLMs to your resume, ENROLL TODAY and build the job-ready skills employers are looking for in just two weeks!

Course Syllabus

Module 0: Welcome

Video: Course Introduction
Specialization Overview
General Information
Learning Objectives and Syllabus
Helpful Tips for Course Completion
Grading Scheme

Module 1: Different Approaches to Fine-Tuning

Reading: Module Introduction and Learning Objectives
Video: Basics of Instruction-Tuning
Video: Instruction-Tuning with Hugging Face
Reading: Instruction Tuning
Lab: Instruction Fine-Tuning LLMs
Video: Reward Modeling: Response Evaluation
Video: Reward Model Training
Video: Reward Modeling with Hugging Face
Reading:Reward Modeling & Response Evaluation
Lab: Reward Modeling
Practice Quiz: Different Approaches to Fine-Tuning
Reading: Module Summary and Highlights
Graded Quiz: Different Approaches to Fine-Tuning

Module 2: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference

Reading: Module Introduction and Learning Objectives
Video: Large Language Models (LLMs) as Distributions
Video: From Distributions to Policies
Video: Reinforcement Learning from Human Feedback (RLHF)
Video: Proximal Policy Optimization (PPO)
Video: PPO with Hugging Face
Video: PPO Trainer
Lab: Reinforcement Learning from Human Feedback Using PPO
Video: DPO – Partition Function
Video: DPO – Optimal Solution
Video: From Optimal Policy to DPO
Video: DPO with Hugging Face
Lab: Direct Preference Optimization (DPO) Using Hugging Face
Reading: Fine-Tune LLMs Locally with InstructLab
Reading: Module Summary and Highlights
Practice Quiz: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
Graded Quiz: Fine-Tuning Causal LLMs with Human Feedback and Direct Preference
Reading: Cheat Sheet – Generative AI Advanced Fine-Tuning for LLMs
Reading: Glossary – Generative AI Advance Fine-Tuning for LLMs

Course Wrap Up

Reading: Course Conclusion
Reading: Congratulations and Next Steps
Reading: Teams and Acknowledgments
Copyright and Trademarks

Recommended Skills Prior to Taking this Course

Basic knowledge of LLMs, instruction-tuning, and reinforcement learning. Familiarity with machine learning and neural network concepts is also beneficial.

Advanced Fine-Tuning for Large Language Models (LLMs)

Language

Topic

Enrollment Count

Skills You Will Learn

Offered By

Estimated Effort

Platform

Last Update