Back to Catalog

Solve Challenging Problems using Advanced Prompt Engineering

BeginnerGuided Project

Learn how Chain-of-Thought prompting, Best-of-N Sampling, and Self Verification techniques trade compute for improved LLM accuracy on logical reasoning problems. This machine learning project teaches you to unlock better AI reasoning at inference time. By the end, you'll understand how to make models think harder when it matters most.

Language

  • English

Topic

  • Artificial Intelligence

Skills You Will Learn

  • Prompt Engineering, Artificial Intelligence, LLM, Natural Language Processing, Python, Machine Learning

Offered By

  • IBMSkillsNetwork

Estimated Effort

  • 60 minutes

Platform

  • SkillsNetwork

Last Update

  • November 3, 2025
About this Guided Project
As AI models become more powerful, the question shifts from "how do we train better models?" to "how do we get more from the models we already have?" Test-time compute—the practice of spending additional computational resources during inference to improve output quality—has emerged as a critical technique for enhancing model performance without retraining. This guided project explores how strategic reasoning methods like Chain-of-Thought prompting, Best-of-N sampling, and self-verification can transform a model's ability to solve complex problems. 

What You'll Learn 

By the end of this project, you will be able to:
  • Explain the concept of Test-Time Compute and its role in improving model performance during inference without additional training or fine-tuning.
  • Apply Chain-of-Thought prompting to enable step-by-step reasoning in large language models, making their problem-solving process transparent and more reliable.
  • Use Best-of-N sampling to generate multiple candidate outputs and select the most accurate one through systematic evaluation and comparison.
  • Implement self-verification strategies to allow models to check and refine their own answers, creating feedback loops that improve consistency.
  • Analyze how increasing test-time compute affects solution accuracy, consistency, and efficiency, learning when these techniques provide meaningful improvements versus diminishing returns.

Who Should Enroll

  • ML engineers and AI practitioners who want to optimize model performance at inference time and understand the reasoning strategies behind recent AI breakthroughs.
  • Developers working with LLMs who need practical techniques to improve output quality and reliability without retraining or fine-tuning models.
  • Data scientists interested in exploring how computational budgets at inference time can be strategically allocated to enhance reasoning and problem-solving capabilities.

Why Enroll

This project gives you hands-on experience with the inference optimization techniques that are reshaping how we think about AI capabilities. Rather than accepting a model's first answer, you'll learn to implement systems that reason through problems systematically, explore alternative solutions, and verify their work. 

What You'll Need

To get the most out of this project, you should have basic Python programming skills and some familiarity with working with language model APIs. Understanding of fundamental machine learning concepts is helpful but not required. All dependencies are pre-configured in the environment, and the project runs best on current versions of Chrome, Edge, Firefox, or Safari.

Instructors

Tenzin Migmar

Data Scientist

Hi, I'm Tenzin. I'm a data scientist intern at IBM interested in applying machine learning to solve difficult problems. Prior to joining IBM, I worked as a research assistant on projects exploring perspectivism and personalization within large language models. In my free time, I enjoy recreational programming and learning to cook new recipes.

Read more

Contributors

Abdul Fatir

Data Scientist

Abdul specializes in Data Science, Machine Learning, and AI. He has deep expertise in understanding how the latest technologies work, and their applications. Feel free to contact him with questions about this project or any other AI/ML topics.

Read more

Joseph Santarcangelo

Senior Data Scientist at IBM

Joseph has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.

Read more