Back to Catalog

AI meeting companion: From voice to insight

IntermediateGuided Project

Create an app to capture audio (like lectures) and summarize it. Build the app using OpenAI Whisper (text to speech), then summarize it with an open source LLAMA 2 LLM hosted by IBM watsonx. You deploy the app in a serverless environment using IBM Cloud Code Engine.

4.6 (10 Reviews)

Language

  • English

Topic

  • Artificial Intelligence

Industries

  • Information Technology

Enrollment Count

  • 73

Skills You Will Learn

  • Python Programming, Artificial Intelligence

Offered By

  • IBMSkillsNetwork

Estimated Effort

  • 45 min

Platform

  • SkillsNetwork

Last Update

  • May 16, 2024
About This Guided Project
Imagine that you're a student who records your teacher's lectures. But, what if you need to turn that recorded lecture into text? There's a handy AI app that can do that for you, accurately. It also summarizes the lecture and highlights the main points.

In this project, you use OpenAI's Whisper to transform speech into text. Then, you use IBM watsonx AI to summarize and find key points. This stage couples with prompt engineering through PromptTemplate in Langchain. You'll make an app with HuggingFace Gradio as the user interface.

The output from the LLM not only summarizes and highlights key points, it also corrects minor mistakes made by the speech-to-text model, ensuring a coherent and accurate result.

A look at the project ahead

Following are the key objectives for your project:
  1. Speech-to-text conversion: Use OpenAI's Whisper technology to convert lecture recordings into text, accurately.
  2. Content summarization: Implement IBM watsonx AI to effectively summarize the transcribed lectures and extract key points.
  3. User interface development: Create an intuitive and user-friendly interface using HuggingFace Gradio, ensuring ease of use for students and educators.
  4. App deployment: Learn and apply the skills necessary to deploy the application online by using IBM Code Engine, making the tool accessible to a wider audience.

What you'll need

To work through this guided project, you need a general knowledge of Python a well as a browser. 

Instructors

Sina Nazeri

Data Scientist at IBM

I am grateful to have had the opportunity to work as a Research Associate, Ph.D., and IBM Data Scientist. Through my work, I have gained experience in unraveling complex data structures to extract insights and provide valuable guidance.

Read more

Contributors

Fateme Akbari

Data Scientist @IBM

I'm a data-loving Ph.D. Candidate with a passion for making a real impact. I thrive on applying science to improve the world we all share, not just for humans, but for all creatures. I've racked up years of experience working with data and I love diving into tough problems that make my brain sweat. But when I'm not crunching numbers, you'll catch me out in nature, either trekking or biking, and having a blast at ball games.

Read more

Vicky Kuo

Data Scientist

I believe that success isn't just about individual milestones, but also about uplifting and encouraging others to reach their potential. This is why I'm passionate about combining my technical background with my eagerness to help people overcome technological hurdles and accelerate growth. When I’m not on the job, I love hiking with my two dogs or relaxing in a coffee shop. There's nothing better than having an insightful conversation over coffee, or even better, some volunteer work! Please feel free to reach out to me on LinkedIn.

Read more