Back to Catalog

Build a Fashion Design Web App with DALL-E, Gradio and SAM

IntermediateGuided Project

Build an AI fashion assistant with CLIPSeg, SAM, and DALL·E 2 that lets users virtually try on, restyle, and redesign clothes through text prompts. Use CLIPSeg and SAM to detect and segment clothing items or accessories in any image, then integrate DALL·E 2 to inpaint and redesign those regions from simple text prompts. Experiment with new fabrics, styles, and colors, all generated intelligently and seamlessly. By the end, you’ll have created a shareable web app that empowers anyone to visualize and prototype fashion ideas instantly through the power of computer vision and generative AI.

Language

  • English

Topic

  • Artificial Intelligence

Skills You Will Learn

  • Computer Vision, Image Inpainting, DALL-E, Image Segmentation, PyTorch

Offered By

  • IBMSkillsNetwork

Estimated Effort

  • 45 min + 45 min

Platform

  • SkillsNetwork

Last Update

  • November 26, 2025
About this Guided Project
 Have you ever wished you could redesign your favorite outfit with just a few words: "make the sleeves longer," "turn this dress into silk," or "change the color to emerald green"?

Fashion design is a deeply creative process, but traditional editing tools demand time, precision, and technical skill. What if AI could understand your description, automatically identify the clothing item you want to change, and instantly generate your new design?  In this project, you’ll build an AI-powered fashion design assistant that combines the CLIPSeg segmentation and Segment Anything Model (SAM) with DALL·E 2 to make text-guided fashion editing effortless. Together, these models form a seamless workflow: SAM figures out what to edit, and DALL·E 2 decides how to recreate it. You’ll integrate both into a user-friendly Gradio web app, where users can upload an image, select a fashion item, describe the desired transformation, and see the result generated in seconds. Along the way, you’ll learn how to bridge computer vision and generative AI to build real-world creative tools.  

What You’ll Learn

By the end of this project, you will be able to: 
  • Apply the CLIPSeg and Segment Anything Model for automatic mask generation: Use SAM to detect and segment fashion elements from uploaded images, enabling precise and flexible edits without manual input.
  • Leverage DALL·E 2 for text-guided inpainting: Use OpenAI’s generative model to redesign masked regions according to user prompts, changing textures, colors, and styles seamlessly.
  • Build an interactive Gradio web interface: Create a shareable AI design assistant where users can visualize edits, experiment with different looks, and generate new outfit ideas from plain language.
  • Combine vision and generative AI workflows: Understand how to connect segmentation models with diffusion-based image generators for practical, creative applications.

Who Should Enroll

  • AI enthusiasts and developers interested in combining computer vision and generative models.
  • Designers, artists, or fashion students curious about how AI can accelerate and inspire the design process.
  • Students and researchers who want hands-on experience building multi-model AI systems with real-world creative use cases.

Why Enroll

This project brings the worlds of fashion, creativity, and artificial intelligence together. Instead of manually editing images or spending hours rendering new designs, you’ll learn how to build a tool that understands your intent through text, transforming design into an instant, interactive experience. By the end, you’ll have a working AI fashion design assistant, a deep understanding of SAM and DALL·E 2 integration, and practical experience deploying an AI-powered web app that turns imagination into visual reality.

What You'll Need

To get the most out of this project, you should have:
  • Basic Python programming skills.
  • Some familiarity with computer vision or generative AI concepts (helpful but not required).
  • Curiosity about how AI can revolutionize creative design workflows.
All dependencies are pre-configured in the environment, and the project runs best on the latest versions of Chrome, Edge, Firefox, or Safari.

Instructors

Jianping Ye

Data Scientist Intern at IBM

I'm Jianping Ye, currently a Data Scientist Intern at IBM and a PhD candidate at the University of Maryland. I specialize in designing AI solutions that bridge the gap between research and real-world application. With hands-on experience in developing and deploying machine learning models, I also enjoy mentoring and teaching others to unlock the full potential of AI in their work.

Read more

Contributors

Matthew Wu

Marketer at IBM

Supporting Cognitive Class and IBM through digital marketing and tailored content creation

Read more

Joseph Santarcangelo

Senior Data Scientist at IBM

Joseph has a Ph.D. in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.

Read more