Generative AI in Production

(1 day)

 

Traditional MLOps is a set of practices to productionize traditional ML systems for enterprise applications. Generative AI raises a new set of challenges in managing and productionizing applications at scale. The field of generative AI operations, or GenAIOps, seeks to address these new challenges.

In this course, you will learn about the different challenges that arise when productionizing generative AI-powered applications versus traditional ML. You will learn how to manage experimentation and tuning of your LLMs, then you will discuss how to deploy, test and maintain your LLM-powered applications. Finally, you will discuss best practices for logging and monitoring your LLM-powered applications in production.

Course Objectives

  • Describe the challenges in productionizing applications using generative AI
  • Manage experimentation and evaluation for LLM-powered applications
  • Productionize LLM-powered applications
  • Implement logging and monitoring for LLM-powered applications

Audience

  • Developers and machine learning engineers who wish to operationalize GenAI-based applications

Prerequisites

  • Completion of the “Application Development with LLMs on Google Cloud” or equivalent knowledge.

Course Outline

 

Module 1: Introduction to Generative AI in Production

  • Traditional MLOps vs. GenAIOps
  • Generative AI operations
  • Components of an LLM System

Module 2: Managing Experimentation

  • Datasets and Prompt Engineering
  • RAG & ReAct Architectures
  • LLM Model Evaluation (metrics and framework)
  • Tracking Experiments
  • Lab: Evaluating ROUGE-L Text Similarity Metric

Module 3: Productionizing Generative AI

  • Deployment, Packaging and Versioning (GenAIOps)
  • Testing LLM Systems (unit and integration)
  • Maintenance and Updates (operations)
  • Prompt Security and Mitigation
  • Lab: Unit Testing Generative AI Applications

Module 4: Logging and Monitoring for Production LLM Systems

  • Cloud Logging
  • Prompt Versioning, Evaluation and generalization
  • Monitoring for Evaluation-Serving Skew
  • Continuous Validation 
  • Lab: Use Model Monitoring for benchmarking, automated evaluation and training-prediction skew