SEC545: GenAI and LLM Application Security

About This Course

SEC545 gives you the practical skills to design, secure, and defend GenAI applications by understanding real risks, applying proven security controls across RAG, agents, and MLOps workflows, and developing threat models and defenses that strengthen your organization's AI systems.

The course begins by introducing students to the fundamentals of GenAI, covering key concepts and terminologies such as Large Language Models (LLMs), embeddings, and Retrieval-Augmented Generation (RAG). Students examine the security risks associated with GenAI, including prompt injection attacks, malicious models, and third-party supply chain vulnerabilities, following this by diving into the essential components needed to build a GenAI application, including coverage of vector databases, LangChain, AI agents, and MCP. The course then progresses to focus on hosting options for GenAI applications, discussing options for local deployment, cloud solutions, and platforms like AWS Bedrock.

The course then shifts focus to MLOps, covering data flow orchestration, data pipelines, and operational tasks for training and deploying models. Students explore tools such as Airflow and learn how to apply security throughout the AI deployment process, including hands-on exercises in training models with AWS SageMaker, defending against model serialization attacks, securing AI deployment pipelines, and threat modeling using MAESTRO.

The class concludes with an overview of using AI for security use cases, including incident handling. Students then participate in a Capture the Flag (CTF) exercise, applying what they have learned to identify vulnerabilities and implement fixes in the AI infrastructure they built over the previous four days.