Velocity: AI Agent Observability, Reliability and Production Hardening
About This Course
This 2-day course focuses on the operational discipline required to run AI agent systems safely and reliably in production-aligned environments. AI agents can fail in unpredictable ways — runaway loops, unsafe tool calls, exposed data, excessive cost, unreliable outputs and behaviour that is hard to audit. Participants learn how AgentOps, LLMOps and DevOps practices come together, how to recognise agent failure modes and how to place a CI evaluation gate in front of every deploy.
Working on a LangGraph-based multi-agent banking-report system (Streamlit UI, NLP, DB, Reporting and Email agents under a Supervisor, run via Docker Compose), participants add security and governance controls, reliability and observability controls, and production hardening with cost control — each module driven by an AI-assisted build prompt and verified against the same evaluation harness. The Capstone runs every control end to end and produces a finalised runbook, a governance evidence pack and an operational risk register, presented live to the assessor.
What You'll Learn
• Implement security and governance controls on an AI agent system — RBAC, tool allow-lists, audit logging, prompt-injection guards and human-approval checkpoints. (LO2)
• Explain reliability and observability practices for AI agent systems — retries, timeouts, circuit breakers, runaway-loop caps, correlation-ID tracing, logs, metrics and dashboards. (LO3)
• Explain production-hardening and cost-control techniques for AI agent systems — the hardening checklist, token/cost profiling, caching, model fallback, budget guardrails and runbooks. (LO4)
• Verify an AI agent system’s controls end to end and produce a production runbook, a governance evidence pack and an operational risk register. (LO5)
Entry Requirements
• Programming experience in any modern language.
• Understanding of cloud environments and CI/CD concepts.
• Familiarity with APIs, microservices, or AI agent workflows is helpful but not required.
• Basic knowledge of software testing, application support, or production operations will be useful.
• Participants should be comfortable working with logs, configuration files, deployment concepts, and troubleshooting exercises.
• IT Certification either Diploma or Degree