AI Security & Red-Teaming
AI security and red-teaming is offensive and defensive practice for LLM-powered systems — adversarially probing them for the failure modes that classic application security does not catch. It centres on the attack surface unique to language models: prompt injection (direct and indirect, including data-exfiltration via tool calls), jailbreaks, training-data and system-prompt extraction, and the new risks introduced when agents are given tools and autonomy. The defensive side is just as concrete: input/output filtering, privilege separation for tools, sandboxing, and the monitoring that detects an attack in progress. This is among the fastest-moving security fields, because every new agent capability opens a new attack path.
What you'll learn
- Execute the core LLM attacks — direct and indirect prompt injection, jailbreaks, and system-prompt/training-data extraction
- Reason about agentic risk: how tool access, autonomy, and untrusted content combine into data-exfiltration and privilege-escalation paths
- Build the defensive stack — input/output filtering, tool privilege separation, sandboxing, and human-in-the-loop gates
- Stand up monitoring and red-team evaluations that catch an attack in progress rather than after the breach
Want something you can start today? The catalog lists every track that is open right now.
Explore the catalog