Ep 5 | Joshua Saxe - The Hard Part Isn't Building the Agent: Measuring Effectiveness | [un]prompted 2026

Day 1 | Stage 1 As AI coding tools drive the cost of building security agents toward zero, the hard problem becomes knowing whether they'll actually work against real attacks and vulnerabilities not seen before. Presents a practical journey from naive precision/recall metrics to multi-dimensional evaluation that captures reasoning quality, evidence gathering, and tool-calling logic. Shows how proper measurement unlocks automated agent improvement using genetic algorithms and AI coding tools. Includes live demo. Joshua Saxe is AI Security Technical Lead, Meta. Watch on YouTube: https://www.youtube.com/watch?v=rO2yA52U_i4