AI RISKS Q&A. Putting recent AI risk debates into perspective.

Send us Fan MailIn this episode of The Agile Software Engineering Deep Dive, Alessandro Guida returns to the subject of AI risk with a Q&A exploring some of the questions raised by recent developments in agentic AI and recursive self-improvement. The discussion begins with the Hugging Face incident and what it tells us about agents that pursue goals in unexpected ways, circumvent restrictions, discover new uses for their environment and interact with systems their designers never intended them to reach. From there, the episode examines whether surprising agent behaviour should be interpreted as evidence of independent will, and explains why capable language models combined with tools, memory, permissions and repeated attempts can already produce behaviour that is difficult to predict. The episode then turns to recursive self-improvement. What does it actually mean for AI to help improve AI? Can synthetic training data contribute to better future models? What are the limits imposed by model collapse, evaluation, compute, infrastructure and reliable feedback? And how plausible is the idea that this process could eventually accelerate dramatically? Finally, the discussion looks at the risks that already deserve serious attention: badly engineered agents, excessive permissions, weak containment, cyber misuse, inadequate independent oversight and the competitive race to build increasingly capable AI systems. The central question is therefore becoming less about whether AI will suddenly acquire evil intentions and more about whether we can engineer, secure and govern increasingly autonomous systems well enough to keep their behaviour within acceptable boundaries Support the showThis Podcast is an audio version of the written Agile Software Engineering newsletter.  If you want to go deeper, don't forget to subscribe the newsletter too.