At QCon AI NY 2025, Tracy Bannon presented a talk examining how the rapid adoption of AI agents is reshaping software systems. She warned that organizations risk repeating familiar architectural failures if they treat all “AI” or “agents” as interchangeable.
Bannon noted that much of the confusion arises from labeling very different behaviors and risk profiles as the same. Bots are scripted responders reacting to predefined triggers, while assistants collaborate with humans and remain under human control. In contrast, agents are goal-driven, capable of making decisions and taking actions across systems.
Bannon highlighted a central theme: autonomy does not fail on its own; failures happen when autonomy outpaces architectural discipline. She coined the term “agentic debt” to describe this gap, linking it to issues such as identity and permissions sprawl, insufficient segmentation, and weak validation and safety checks.
She connected agentic debt to broader industry trends, mentioning research showing that many technology decision-makers expect severity in technical debt to rise due to AI-driven complexities. Bannon argued that while AI does not introduce new failure modes, it magnifies existing ones by accelerating change and broadening the impact of mistakes.
Focusing on established architectural principles for agentic systems, she emphasized that organizations know how to manage risk in distributed systems but often overlook these lessons under pressure. Governance, she suggested, should include the minimum controls needed to build trust, such as clear accountability and traceability of actions.
Identity was underscored as a foundational control for other safeguards. Every agent should have a unique, revocable identity, allowing organizations to answer key questions quickly when issues arise, such as what the agent can access and what actions it has taken.
Another recurring theme was decision-making discipline. Bannon urged teams to start by asking “why” instead of “how,” making trade-offs explicit before increasing autonomy. She described decisions as optimizations that improve one aspect while compromising another, like value versus effort or speed versus quality.
Bannon concluded her talk by urging architects and senior engineers to play an active role in how AI agents are integrated, emphasizing the importance of preventing architectural amnesia by designing governed agents rather than creating ad hoc automations. She noted that core software architecture practices remain relevant, and the challenge lies not in learning new disciplines, but in applying established principles effectively.
—
