Know Your Agent: Building Secure, Trusted & Governed AI Agents

We would never hand a stranger our production password, yet we give AI agents the keys and let them act alone. Agents are superhuman in speed, fresher in judgment and zero in accountability. This session argues we should govern agents like people and architect them like machines. Through the lens of the Agile Manifesto at 25, we cover agent identity and least privilege, agents supervising agents with fast decision models, and decision records that answer “who approved that?” under NIST AI RMF, the EU AI Act and India’s emerging rules.

Keytakeaways:

1. Treat every AI agent as an insider: give it its own identity and short-lived, least-privilege access, never a shared key or a developer’s personal token.
2. You cannot prompt your way out of prompt injection: check every agent for the “lethal trifecta” (private data, untrusted content, a way to send data out) and enforce policy at the tool boundary.
3. Stack your reviewers, because human review alone does not scale: deterministic rules, then fast decision models, then LLM judges, then humans, escalating by risk and confidence.
4. A confidence score is a claim, not a fact: check the calibration of any AI judge before you let its number decide what gets approved.
5. Governance is a data model, not a document: record every agent action (who acted, what it saw, what it did, who approved it) and build it into the Definition of Done every sprint.