AI Governance Has a Killswitch Problem Nobody Wants to Admit

There's a question worth asking in your next leadership offsite — not about AI adoption rates or productivity benchmarks, but this: if one of your deployed AI models started producing harmful outputs tomorrow, who has the authority to shut it off? Not "who would we call." Who has the authority.

MIT Sloan Management Review reports that leaders at literally every Fortune 500 company claim to be governing their AI. Ask those same leaders who's responsible for shutting down a harmful model, and most go silent. That silence is the tell. Governance frameworks have proliferated, but accountability has not. Organizations have built the appearance of oversight without the infrastructure of control.

This is not a compliance gap you can paper over with a policy document. It is a structural failure — and it's about to become expensive.

The Surveillance Trap Meta Already Fell Into

The accountability problem has a corollary: when companies do instrument their AI systems to learn from employee behavior, they often create a second liability in the process. HR Executive reported that Meta suspended its keystroke tracking program after sensitive worker activity data was found to be accessible internally. The program was designed to train AI models on how employees work. Instead, it generated an internal data exposure incident and a trust crisis.

The architecture of AI training and the architecture of employee privacy are not naturally compatible. Most organizations building AI-assisted workflows haven't reconciled them. They've chosen speed. The assumption is that employee data collected for operational purposes can be repurposed for model training without meaningful consent or governance friction. Meta's experience suggests that assumption has a shelf life.

If your AI team and your people team have never jointly reviewed what employee behavioral data is being captured, stored, and used for model training — that conversation is overdue. Not because regulators are watching (though they increasingly are), but because the moment it surfaces publicly, the restitution cost is not just financial.

Upskilling Is Not the Same as Governing

It is tempting to treat AI upskilling programs as a governance substitute. They are not. Bank of America's Academy, led by Bernard Hampton, represents one of the more serious institutional commitments to preparing a large workforce for AI — focused on workforce agility and building durable human capability alongside machine capability. MIT Sloan Management Review frames this as a reskilling infrastructure problem, not a training event problem. That distinction matters.

But even the most sophisticated upskilling program does not answer the killswitch question. Employees who understand how to use AI tools are not the same as employees who understand when AI systems should be stopped, overridden, or escalated. Capability without accountability authority is just faster execution of potentially bad decisions.

The missing design element in most enterprise AI programs is not another learning module. It is a defined human-in-the-loop decision chain with named owners, escalation thresholds, and genuine authority — not just advisory input.

The Manipulation Vector Most Governance Frameworks Miss

There is a third dimension to this problem that governance frameworks almost universally ignore. MIT Sloan Management Review identifies three overlapping security threats from AI's effects on human cognition: weaponized persuasion that manipulates employee judgment through personalized messaging, erosion of critical evaluation as employees defer to AI outputs, and AI-enabled social engineering at scale. These are not science fiction scenarios. They are operational risks for organizations where employees regularly act on AI-generated recommendations without structured verification.

The governance conversation in most boardrooms is still focused on model bias and regulatory compliance. Those are real, but they are the visible threats. The subtler risk is that employees — including senior ones — become cognitively dependent on AI systems they cannot meaningfully audit. When the system is wrong, confidently, at speed, and at scale, the human layer doesn't catch it. It amplifies it.

McKinsey Global Institute's outgoing chair Sven Smit has argued that in an AI-driven world, the fastest learner will win. That framing is correct but incomplete. The fastest learner who also knows when to stop the machine wins. The fastest learner who can't distinguish confident AI error from confident AI accuracy becomes a liability multiplier.

The board question is not "how mature is our AI governance policy?" It is "who, by name, can shut this down — and have they ever practiced doing it?"