The AI Governance Trap Smart Leaders Need to Avoid
By Suzie Thoraval
Why commercial judgement matters when AI moves faster than the rules
“For everything that AI can do, AI can’t decide which problems are worth solving.”
When I was an in-house lawyer, I understood that part of my usefulness was to understand the organisation’s strategy and commercial drivers, then work out what the team was trying to do and how close we could legally get to that outcome. ‘Being commercial’ meant knowing where the legal line sat, being honest with people about the risks, and helping the decision-makers make a sound call with their eyes open.
I remember one campaign where the marketing team wanted to present an offer as a discount. When I asked them to walk me through how the discount actually worked, you could argue it was not really a discount at all. There was a legal argument that it could be described that way, provided the qualifications were clear, so we tested the position with a barrister’s opinion, tightened the wording and made sure the images carried clear disclaimer wording. The team got the marketing they wanted, and the business stayed well clear of misleading or deceptive conduct.
I find that in board and committee governance, that skill is still useful because the same kind of judgement is being asked of leaders deciding how to use AI. When a chatbot answers your customer or AI writes your marketing claim, the business is still responsible for what is produced. The Australian Treasury’s Review of AI and the Australian Consumer Law noted that the ban on misleading and deceptive conduct already covers what your AI tells people, and pointed to the Air Canada chatbot case, where a chatbot gave a grieving passenger wrong information about a refund and the airline was held responsible for it.
The rule against misleading people is the same as it has always been. What has changed is how quickly an organisation using AI can cross it if they are not careful.
Why this calls for slow thinking
Many experienced leaders I talk to are proud of how quickly they can make decisions. That ability helps them get through days full of competing issues, limited time and constant pressure. It comes from years of recognising patterns and experiences they have seen before, and most of the time it serves them well.
Nobel Prize winner Daniel Kahneman called this fast thinking, the confident, almost automatic judgement that draws on what we have seen before. AI creates a different challenge. Some of a leader's experience will still help, but some will give false comfort.
A tool may look like ordinary software while creating new risks around privacy, bias, accuracy, transparency, accountability and customer harm. A chatbot may look like a service improvement while actually making promises the organisation never intended to make.
That is where slow thinking matters. Your experience still counts, but your judgement is needed to see the parts of a new situation that you have not seen before. You need to be willing to slow down, get advice, and think this through before you say yes.
Unlike the European Union’s AI Act, which uses a risk-based approach to sort AI uses into categories with obligations attached, Australia does not yet have a single, comprehensive AI Act that does this. Instead, Australia governs AI through the laws that already apply, supported by principles-based guidance and emerging guardrails. The AICD and Human Technology Institute’s director guide to AI governance helps you as a leader or director to ask better questions, but it cannot tell you whether a particular use of AI in your organisation has crossed a line. That is your leadership judgement to make.
Organisations and leaders are feeling the competitive pressure to adopt AI quickly, for fear of being left behind. But I think that you still need enough adaptive stability to resist that pressure long enough to consider the risks AI adoption is inviting into your organisation.
AI adoption - action you can take to help you decide
Here are three actions I used as a commercial lawyer that you may find useful in considering AI adoption:
Ask people to show you how the AI will actually work. That means being walked through what the system does with your data, how it reaches its answers, how the output is checked, and what could happen if the answer is wrong.
Check your thinking with someone who knows more than you do. Good judgement, particularly for novel situations, is enhanced by a diversity of views. For example, a privacy specialist, someone technical, or your risk and assurance people.
Put governance safeguards in place. For AI, that might be a human person checking high-stakes decisions, being upfront when something was produced by AI, or limits on where a tool is used by staff. Safer AI adoption means building security, governance and risk thinking into the decision from the start, with clear ownership and oversight as the tool is tested, approved and used.
Use Adaptive Stability
AI adoption requires us to use adaptive stability. The stability is the steadiness to slow down the decision and stay grounded in what must be maintained, in this case compliance with laws, trust, accountability and care for the people affected by the technology, instead of reaching for the fast answer for fear of being left behind. The adaptation is the openness to adopting AI and working out how you can do it safely.
Being commercial means helping the organisation move forward with the risks considered, the guardrails in place, and the people who own the decision making it with their eyes open. You can trust your experience as long as you also seek the new information that a new situation deserves.
Where are you reaching for a quick answer when the moment needs your considered judgement?
