This Week's Term: Cognitive Surrender - accepting an AI's answer as your conclusion without checking it against your own reasoning.
This meaning of the term comes from Steven Shaw and Gideon Nave, two researchers at Wharton, in a working paper they released in January, "Thinking - Fast, Slow, and Artificial". They set it against cognitive offloading, where a tool takes over a task while you go on thinking and deciding yourself. Think of a GPS. Following its directions is offloading. Stop watching the road and keep driving wherever it sends you, and that is surrender.
When does relying on AI become surrender?
In experiments with short logic puzzles, the researchers secretly had the AI assistant give a wrong answer about half the time, usually the tempting answer that looks right at first glance. When people consulted the AI and it was wrong, about 73% of the time they still went with its answer.
Back in issue #39 I shared a comparison from Audrey Tang, Taiwan's Cyber Ambassador: letting AI do our thinking for us is like sending a robot to the gym to lift the weights. I might want an AI to help plan my training or to watch me and give feedback, but I would not send it to the gym to lift the weights in my place.
Checking an AI answer in your own field takes your own expertise. That is deep knowledge, one of the advantages in this issue's leadership post, Four human advantages in the age of AI.
How do you decide what to hand over?
The researchers note that leaning on the AI is not always a mistake, for instance on tasks where it is reliably better than we are. A few things to try, so that handing over a decision stays your choice:
- Choose which decisions to hand over. Routine, well-structured tasks where the AI is usually right may be fine to give away. Judgment calls you are accountable for probably need you, or someone on your team, to make them.
- Write your own answer first. Put down one line before asking the AI. If it disagrees with you, work out why before you take its answer.
- Ask the AI to argue against your decision. For example: "Here is the decision I am considering: [decision]. Give me your recommendation, then list what would make it wrong, and how confident you are in each part. If more information is needed, interview me."
- Give credit for catching the AI's mistakes. In one experiment, a small reward for correct answers plus immediate feedback more than doubled how often people rejected wrong AI answers. Recognizing a team member who notices the AI got something wrong may have a similar effect.
Recommended resource
Gideon Nave and Steven Shaw explain where the term comes from on the Knowledge at Wharton podcast.
Your action step
Before your next AI-assisted decision this week, write down your own one-line answer first. If the AI disagrees with it, work out why before you go with the AI's answer.
Which decisions should stay with a person in your team, even when the AI could make them?
If you are deciding which decisions your teams should keep and which to hand to AI, that is the kind of question I take on in AI strategy advisory engagements and in sessions as an AI keynote speaker.