Whenever we spend multiple days on one particular thing, it is such a good feeling when someone looks at it and tells us that they agree. Maybe we are trying to convince our boss to change the metrics we look at to measure success, and we ask a co-worker whether our argument holds up before we put it in front of them. But the flip side is that the more we work on something, the less willing we are to hear that our idea is flawed and might need to be changed. We say we want another opinion, but often we just want someone (or something) to validate what we already think.
This week's studies made me think about what happens after AI gives us the agreement that we often seek. In one study, students who received flattering feedback chose to read less of the information and data that went against their original position. Another study looked at students who actively had to explain and then reconsider their own reasoning while they were using AI. And a third study tested whether showing people sources, along with a specific warning about accuracy, changed how often they chose an incorrect answer. These are all different uses of the same technology, but they don’t necessarily leave us with the same feelings or the same amounts of work.
STUDY 01
After the praise they read less of the opposing evidence
Business students made an initial judgment about a fictional company's expansion, received either flattering or evidence-focused AI feedback, and then chose four of eight available materials to open. Students who received the flattering feedback devoted 36.7 percent of their choices to material opposing their original judgment, compared with 47.5 percent in the other group. The finding concerns what they chose to read, not proof that they had lost the ability to think critically.
Read the study ↗
STUDY 02
The students had to explain why they thought they were right
Students in two data science classes wrote their own analyses before receiving feedback, with one class getting written instructor comments and the other using AI to question their reasoning, consider alternatives, and explain their revisions. The AI-feedback class later scored higher on a critical-thinking assessment and on its course tasks, after researchers matched students with similar starting characteristics. The advantage did not appear in every part of the thinking assessment. Because each arrangement was used in only one class, the study could not separate the feedback's contribution from other differences between the classes.
Read the study ↗
STUDY 03
Sources and a warning changed which answers people accepted
Researchers showed adults a mix of correct and incorrect AI-style answers, with some people also seeing source information, an accuracy warning, or both. When neither appeared, people accepted 45 percent of the incorrect answers, compared with 21 percent when both appeared. The combined condition also produced the most checking, although this was a short, low-risk task rather than a test of how people use AI over time.
Read the study ↗Also worth your time
The more resistant assistant was less popular. An AI instructed to prioritize accuracy over agreement held up better under pressure in benchmark tests, but people discussing policy or writing emails rated it as less respectful and satisfying than the other versions. Read the study ↗
Human help made difficult math feel more manageable. Students working through difficult math reported less difficulty and attempted more questions with human tutoring than with AI tutoring or web search, although the experiment did not measure who learned more or got more answers right. Read the study ↗
Liking the advice did not change the decision. People discussing a donation rated an AI adviser as more trustworthy and convincing than a human adviser, but donation decisions did not differ between those groups or a group that received no advice. Read the study ↗
We still need to check what people remember later. A review of selected AI-learning studies found that none measured performance with AI, immediately without it, and later without it while verifying that AI was unavailable at both independent tests. Read the review ↗
SOMETHING TO PLAY
Try Signal
Signal gives you a convincing claim and some evidence to inspect. Your job is to decide whether the evidence actually supports what you were told, and name the detail that helped you decide. You might agree with the claim at first and then find something that changes your mind. Give it a try at The Quiet Cost Practice.
Open Signal →The Reckoning
What bothers me about the business study is that the information that might have changed somebody's mind was actually still right there for the students to open. But after they received feedback that supported their position, they chose to spend less of their remaining search on the material that would have challenged it. I am already imagining how this would happen if we have a real decision to make and a lot of other things that we feel like we have to get done. Once something tells us that our current idea is good, it becomes a little easier to feel like we have all the evidence we need to send it through.
And I don't think the answer is to do a complete 180 and ask AI to disagree with everything we say. An assistant that argues with every single sentence would get old pretty quickly, and disagreement by itself doesn't necessarily mean that it is right, either. What I liked about the data science study was that the students had something specific that they needed to work through. They had to explain their choices, consider another interpretation, and decide exactly what it was they needed to change. The results of the study don't necessarily tell us which part of that process helped, but I would like to see more uses of AI that force us to examine our thinking like that.
The sources and warnings study also gives us something useful to work with as we begin to turn so much of our research over to AI. My main takeaway was that if we only look for information that agrees with us, having more information available won't necessarily help us notice all of the details we missed as we tried to find the truth.
I also worry about how that carries into the things we do with other people in our day-to-day lives. We might show up to a conversation feeling very prepared because we have already spent half an hour going through our entire idea with AI. But if that conversation mostly reassured us, we may be less interested in hearing why the person we are talking to sees it differently than we do. I would love it if, instead, we went into that conversation with the hopes that the person might change our minds, but that might be wishful thinking at this point.
Your turn
Think of an idea that AI recently helped you feel more confident about. What would make you reconsider it? Write that down, then look for one source or ask one person who might see it differently.
Michael McNamara
Illustrations created with AI. They are editorial images, not depictions of research participants.