When Critical Thinking is Undermined
I recently served as a panelist for a screening of the documentary film, What the Health, at the University of Pennsylvania’s School of Veterinary Medicine. At this screening, an audience of people whose careers involve ensuring the health of livestock watched a film that advocates for removing meat and dairy entirely from the global diet. I am a cognitive psychologist, and so my role was to help them understand how the film might influence consumers and impact their industry.
I told them that the film makes it harder (not easier) for consumers to understand the complex issues of chronic disease and the global food supply. The film is not good journalism, and was not intended to be. Good journalism fosters critical thinking. “What the Health” does not do that. Instead, it uses emotion, imagery, and a very selective presentation of the facts to try to garner support for the film’s thesis: that the widespread adoption of a vegan diet will all but eliminate chronic disease.
The film has been extensively critiqued elsewhere by physicians and by journalists; my aim here is to offer some unique perspectives from behavioral science. I will spell out a framework (developed with my colleague Brandon McFadden) that can be used to assess and develop critical thinking in classrooms, in media, or in any forum for public discourse.
A Framework for Critical Thinking:
The framework that McFadden and I have developed draws on key principles from society’s critical thinking institutions (such as the scientific method), and it builds on insights from behavioral science that have shown extensive deficiencies in human critical thinking tendencies.
We suggest that effective critical thinking involves three types of activities:
1. Diligent clarification to make sure that claims of fact are true, unambiguous, and comprehensive. Critical thinkers are thus diligent in seeking the truth, the whole truth, and nothing but the truth.
2. Slow thinking to allow for appropriate inferences based on logic, probability, and evidence. Critical thinkers do not jump to conclusions.
3. Humble self-reflection to ensure clarity about what is known, what is not known, and to explicitly express uncertainty about beliefs in an honest and consistent manner. Critical thinkers are not overconfident.
We know from a large body of work in cognitive psychology that people are systematically imperfect in engaging in these critical thinking activities (more on that below). We can look at a film like What the Health and see what it does to either remedy or exploit these imperfections. Too often, it exploits them.
1. Diligent clarification:
Failing to diligently clarify facts in daily life can result in poor decisions (e.g., when we ignore facts that we don’t like) and damaged relationships (e.g., when we act unfairly to other people). Here are some well-known examples from the behavioral science literature, and illustrations from the film.
Confirmation bias: People tend to seek information that fits their pre-existing views. As Vox writer Julia Belluz notes, What the Health cherry-picks study results. For example, the film reports a link between dairy consumption and cancer, while completely ignoring high quality literature reviews that do not support this causal relationship, as well as some studies finding that dairy consumption can have health benefits.
Cheating: Nina Mazar and colleagues have shown that many people cheat at least a little when given the opportunity. One form of cheating is to misrepresent the statements of others, and this happens frequently in What the Health. For example, a scientist at the American Diabetes Association is dramatically quoted as saying he “won’t talk about diet.” The film tries to make it look like he is hiding something when in fact he is only saying that many diets are healthy and he doesn’t want to have a pedantic conversation about any single diet because there is near scientific consensus that no single diet is best.
2. Slow thinking:
Critical thinkers have to connect facts in appropriate ways. Principles of logic, probability, and the scientific method provide guidance here, but the behavioral science literature is rife with examples of how people fail to apply these principles. Here are some examples, again with illustrations from the film.
Affect heuristic: People make inferences about the risk or severity of a situation by considering how strongly the situation makes them feel, rather than by objectively assessing risks or outcomes. Many people perceive greater safety in automobile travel than in air travel because cars (which are ubiquitous and on the ground) just “feel” safer than airplanes (which are less commonly used and remarkably high up in the air). To exploit this tendency, the film liberally shows disgusting images from animal slaughter as it makes claims about the health outcomes of consuming animal products. This does not help critical thinking. It would be like trying to help people evaluate the health outcomes of surgery by showing them graphic and potentially disgusting images of such life-saving surgeries. Ethical concerns about animal welfare are real and legitimate, but they should be considered separately from issues of human health outcomes. The affect heuristic will tend to blur the distinction.
Fundamental attribution error: People tend to overestimate the role of traits (or general disposition) when interpreting the behavior of others. In cases of moral lapse, we tend to give ourselves a pass, blaming the situation (e.g., I was tired or hungry), but we are not as charitable with others. For others, we tend to attribute moral lapses to deeper traits or dispositions (e.g., he is a bad person, or she is a liar). We see a version of this in the film, where patient groups like the American Diabetes Association (ADA) are portrayed as evil agents engaged in full blown conspiracies. Groups like the ADA do take financial contributions from pharmaceutical companies, and this is indeed a conflict of interest. One of the other panelists at the film screening I attended, Matthew McCoy, has thoroughly documented this. But McCoy noted during the panel discussion that the film goes well beyond legitimately pointing out the potential for bias. Instead, the film tries to turn it into a simple black and white issue, suggesting that the patient organizations are evil and conspiring to hide the truth from the public just to keep themselves in business. That is a major leap in reasoning, and a failure of slow, critical thinking. These kinds of leaps happen frequently in the film. In another example, one of the film’s “experts” is discussing how the treatment of chronic diseases is more about symptom management than about cures. He calls this “a front of massive proportions.” While most public health officials believe that we need stronger disease prevention efforts, almost no one infers that the focus on treating symptoms has evil intentions, and is akin to a criminal front.
Denying tradeoffs: A fundamental reality of life is that when resources are scarce (e.g., limited time and financial budgets), or when preferences compete (e.g., convenience vs. safety), tradeoffs must be made. People often resist tradeoffs, and act like we may not need to make them, but this is rarely true. We see this in the film when the narrator complains that the ADA lists numerous recipes that use dairy products. He notes that some studies have found links between dairy and diabetes, and then asks, rather than promoting these products, “Why wouldn’t they be warning all parents about this even if there were only a slight chance.” This has been called zero-risk bias. Appropriately and pragmatically, the ADA recognizes that people want to make tradeoffs between very small health risks and very large social and hedonic benefits. People may take too much risk sometimes, but to suggest that no risk should be taken is not helpful. People drive their children to sporting events and parties even though there is some risk in leaving the house. Zero-risk involves trading off a lot of fun and opportunity. Understandably, very few people want to live that way.
3. Humble self-reflection:
Good critical thinkers express appropriate uncertainty in their claims. And they also understand their reasons for knowing things; that is, they directly connect their feeling of knowing to actual facts. Many well-known cognitive biases prevent people from doing this well.
Overconfidence: Don Moore has described overconfidence as the mother of all biases, both because it is so prevalent, and because it acts as an enabler for other biases. That is, we can fail to express doubt in opinions that we arrived at by jumping to conclusions. The film maker rarely expresses uncertainty about the power of an all-plant diet. To the contrary, we hear testimonials of people who claim that they have been cured by all-plant diets: a former NFL lineman says his tendinitis was cured, a middle aged woman says she could feel the inflammation leaving her body, and the narrator himself says he could feel the change in his blood and in his body. Any changes that occurred for these individuals cannot be attributed with certainty to a change in diet – true causality is very hard to establish in any individual context, and responsible journalism is careful to express that uncertainty.
Knowledge illusion: People tend to believe they understand the details of how the world works even when they don’t. One famous study showed that most people can’t accurately explain how a bicycle works, even though they are very competent at using one. The film maker supports knowledge illusions by showing simplified graphics of biological processes like fat accumulation, and blood flow. Seeing images like this can convince viewers that they are seeing something real, and that they understand the details of causal processes that even scientists don’t fully understand.
Narrative appeal: People like stories. The human mind is well suited to acquiring information through narrative. Stories are great, but critical thinkers need to self-reflect and be aware that stories are not necessarily the best way to accumulate facts and evaluate evidence. The film uses the narrator’s personal journey as a sort of background story for the flow of information, and this makes the film appealing, but it probably doesn’t help viewers think critically about the key issues and social conflicts.
Isn’t the film just a little harmless non-critical thinking?
Does it really matter? After all, the film has significant merit. It does a nice job of highlighting the very real burden of chronic disease. It also does a nice job of refuting the common notion that an “all-plant” diet is necessarily unhealthy. All-plant diets certainly can be healthy. (The exaggeration in the film is in suggesting that such a diet is healthier than all diets that include meat and dairy.) Finally, the film may not even be wrong in the direction of its recommendation – it is a common view in nutrition science that the American diet is higher in meat (especially processed meat) than it ought to be. So why quibble?
Here is why: We live in an era where there is staggering public skepticism about science. I believe that the standards of critical thinking about science in the media should be higher than they are. That means we need to talk about what critical thinking is, and call out venues or authors who hurt rather than help that mission.
I am not suggesting that all forms of media need to engage in careful critical thinking. Story-telling and humor have their place in public discourse. But we have to call out media limitations, especially when they oversimplify causal explanations that have impact on people’s lives. And this is doubly so for films like What the Health which attempt to wear a veil of critical thinking through much of the presentation.
Oversimplifying a causal explanation can distract attention from important scientific research. Oversimplifying is unfair to the sufferers of chronic disease. This was compellingly pointed out by a veterinary student who attended the same film screening that I did. At the end of the panel discussion, she told the audience of her experience as a patient with metastatic cancer. She had in fact been a vegetarian since she was a small child. She was highly skeptical of the film’s insinuation that an all-plant diet would prevent and reverse all chronic disease. Her disease had continued despite her healthy diet, and she thought that the film came uncomfortably close to victim blaming by suggesting that serious diseases like hers were so easily avoided and remedied. The millions of people with chronic disease deserve a higher level of critical thinking.
Disclosure: I have consulted for agribusiness, food service, and pharmaceutical companies, as well as non-profit organizations, on issues related to consumer behavior.
