Deborah Stone is a renowned scholar who has taught at Brandeis, MIT, and other universities around the world. Her award-winning book Policy Paradox has captivated readers through three decades, four editions, and six translations—but who’s counting? Her new book, Counting: How We Use Numbers to Decide What Matters, is now available in paperback from Liveright and can be adopted for spring 2022 courses. She lives in Brookline, Massachusetts.
Image Credit: Goodman/Van Riper
Numbers have become our public truth meters. Not only in the natural and social sciences, where statistics have long reigned supreme, but also in law, public health, history, humanities, environmental studies, international development, and human rights. Today’s students need data literacy more than ever.
Data literacy courses typically cover much the same material as basic statistics courses—sampling, correlation, probability, causal inference, and graphics. But here’s the rub: The most sophisticated statistical techniques are only as good as the raw data the statistician starts with. The biggest challenge in teaching data literacy is that students will take statistical analyses at face value without digging deeper into how the raw numbers were made.
Teaching data literacy (and statistics, too) should begin with basic counting. We and our students need to understand the essential thought process that enables people to count. This post suggests ways to bring that understanding to life and engage students in critically analyzing raw numbers.
The verb “to count” has two meanings. It means to tally up how many of something there are. You might tell a student, “You got 15 out of 20 questions right.” But “to count” also means to matter, to be important, or to be taken into consideration. A student might ask you, “Does class participation count in my grade?” Every teacher must decide what counts as a demonstration of learning. In other words, counting requires classifying things according to features we think are important. Paradoxically, before we count in the sense of “tally,” we must decide what counts in the sense of “to matter.”
Counting takes readers through every aspect of how numbers come into being, how their “authors” invest them with meaning, how they can be used to assert power in both oppressive and liberating ways; and how they can even change how people think. The key lesson is that every number is the result of human judgments about what to count and what not to count. For example, the government method of counting unemployment excludes people who aren’t actively applying for jobs and those who couldn’t start a job right away due to illness, disability, or family care responsibilities. These criteria embody long-standing cultural attitudes about the morality of working and not working.
To get students actively inquiring about numbers, I start by finding current news and scholarly articles about hot topics related to the course I’m teaching—for example, the number of arrests at a Black Lives Matter protest, the number of COVID-19 cases in a region, or articles about accuracy in the 2020 census. Then I pose a series of analytic questions to help students explore how a number comes into being and what human influences shape it. Even if they don’t have solid evidence to answer these questions, I encourage them to speculate (aka hypothesize).
- What’s the author’s purpose for measuring or counting something?
- Did the author perform the original count or rely on data that had already been collected by other researchers, public officials, or private organizations?
- Did the person or organization that counted have a stake in getting a big number or a small number? If so, how did the stake influence the decision about what to count?
- Figure out as best you can what criteria the original “counter” used to decide whether to include or exclude something from the category being counted. How explicit and how clear are the criteria? Is there room for confusion or disagreement?
- And most important, are there things you think should have been included in the category that weren’t, or things that were excluded but should have been counted? Why? Argue for your position.
In higher-level courses where students work on their own seminar papers or theses, I suggest they identify one statistic that is central to their field and widely accepted as an indicator, such as the poverty line, the crime rate, the number of people lacking health insurance, inequality of income and wealth, or Transparency International’s corruption rankings. I ask students to work through the same series of questions, plus a few more:
- How might our understanding of the issue be different if we used different criteria for counting?
- Which people and interests are helped or harmed by someone or something being counted or not?
- Write the “origin story” of your number as a short story, as if the number were a character telling its coming-of-age story.
Sometimes I ask students to read a statistics-based article as if it were a literary work.
- What rhetorical devices do the authors use to make their numbers seem more credible and authoritative? (Notice, for example, how authors sneak in their own self-peer-review by telling readers their study was “objective,” their methods “rigorous,” and their conclusions “evidence-based.”)
- How do authors use dramatic techniques and verbal costumes (names, labels, verbs, adjectives, adverbs) to convince readers that numbers support their argument?
- Does an author use a metaphor to “give a feel” for size, and if so, how does the metaphor influence how you and others might interpret the number as being good or bad? (Bill McKibben likens the amount of heat that is now being trapped near the Earth every day to the heat from 400,000 bombs like the one the U.S. dropped on Hiroshima.)
Attitude and opinion surveys are major tools in many fields (think elections, climate change, or pandemic policy). Rather than focus on sampling methods, as survey methods courses do, I get students thinking about how the content of survey questions can reflect particular values or points of view and thereby shape how respondents think and what answers they give. The American National Election Studies sometimes measure the prevalence of racist attitudes by asking respondents to “rate blacks in general” on three character traits, using 7-point scales that run from “hardworking” to “lazy,” “peaceful” to “violent,” and “intelligent” to “not intelligent.” My students are always shocked by the racist tropes built into these questions, but someone usually comes to the surveys’ defense. As a Nigerian student put it, “How else are we going to find out if white people think Africans live in trees if we don’t ask them?” Then we ponder how it might be possible to surface racist thinking without putting forth racist ideas. Finally, as either a group discussion or a written exercise, I’ll ask students to compose questions that build in values they support, such as equity and inclusiveness. The point is not to write a perfect question, but rather, to be aware that all counting rests on values of the people who count.
Real data literacy means understanding how and why numbers cannot be objective. The criteria for deciding what counts in any category include personal beliefs, cultural and societal norms, political interests, and moral values. Nevertheless, numbers are a ubiquitous language of public discourse, and they are extremely useful and powerful. Students should come away from a data literacy course knowing one big thing: A number isn’t just a numeral; its somebody’s argument about what’s important.