James Wehrli is a teacher of economics and political science at the American School of Paris, where he also serves on the school’s Work Council.
Truth be told, if I couldn’t teach, I would be very happy just writing test questions all day. When I write multiple-choice tests for my students, I tell them, “I get paid to lie to you 75 percent of the time!”
Writing test questions—whether multiple choice, free-response, short-answer, or stimulus-based—gives me a better understanding of the material and what is important to know about the topics I’m teaching. And, if done correctly, exams can also be an opportunity for students to make larger connections about the course material they’ve studied all year and even relate it to concepts from other courses.
Tests are a learning experience for the test-taker. Tests are not only a way to find out how much students know, but they are also a way to guide the test taker as to what is most important about what they are learning. Whether it is for a chapter test, a semester exam, or a test bank, I write questions that focus on details and the big picture. Students shouldn’t lose the forest for the trees, but they also should not forget about the trees as well. If students do not understand the basics, then they will never get the big picture; but if they get too mired in the details, they won’t see the broader perspectives.
If several students got a question wrong, they probably did not understand that topic very well (or the question was poorly written). But it also might be that the teacher must try harder to break students free of the common-sense responses that students rely on before truly understanding so many of the counterintuitive ideas contained in a comparative government course. Howard Gardner calls these common-sense answers the “unschooled mind,” the economist John Kenneth Galbraith referred to them as “conventional wisdom,” and the psychologist Daniel Kahneman referred to them as “System 1.”
While it is true that some test questions should be very straightforward, such as definition questions (e.g., a rentier state is ….) or an identify question (Which legislative house in Britain is more powerful? Which of the following counties has a presidential system?), teachers have to teach concepts and provide questions that are counterintuitive (e.g., a disadvantage of removing corrupt civil servants is that they are replaced, at least in the short run, with inexperienced government workers who also decrease political efficacy).
Test questions are a way to break students away from these uninitiated answers. For example, before taking a comparative government course, many American students believe that democracy only comes in one package. They do not even realize that most countries have a unitary (as opposed to federal) system of government and that most democracies use a parliamentary (as opposed to presidential) system of government. Teaching in Europe makes it easier to overcome some of these obstacles since the students I teach have often lived in many different countries. For example, when I teach about universal health care, it is much easier to show students the advantages and much more difficult for them to understand how the United States has not adopted this system. When I taught in the United States, the challenge was in reverse. For an American audience, the health care issue is counterintuitive, but for a European audience of students is it more commonsensical. These are the kinds of questions that really show me if the students have become “schooled.”
The Process
- Purpose. When writing test questions for my class, I start with what I want the students to learn from the questions.
- Priorities. The questions I select for an assessment—either questions I write myself or ones that I select from a test bank—signal to my students the most important topics in a unit. Too often students have a difficult time understanding what to prioritize when studying a unit. It is my hope that after a semester or a year of taking my assessments, the students will have better insights into what is important. My goal is not for the students to become better test takers, but to learn how to glean the important material from what they are reading and learning in class.
- Levels of Difficulty. It is also important to write questions of varying levels of difficulty. Not all questions have to be challenging, and some teachers might be surprised that the basic questions are the ones that a fair number of students might get wrong.
- Variation on Theme. On tests, I also might ask the same question in a different way to see if the student truly understands the concept. For example, if I want the students to learn a definition, then I will create questions with different choices for what the correct definition is. I will also create questions where the definition is in the stem (the question) and the different choices contain definitions. For example, LEG-1.A.1, in the AP® Comparative Government and Politics CED, states “Legitimacy refers to whether a government’s constituents believe their government has the right to use power in the way they do.” For a multiple-choice question, this can be one of the choices for the stem, “Legitimacy refers to …” Most textbooks contain a glossary where this same process could be done.
- Bloom’s Taxonomy. Once I have decided what topics should be tested, I use the textbook Essentials of Comparative Politics with Cases, Eighth AP® Edition, and the Course and Exam Description (CED) to guide me in constructing questions. Textbooks and CEDs are set up in terms of Bloom’s taxonomy, which categorizes student learning into different levels Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation, or the newer version: Remember, Understand, Apply, Analyze, Evaluate, and Create. In terms of the AP® Comparative Government and Politics course, the curriculum covers everything except the “create” level. The higher levels of Bloom’s Taxonomy help to steer students away from the “common-sense” types of understanding mentioned earlier. Contrary to popular belief, if they are written well, multiple-choice questions can address all different levels of Bloom’s Taxonomy, and they can also work to break students away from “common sense” answers that are more common than accurate.
- Higher Order Questions. In terms of a higher order question, p. 354 of Essentials of Comparative Politics with Cases discusses “the long-term consequences” of political reforms. A question could be constructed that links Prime Minister Tony Blair’s “promises to devolve more authority to the people” and “Scotland’s demand for an independence referendum.” This section of the chapter also discusses the advantages and disadvantages of the United Kingdom having an unwritten constitution. This can be turned into a free-response question asking students to explain these advantages of disadvantages.
- Timing. I usually do not put the tests together until after I have taught the unit. When I taught the unit, I had a fundamental idea of what I wanted to get across to the students. And just like when a TV network is putting together its schedule or a magazine is selecting what articles to put in each of its sections, I figure out what questions will be put into each slot. I inform the students that I do this as well.
- Wording: When I write questions, I try to stay away from “not” and “except” questions. You should also avoid including questions that have “none of the above” or “all of the above” as choices. These make the questions much more difficult for students to understand. The teacher should also never try to trick the student. For example, all of the choices in a multiple-choice question should be real answers. If a question asks the student to identify a supranational organization, all of the choices should be actual organizations and preferably ones that the student should be familiar with.
Writing test questions can take time, one of your most valuable resources. But each year, I see how the tests I offer my students not only provide a perspective on how well they understand the material but also help guide them to be better learners. The ability to identify the most important themes and concepts from each unit will serve them not only in future classes but in the general pursuit of knowledge and help to encourage them to be life-long learners. And for me, I always leave with a greater appreciation and understanding of the material I’m presenting in the classroom.
AP® is a trademark registered and/or owned by the College Board, which was not involved in the production of, and does not endorse, this product.
MEET THE AUTHOR
James Wehrli is a teacher of economics and political science at the American School of Paris, where he also serves on the school’s Work Council. He has taught at the high school and college levels in the United States and has held several positions with the College Board—including AP® Consultant, Test Development Committee member, and College Board Advisor for AP® Comparative Government and Politics—while also editing and authoring a variety of AP® review books and teachers’ guides. In 2006, he was awarded National Board Professional Teaching certification in Social Studies—History. He and his wife, Teresa, have a daughter and a son, and they live outside of Paris with their two cats.