go back--

Why Students Think They Understand When They Don't by Daniel T. Willingham
associate professor of cognitive psychology and neuroscience at the University of Virginia and author of Cognition: The Thinking Animal. His research focuses on the role of consciousness in learning.  AMERICAN EDUCATOR WINTER 2003/04,  page 38 - 41, 48

http://www.aft.org/pubs-reports/american_educator/winter03-04/cognitive.html

How does the mind work-and especially how does it learn? Teachers make assumptions all day long about how' students best comprehend, remember, and create. These assumptions-and the teaching decisions that result-are based on a mix of theories learned in teacher education, trial and error, craft knowledge, and gut instinct. Such gut knowledge often serves us well but is there anything sturdier to rely on?

Cognitive science is an interdisciplinary field of researchers from psychology, neuroscience, linguistics, philosophy, computer science, and anthropology who seek to understand the mind, In this regular American Educator column, we will consider findings from this field that are strong and clear enough to merit classroom application.

Question: Very often, students will think they understand a body of material. Believing that they know it, they stop trying to learn more. But, come test time, it turns out they really don't know the material. Can cognitive science tell us anything about why students are commonly mistaken about what they know and don't know? Are there any strategies teachers can use to help students better estimate what they know?

Answer: There are multiple cues by which each of us assess what we know and don't know. But these cues are fallible, which explains why students sometimes think that they know material better than their classroom performance indicates.

* * *

How do we know that we know something?

If I said to you, "Could you name the first President of the United States?" you would say, "Yes, I could tell you that." On the other hand, if I said, "Could you tell me the names of the two series of novels written by Anthony Trollope?" you might say, "No." What processes go into your judgment of what you know? The answer may at first seem obvious: You look in your memory and see what's there. For the first question, you determine that your memory contains the fact that George Washington was the first U.S. President, so you answer "yes." For the second question, if you determine that your memory contains little information about Trollope (and doesn't include the novel series named Barchester and Palliser), you would answer "no."

But, if the mechanism were really so simple, we would seldom-if ever-make mistakes about what we know. In fact, we do make such mistakes. For example, we have all confidently thought that we knew how to get to a destination, but then when put to the test by actually having to drive there, we realize that we don't know. The route may seem familiar, but that's a far cry from recalling every turn and street name.

The feeling of knowing has an important role in school settings because it is a key determinant of student studying (e.g., Mazzoni & Cornoldi, 1993). Suppose a third-grader has been studying the Vikings with the goal of understanding where they were from and what they did, At what point does the third-grader say to him or herself: "I understand this. If the teacher asks me, 'Who were the Vikings?' I could give a good answer."

Every teacher has seen that students' assessments of their own knowledge are not always accurate. Indeed, this inaccuracy can be a source of significant frustration for students on examinations. The student is certain that he or she has mastered some material, yet performs poorly on a test, and may, therefore, conclude that the test was not fair. The student has assessed his or her knowledge and concluded that it is solid, yet the examination indicates that it is not. What happened? What cues do students use to decide that they know something? .

Cognitive science research has shown that two cues are especially important in guiding our judgments of what we know: (1) our "familiarity" with a given body of information and (2) our "partial access" to that information. In this column, I'll discuss how these two cues can lead students to believe that they know material when they don't. And, in the box on page 41, I suggest ways that teachers can help students develop more realistic self-assessments of their knowledge.

"Familiarity" Fools Our Mind into Thinking We Know More than We Do

The idea of familiarity is, well, familiar to all of us. We have all had the experience of seeing someone and sensing that her face is familiar but being unable to remember who that person is or how we know her.

Psychologists distinguish between familiarity and recollection. Familiarity is the knowledge of having seen or otherwise experienced some stimulus before, but having little information associated with it in your memory. Recollection, on the other hand, is characterized by richer associations. For example, a young student might be familiar with George Washington (he knows he was a President and maybe that there's a holiday named after him), whereas an older student could probably recollect a substantial narrative about him. (See Yonelinas, 2002, for an extended review of the differences between recollection and familiarity.)

Although familiarity and recollection are different, an insidious effect of familiarity is that it can give you the feeling that you know something when you really don't. For example, it has been shown that if some key words of a question are familiar, you are more likely to think that you know the answer to the question. In one experiment demonstrating this effect (Reder, 1987), subjects were exposed to a variety of word pairs (e.g. "golf" and "par") and then asked to complete a short task that required them to think at least for a moment about the words. Next, subjects saw a set of trivia questions, some of which used words that the subjects had just been exposed to in the previous task. Subjects were asked to make a rapid judgment as to whether or not they knew the answer to the question-and then they were to provide the answer.

If the trivia question contained key words from the previous task (e.g., "What term in golf refers to a score of one under par on a particular hole?"), those words should have seemed familiar, and may have led to a feeling of knowing. Indeed, Reder found that subjects were likely to say that they knew the answer to a question containing familiar words, irrespective of whether they could actually answer the question. For questions in which words had not been rendered familiar, subjects were fairly accurate in rapidly assessing their knowledge.

A similar effect was observed in an experiment using arithmetic problems (Reder & Ritter, 1992). On each trial of this experiment, subjects saw an addition or multiplication problem (e.g., 81 + 35) and they had to rapidly decide whether they would calculate the answer or answer from memory. If they chose to calculate, they had 20 seconds to do so; if they chose to answer from memory, they had just 1.4 seconds. Sometimes problems repeated, so subjects might have had the answer to a complex problem in memory. Subjects were paid depending on their speed and accuracy, so the decision about whether or not to calculate was important. As in the trivia question experiment, subjects were accurate in knowing when they could retrieve an answer from memory and when they needed to calculate itexcept in one situation, when the experimenters repeated a two-digit problem but changed the operation (e.g., addition to multiplication). In that case, subjects were just as likely to try to retrieve an answer from memory for a problem they had actually just seen (e.g., 81 + 35) as they were for a problem they had not just seen but which used familiar operands (e.g., 81 - 35). The experimenters argued that subjects made their judgment about whether to calculate based on the familiarity of the problem components, not on the whether the answer was in memory.

"Partial Access": Our Mind Is Fooled When We Know Part of the Material or Related Material

A second basis for the feeling of knowing is "partial access," which refers to the knowledge that an individual has of either a component of the target material or information closely related to the target material. Suppose I ask you a question and the answer doesn't immediately come to mind, but some related information does. For example, when I ask for the names of the two series of Trollope novels, you readily recall Barchester and you know I mentioned the other series earlier; you even remember that it started with the letter P, and you believe it had two or three syllables. Your quick retrieval of this partial information will lead to a feeling of knowing the relevant information-even if Palliser is not actually in your memory.

The effect of partial access was demonstrated in an experiment (Kariat & Levy-Sadot, 2001) in which subjects were asked difficult trivia questions. If subjects couldn't answer a particular question, they were asked to judge whether they would recognize the answer if they saw it (i.e., to make "a feeling-of-knowing judgment): The interesting twist: Some of the questions used categories for which lots of examples came to mind for their subjects (e.g., composers) and matching questions used categories for which few examples came to mind (e.g., choreographers)-that is, these subjects could easily think of at least a few famous composers, but couldn't think of more than one or two choreographers, if any.

The results showed that whether or not they could actually recognize the right answer, people gave higher feeling-of-knowing judgments to questions using many-example categories (e.g., "Who composed the music for the ballet Swan Lake?") than to questions using few-example categories (e.g., "Who choreographed the ballet Swan Lake?"). The experimenters argued that when people seethe composer question, the answer doesn't come to mind, but the names of (several composers do. This related information leads to a feeling of knowing. Informally, we could say that subjects conclude (consciously or unconsciously), "I can't retrieve the Swan Lake composer right now, but I certainly seem to know a lot about composers. With a little more time, the answer to the question, could probably be found." On the other hand, the choreographer question brings little information to mind and, therefore, no feeling of knowing. *

These studies, and dozens of others like them, confirm two general principles of how people gauge their memories. First, people do not assess their knowledge directly by inspecting the contents of memory. Rather, they use cues such as familiarity and partial access. Second, most of the time these cues provide a reasonable assessment of knowledge, but they are fallible.

* Another important aspect of this phenomenon is that the accuracy of partially retrieved information is irrelevant to the feeling of knowing. In an experiment illustrating this phenomenon, Asher Koriat (1993) asked subjects to learn strings of letters. Later, subjects were asked to recall as many letters as possible and then judge whether they would successfully recognize the entire letter string from among several choices. Subjects' confidence that they would recognize the letter string increased with the number of letters that they had recalled, regardless of whether or not those letters were correct. The more they thought they were pulling out of memory, the more confident they were that they really knew the whole string and would recognize it when they saw it.

How Students End Up with "Familiarity" and "Partial Access" to Material

If a student believes that he knows material, he will likely divert attention elsewhere; he will stop listening, reading, working, or participating. Mentally "checking out" is never a good choice for students, but all the more so when they disengage because they think they know material that, in fact, they do not know. The feeling of knowing becomes a problem if you have the feeling without the knowing. There are some very obvious ways in which students can reach this unfortunate situation in a school setting. Here are several common ones:

If a student believes that he knows material, he will likely divert attention elsewhere; he will stop, listening, reading, working, or participating.

1. Rereading. 'To prepare for an examination, a student rereads her class notes and textbook. Along the way, she encounters familiar terms ("familiar" as in she knows she's heard these terms before), and indeed they become even more familiar to her as she rereads. She thinks, "Yes, I've seen this, I know this, I understand this." But feeling that you understand material as it is presented to you is not the same as being able to recount it yourself.

As teachers know, this gap between feeling that you know and genuine recollection can cause great frustration. I have frequently had exchanges in which one of my students protests that despite a low test grade, he or she really knew the material. When I ask a general question or two, the student struggles to answer and ends up sputtering, "I can't exactly explain it, but I know it!" Invariably, a student with this problem has spent a great deal of time reading over the course material, yielding a lot of familiarity, but not the necessary and richer recollective knowledge.

2. Shallow Processing. A teacher may prepare an excellent lesson containing a good deal of deep meaning. But this deep meaning will only reside in a student's memory if the student has actively thought about that deep meaning (see "Students Remember...What They Think About," American Educator, Summer 2003, http://www.aft.org/pubs-reports/american_educator/summer2003/cogsci.html). Let's say, for example, that a teacher has prepared a lesson on the European settlement of Australia and on the meaningful issue of whether that settlement should be viewed as a colonization or invasion. But, let's say that a given student did not process and retain the deep meaning intended by the lesson. He did absorb key terms like "Captain Cook" and "Aborigines." His familiarity with these key terms could mislead him into believing he was ready for a test on the subject.

3. Recollecting Related Information. Sometimes students know a lot of information related to the target topic, and that makes them feel as though they know the target information. (This is analogous to the subjects in the experiment who knew the names of many composers and so felt that they (Continued on page 48)


How To Help Students See When Their Knowledge Is Superficial or Incomplete

What can be done to combat spurious feelings of knowing in students? Remedies center on jostling students away from a reliance on familiarity and partial access as indices of their knowledge, and encouraging (or requiring) them to test just how much knowledge they recall and understand.

Make it clear to students that the standard of "knowing" is the "ability to explain to others," not "understanding when explained by others." I have found the following analogy helpful in explaining the difference in the two types of knowing: You and a friend are watching a movie that only you have seen before. As the plot unfolds, each event, even those meant to be surprising, seems predictable and familiar. Yet if your friend asks you, "How does it end?" you can't quite remember. To truly know about a movie (or a mathematical concept or historical event), you must be able to discuss it in your own words.

Require students to articulate what they know in writing or orally, thereby making what they know and don't know explicit, and therefore easier to evaluate, and easier to build on or revise. Suppose that you've just gone over a rather tricky point in class. You want to be sure that they've understood the lesson. As we all know, asking "Does everyone understand the main point here?" yields only silence. Calling on one student makes it clear to that student whether or not he or she understands the main point, but brings little benefit to other students. An alternative is to have students pair off and then take turns explaining the main idea to each other. (This will work best if the teacher provides clear criteria by which students can judge each other's answers; otherwise it can be a case of the blind leading the blind.) The process of having to explain aloud to someone else makes it clear to students whether or not they understand what they are meant to understand. The process breaks the ice of silence, and if the teacher afterwards asks if there are questions, students are usually more willing to ask for help. Indeed, observing the pairs will usually make the extent of students' understanding clear to the teacher.

Begin each day (or selected days) with a written self test. The teacher may pose a few questions reviewing the material from the previous lesson. The success of this strategy depends on students writing their answers rather than having the class shout out answers or calling on students who raise their hands. Again, the question you pose will likely lead to a feeling of knowing in most students because it is material they were recently taught. If, moments after hearing the question, they hear the answer provided by another student, they will likely think, "Sure, right, I knew that" because of this feeling of knowing. To get an accurate assessment of memory, each student must see whether he or she can recollect it.

Ask students to do self tests at home or in preparing for examinations. For students who are a bit older, teachers can facilitate this process by organizing "study buddies" who agree to meet at least once before an examination, or at regular intervals, to test one another. Study buddies ask one another questions to ensure that they understand the material, and then go over whatever they don't understand. This procedure brings several benefits. It's another way to force students to actually recall information, rather than to simply recognize what is in the book. The process of generating questions for a partner is also an excellent way to encourage students to think deeply about the material; it is tantamount to asking oneself, "What is really important here? What must I know about this material?" That students pose questions for each other means that students will share their perspectives on the material. A point that one student missed or understood dimly will be supported by the other student's knowledge.

Help students prepare for examinations with study guides. All students, but especially younger students, need help identifying the core information to be tested. Teacher-developed study guides are an excellent way to be sure that students are aware of the critical questions and key elements of the answers. Whether they study alone or with a buddy, the guide assures that all students will tackle the most difficult concepts r materials being tested. -DW