Quiz Results
Lat Updated 2/25/2010
The Surprise Effect
Students are given the following instructions: This is not a graded quiz, it will not count towards your grade, nor is it a measure of intelligence. Rather, you will be asked a number of questions, and you should answer with a low (first) and a high (second) guess. Your answers should honestly indicate a range such that you are 90% sure that the correct value of the answer for the question lies within that range. For instance, if you were asked, "how old is Wallis?", you might answer between 37 and 45. Try to give your best estimate of the range, do not over or underestimate. If your subjective confidence were highly calibrated (i.e., reflects an accurate estimate of the likelihood your answer is correct) you should get approximately 90% of the answers correct. Those who score significantly below 90% correct exhibit subjective overconfidence.
| Percent correct | 0% | 5% | 10% | 15% | 20% | 25% | 30% | 35% | 40% | 45% | 50% |
| # of people | 24 | 2 | 3 | 1 | 1 | 1 | 1 | ||||
| Percent correct | 55% | 60% | 65% | 70% | 75% | 80% | 85% | 90% | 95% | 100% | |
| # of people | 2 | 1 | 1 | 1 |
The True/False Quiz
Students are given the following instructions: Answer the following questions as honestly as possible using only your background knowledge. Do not consult any reference materials. This is not a graded quiz, your score on this quiz will not affect your grade, nor is it a measure of intelligence. Rather, you will be asked judge whether or not each of a series of statements is true or false. You should answer "true" if you feel your confidence that the statement is true equals or exceeds 50%. If your confidence that the statement is true is below 50%, answer "false."
Chance alone dictates that the percentage correct by guessing alone should be approximately 50%. If one assumes that subjective confidence is calibrated (i.e., reflects an accurate estimate of the likelihood your answer is correct), the number correct should exceed 50% quite dramatically. Thus, a mean correct that fails to exceed 50% by a significant amount indicates subjective overconfidence.
| Number of Students | Number of Questions | Mean Percentage Correct | Mean Percentage Incorrect |
| 18 | 17 | 45.55% | 54.45% |
The Old In-Class Quiz
The old quiz was an in-class quiz in which students are asked to answer a list of true or false questions like the ones below. The list consisted of pairs of questions. Students were asked to first indicate whether each statement in a pair of statements was true or false. Students were then asked to rank their answers for each of the two statements in terms of higher and lower confidence.
| Questions | T/F | Con |
| Leonardo Da Vinci invented the scissors. | ||
| People only use about ten percent of their brains. Scientists are actively searching to discover how to unlock the rest of the brain's potential. |
They mark their answers in the "T/F" column. After answering the questions students are asked to indicate with a "c" in the "con" column, the half of the question set for which they are more subjectively confident in their true/false answers.
Truth and Subjective Confidence Quiz Scores
| Question Group | Percent Correct | Total Number of Questions |
| Higher Confidence | 43% | 138 |
| Lower Confidence | 57% | 138 |