RESEARCH ON STUDENT LEARNING

Andrew Ahlgren

Several Project 2061 documents deal with this issue, including:

To GEOTIMES:

Martha Schwartz makes some insightful points in her letter to the editor, concerning the new California science standards (Geotimes, February 1999; see also "Political Scene," October 1998). Before commenting on issues that underlie the controversy, I would like to make four technical points about Dr. Schwartz's letter:

First, although she reports that the American Federation of Teachers rated the new California standards highly, the AFT judged documents only on their specificity and clarity, not on their wisdom or feasibility.

Second, California's current standards document contains a substantial layer of reasonable goals, many of them borrowed from the AAAS Project 2061's Benchmarks for Science Literacy and the National Research Council's National Science Education Standards.

Third, the issues of goals and instructional methods, although often bundled in public discussion of "the reform movement," are nonetheless separable. Advocates of realistic goals may or may not also be advocates of more classroom inquiry.

Fourth, Dr. Schwartz asserts that the new California standards are "content-driven, clear, coherent, and specific." They are undeniably "content-driven" and satisfactorily "clear and specific," but their "coherence" has never been demonstrated and is in some doubt.

Dr. Schwartz is correct that the Benchmarks aim at a coherent body of basic knowledge and skills (carefully developed over seven years) that one day would be achievable by all students. In the long run, as she says, additional goals will have to be identified for students with special interest or ability in science. For the present, however, the Benchmarks are still ambitious: College professors have told us that their incoming students (including majors) are only at about the grade 6-8 level.

The quarrel between those advocating "tougher standards" and those calling for "better understanding" is largely based on different underlying beliefs about how well students learn now (or perhaps did in some imagined past). All sides in the debate want "higher standards." For some, that means increasing the amount and apparent difficulty of what students are expected to learn. For others, it means demanding that students can make sense of and use the important (if less numerous) ideas they study. The "tougher standards" group believes that many students, when properly motivated and taught, obviously learn a lot of science; so they argue that higher expectations will be effective in improving achievement of all students. The "better understanding" group believes that much of what is taught is unacceptably superficial and poorly learned--so they argue for more attention to the most important facts and principles.

The facts about student achievement should be settled by empirical studies of what students learn. Although published research on science learning typically finds that even good students absorb principles far less well than is commonly believed, such academic research has received little attention. Perhaps the best that the gloomy research findings can do is stimulate educators to investigate their own students' understanding more thoroughly--yet this approach is impractical in most classroom settings.

An example of a revealing interview question is: "Why are fish fossils found on the tops of mountains?" Interviewers have to listen conscientiously, being careful not to put words in the student's mouth and not to accept easily memorized slogans before probing for what the student actually knows. For example, many students will say the fish fossils mean the mountain was once under water. Competent reviewers will then ask why those mountains are not under water now. (It's surprising to hear so many students say that the water receded, rather than that the mountains rose.)

Similarly, the effects of making standards tougher should be explored empirically, not left to philosophy. Advocates of "tougher standards" believe that students who do not attain them all will at least learn more than they did before. Advocates of 'better understanding" fear that tougher standards will make instruction even more superficial and will confuse many students more than they are now--with the result that they will learn less than before.

But these differences do not have to be settled before a practical compromise can be worked out. I propose that the California commissioners simply agree to assure that students achieve at least the Benchmarks. When students demonstrate their mastery of the Benchmarks, teachers in each grade range could go considerably beyond them. Since the commissioners believe that achieving the Benchmarks will be relatively easy, they should be happy to accept such a deal. The immediate payoff for California would be that the scientific organizations would stay off their backs during the trial, and might even apologize for their complaints if, in fact, the trial turns out well.

--Andrew Ahlgren, Associate Director, AAAS Project 2061, Washington, D.C.

Back to top

Serious Underlying Issues in Research and Learning

Part II: Serious Underlying Issues

We believe that Part I: REFUTATION above relieves readers from taking the previous testimony very seriously. But some arguments made in that testimony do reflect serious issues for science education. Below we formulate the issues as:

1. To which students are standards intended to apply?
2. How well do students learn science content now?
3. How limited are students in what science content they can learn?
4. What are the effects of an excess of content beyond what a student can learn?
5. What is an appropriate balance between help and demand in getting students to learn more?

1. To which students are standards intended to apply?

Clearly some students can learn science better than others. Standards for what students are asked to learn can be written for the top students, recognizing that most students would learn less than that. Or standards could be written for the least able students, acknowledging that most would learn more than that. Or, standards could be written for average students, expecting that most students would learn more or less than that. The current draft of California science standards is a useful case study. We have not found out yet what the target student population is for them, but as best we can infer from the number and difficulty of content items, they are intended for the very top, science-specializing students--the sort of students who would eventually appear in college science courses. (One draft of the California standards, commendably, demanded at the high-school level an additional layer for science-specializing students; the approved version does not.)

However, the California standards have also pushed a great deal of high-school-level content down into the lower grades. Some of that content could be learned with some understanding at those grade levels by very good students, raising the question of whether there should be an advanced layer of content specified for those grades as well. A more advanced layer of content can in principle be saved for the end of the curriculum as separate-track courses, or laced throughout the curriculum as enrichment. Moreover, we believe that some of the early-grades content has been pushed too low to be understood by hardly any students and should be moved to higher grades. An example of facts without understanding: students can recite a lot of facts about the parts of cells without understanding the fundamental idea that organisms are composed of cells; students typically consider cells to be extra occupants of organisms that perform special functions (like blood cells do). Of course you can teach students to say "organisms are composed of cells," but interviewing them about what that means usually shows their lack of understanding it.

In our own work, beginning with Science for All Americans and Benchmarks for Science Literacy, we have tried to specify a basic science literacy that almost all students, regardless of whether they plan to specialize in a technical field or even go beyond high school, might reasonably be expected to learn in good educational situations. (We have not called our recommendations "standards.") This leaves for others the task of deciding what to ask beyond the benchmarks. We hope that those who attempt additional layers, whether in high-school or in lower grades, will build on the benchmarks as a base. A necessary caveat here is that at present most college graduates fall short of the high-school benchmarks. We are working to improve future curricula to ameliorate that situation.

The main point we want to make for this issue is that almost every statement made, by us or anyone else, has to make clear who the standards are for. Many of the issues discussed in general terms below could be expanded to take into account significant differences among students.

2. How well do students learn science content now?

We would answer "poorly." There is ample evidence from formal research and large-scale testing programs that students are not learning as much as we would like them to. But we think that research evidence goes beyond that, showing that students understand considerably less. Certainly they learn less than most teachers, parents, and policy-makers believe--indeed, less than these audiences are willing to believe. It would be too upsetting to learn that our vast enterprise in science education is achieving so little for all but the very best students--and not all of those, either. (Advanced Placement classes are hardly free of surface articulateness masking deficits in understanding.)

Some people believe there is widespread success in student learning of science because they know students who can say scientific-sounding things. But, as was said earlier, poor understanding of science content is often masked by students' ability to recite the right words. Careful interviewing of students about their knowledge is necessary to show how shallow their understanding is. (We claimed earlier that the research shows that college students have pretty much the same science misconceptions as elementary school students except that they express them with larger vocabularies.) So the seemingly negative findings of published research are sometimes dismissed--out of hand, as in the previous testimony to the subcommittee.

Since the extent of student understanding is measurable, not a matter of philosophy or personal judgment, the issue ought to be resolved empirically. We have been trying to figure out what kind of evidence the doubters would find convincing. We currently believe that the most convincing evidence is observing the deficit of understanding in one's own students. (Other students are sort of unknowns and who knows whether other teachers are doing as good a job of instruction as oneself.) It is essential, of course, that the interviewer decides beforehand that the questions are answerable by his or her students. Because interviewing takes some skill and restraint to avoid putting words in students' mouths, we will be glad to point interested faculty to descriptions in the research literature of what credible interviews are like.

Perhaps the chief implication for this issue is that the National Science Foundation should greatly increase its grant funds for good research on children's learning! (Or the National Research Council could take on such research itself as part of their public service in illuminating major issues of national importance through sound research.) We have come to believe that the design of such research should involve proponents from both sides of the argument to ensure that the results will be believed.

3. How limited are students in what science content they can learn?

The answer to this question of course depends on the answers to Questions 1 and 2, and it is a much harder question to answer. What if all children came to school well prepared and motivated, all teachers were knowledgeable in the subjects, adequate resources were available to schools, and the characteristics of good instruction were already understood? Wouldn't that make possible student achievement considerably greater than we observe now? Greater for sure; but how much greater is an open question. We tried to consider that in formulating Science for All Americans and Benchmarks for Science Literacy in the first place. (It is important to note, however, that currently most college graduates do not come up to benchmarks.)

An underlying assumption in the previous testimony is that students, when properly taught and motivated (as his own students would be), can learn lots of science. Because this discussion is about science education, there is a special obligation to base arguments on evidence. We suggest again that he interview some of his own students six months after they have completed a course. We predict that he will be shocked and appalled by how little impact he has made on his students' persistent misconceptions in biology. We even hope that, with his own misconceptions thus dispelled, he might wish to change much of his testimony about how easily students can learn.

Another pronouncement in the previous testimony is best taken a sentence at a time. The first is "Learning follows from instruction, after all." This is true only if it means students are more likely to know something after they have been instructed in it. But it surely is not true that learning necessarily follows from instruction. (The equivalence implied in the previous testimony between instruction and learning is a major weakness that is pointed out several times in this review.) Recognition that learning does not necessarily follow from instruction should make educators conscientious in trying to dispel misconceptions--and interest them in assessing students after instruction on how well they have actually understood and retained the concepts.

The second sentence begins, "The fact that children have misconceptions prior to instruction should not be surprising…" Although it's true that misconceptions should not be surprising, educators are typically surprised at how extensive and persistent students' misconceptions can be. But the last claim is where the trouble arises: "…nor should it prevent us from attempting to teach them the concepts." That would certainly be true if instructional time were unlimited--but it is not. Instructional time spent on one topic takes away from the time spent on others. Because dispelling misconceptions about important facts and concepts takes more time and effort than has been believed, greater attention has to be given to the priorities of topics. The difficulty of ideas does have to be taken into account in planning an entire curriculum, and rampant misconceptions in a topic might be one consideration in estimating how much time would be required for students to learn it acceptably well.

4. What are the effects of an excess of content beyond what a student can learn?

This is not a simple question. It asks about the extent to which standards much higher than many students are likely to reach will...

The previous testimony included the claim that "The prevailing philosophy among education specialists is that a teacher does harm to students by introducing material that is not developmentally appropriate." This has some truth to it, if material that is not "developmentally appropriate" refers to material whose content most students are not yet able to learn. (Although "prevailing" correctly acknowledges a consensus, "philosophy" is more fairly characterized as "inference from their personal experience and from studies of children's learning.") But prevalence of belief is hardly proof, and solid evidence is generally lacking.

Do tough standards that are over most students' heads maximize their learning? Of course some demands are necessary; the question is how far over their heads is helpful, and can it be too much? Answers other than rhetoric require some empirical knowledge: Do students who cannot learn an entire syllabus typically learn well the part they can understand, or, at some degree of excess, do students get overloaded and confused--and are likely to learn much less than they otherwise could if they were less burdened? We know of no formal research in education that bears directly on this question. The field is therefore left to the clash of philosophical convictions (say, about the motivating power of high standards) and personal experiences (say, in trying to help frustrated students). There may be little improvement in the arguments until appropriate research is funded. But again, proponents of both views should probably be involved in the research design to raise the likelihood that the results will be believed.

5. What is an appropriate balance between help and demand in getting students to learn more?

Quantity of instruction vs. quality of learning is an obvious trade-off. Pro-quantity and pro-quality proponents have to agree that either quantity or quality is useless without the other, yet they compete for instructional time. The more the number of ideas, the less time is available to each one; increase the instructional time for any idea, and less time is available for others. The only questions are what the optimal balance should be and what the highest-priority content should be. (What high-quality instruction is like is, for the moment anyway, another question.) We believe that the difference in emphasis between the camps results from a difference in perceptions: of how well most students know science now, and of how difficult it is to teach them more successfully. We believe that pro-quantity proponents eventually will find out how meager students' knowledge is, distinguish between what is taught and what is learned, recognize how persistent students' misconceptions are, and, therefore, will appreciate the need to reduce unlearnable excess of content in the curriculum, and will adjust their opinions of what an appropriate quantity/quality balance is--giving more importance to quality and, necessarily, less to quantity.

----------------------------------------------------------------------------------------
Dr. Ahlgren is associate director of the AAAS Project 2061 and emeritus professor of education at the University of Minnesota. His undergraduate degrees are in physics and psychology from the University of Chicago, and his graduate degrees include a master's in education from Chicago, a master's in physics from Purdue, and a doctorate in education from Harvard. He was a major participant in the development of Harvard Project Physics, Science for All Americans, and Benchmarks for Science Literacy. Dr. Ahlgren's research on a range of topics in education has appeared in such publications as American Biology Teacher, American Educational Research Journal, Applied Psychological Measurement, Developmental Psychology, Education Leadership, Journal of Research in Mathematics Education, Journal of Research in Science Teaching, Nature, and The Physics Teacher. Ahlgren was also a consultant for UNESCO in developing a national Teaching of Science & Technology Institute in Thailand and was a Fulbright lecturer in science education at the Weizmann Institute of Science in Israel.

Back to top

Some Fundamental Differences

There is, in all this furor about standards, a fundamental difference of opinion. At one, perhaps mythical, extreme, some sincere educators might believe that students should never be asked to take on any challenging task. At the other, perhaps also mythical, extreme, some sincere educators might believe that all students can learn anything if it is demanded of them. Most educators (or more generally, people concerned with education) are well toward the middle between these two extremes, although each side of the middle is prone to caricaturing the other side as the extreme.

On the more optimistic side of the middle, sincere educators believe that aiming standards above almost all students' heads is the best way to stretch them towards high achievement--thus the more challenging the standards, the more beneficial effect they will have. On the more pessimistic side of the middle, sincere educators believe that students should actually understand what they study, which requires taking account of their current level of understanding--thus some topics have to be de-emphasized to allow time for that.

The optimists will reply that lowering standards encourages teachers to teach less and allows students to learn less. The pessimists will reply that aiming standards over students' heads will discourage most of them from learning anything meaningfully--by confusing most students and forcing teachers to move hastily through an excessive amount of material, accepting superficial evidence of learning (such as memorizing technical terms).

A premise of the optimists is that many teachers do know how to teach science well (or that someone who can train them does know how) and that many students are now learning science well. If those premises were valid, even pessimists would admit that increased demand might well produce increased achievement.

The pessimists' premise is that teachers generally do not know how to teach science well and that very few students--now or in the past, in school or in college--actually understand, retain, and are able to use much of what they study. If these premises were valid (as appears to be the case in videotapes of Harvard and MIT graduates responding to standard science questions*), even optimists would concede that increased demand might well produce more confusion and shallowness and diminished achievement.

So there is a double argument that hangs on answers to the questions: (1) What is the current state of teaching and understanding? and (2) Whatever that level is, what kind of changes would improve it? The optimists' answers to those questions would be (1) reasonably good for good teachers and (2) more demands. The pessimists' answers would be (1) poor for most teachers and (2) more concentration on understanding.

Differences about where the problems lie are overlaid by some philosophical differences. Pessimists suspect that the optimists really do not care whether most students learn, as long as learning is good for the students who may go on to be scientists (and may appear later in their own classrooms). Optimists suspect that the pessimists are "self-esteem" advocates or "post-modernists" who believe there are no scientific facts, only cultural constructions of what the world is like. We would guess that there are a few proponents of each side that justify those suspicions. But they are few, and the argument would be better built without them.

We need less invective and more facts about how well students learn.

*A Private Universe, available from the Annenberg Science Education Collection, 1-800-LEARNER

Back to top

Towards Ecumenism in the Research Argument

With a lot of acrimony floating around between pro-quantity and pro-quality advocates in the California standards wars, there are several points that are in danger of being lost:

1. Many of the new California standards are pretty good, in the sense of being both important and learnable. Most of these seem to be borrowed from (or at least are consistent with) Benchmarks or NSES. Journalists seem to cite these as examples and ask what is unreasonable about them, as if all the standards were comparably reasonable, and are understandably perplexed by all the opposition to them. The judgment of poor quality for the California standards as a whole arises from the excessively numerous and difficult expectations that have been added to the reasonable ones--and from the placement of many reasonable standards at an unrealistically early grade level.

2. Neither pro-quantity nor pro-quality advocates are bad people. They all want more and better science education for students, and they are willing to fight vigorously against what they believe to be poor thinking on the other side. (For example, being unfamiliar with how to find out what students really think, pro-quantity advocates don't believe that children who can recite in school what the teacher wants to hear actually have very different ideas about how the real world works.) But neither the pro-quantity or pro-quality side believes that the other has credible evidence. Research methods have to be described well enough that the skeptics can find out for themselves. And convincing studies will have to be developed with input from both camps. When either side sees what the facts are, they may convert.

Back to top