Everything below this line is from late 1999.

The article below came to me by a route that would have been impossible just a few short years ago.

On Sunday, Nov. 7, 1999, The New York Times newspaper included a section entitled Education Life. The last article in that section, called Endpaper, addressed a number of concerns that I share regarding the state of education in this country today. I wanted to be able to place a link to that article on my website.

I was unable to find it at the Times' site, but I did an email search on the author, and located his address at Princeton. I sent an email to him stating my desires.

Monday I received a response. He told me that the actual published version was in the hand if the Times, but he sent me an earlier pre-publication draft. I then asked if it was OK for me to use this draft on my website. He wrote, "This is fine with me."

We run from one fad to another in the education field; from phonics-first to whole language and back to phonics; from subject-based to holistic learning; from curriculum-based to child-centered learning; from neighborhood to magnet to charter schools; from old to new math; from English-only to bilingual education to language immersion programs. Fads are common because of the shortage of compelling evidence upon which to anchor educational practices. Instead, education practices swing with the winds of educational philosophies and politics.

It seems that any remedy can be pushed by administrators, politicians and pundits without the slimmest reed of evidence demonstrating that it works. We do not allow this behavior when it comes to prescribing medicine for our children. Instead, the Food and Drug Administration requires scientific testing before new drugs can be prescribed. Why should we treat children's minds differently than their bodies?

With the economy roaring along, crime down, and peace at hand, education ranks as the top concern among voters according to many polls. As a consequence, politicians in both parties are pushing dueling education proposals. Republicans, on the one hand, propose cutting federal regulations, giving students vouchers, and occasionally advocate eliminating the Education Department all together. Democrats, on the other hand, have been protective of the Education Department and propose reducing class size and repairing decrepit schools.

Although it would be a mistake to eliminate the Education Department, the Republicans may be on to something. We typically do not know enough about what works in education to prescribe policy to local schools and teachers. It is rarely acknowledged, however, that local schools -- public and private -- do not know enough either. Instead of eliminating the Education Department, it should be vested with the task of determining which educational reforms work, which don't, and which are the most cost effective. This information should then be widely disseminated, so all concerned parties can make sound decisions.

Understanding what works well in education is a national concern. The federal government is the appropriate branch of government to perform, fund, and evaluate the controlled experiments that are needed to determine the best way to educate students.

In 1999 the federal government spent $14 billion on space exploration, yet less than $300 million on research on how to improve education. With total educational expenditures exceeding $600 billion in 1999, wouldn't it be worthwhile to invest more to figure out how to spend the money more wisely? So far the primary research function of the Education Department has been to collect multipurpose datasets on test scores. Less than 1 percent of the Education Department's budget goes to research, and hardly any of that to conduct and evaluate experiments. If research on education improves the efficiency of schools by only 0.05 percent, it would be worth the entire current federal investment in educational research.
The problem is not insufficient money for research alone. Rather, it is that controlled experiments are rarely done in the education field.

An example of a current thrust in education that cries out for scientific evaluation is the proliferation of computer use in schools. The race to wire schools is on, yet there is no compelling evidence either way about the effectiveness of this strategy. Consider an example: Should putting computers in the classroom be a priority for inner-city fourth graders who read at the first-grade level? Would these students be better served learning the basics first? Or perhaps giving each of these students a lap top is the right approach; the repetition inherent in computer-based training, as well as the magnetism of computer technology, could enable such children to read faster than conventional classroom methods, especially in an overcrowded classroom. There is only one sure fire way to determine the best strategy: conduct a randomized experiment. An experiment would involve randomly selecting schools or classrooms that would not receive computer equipment, and some that would. Researchers would then follow-up on the performance of the students who went to school under these different conditions.

Randomly denying some children computers is difficult. But this does not create a fatal ethical dilemma for two reasons. First, some children are going to be denied computers anyway since we are not spending enough money to computerize all classrooms at once. Indeed, it could be argued that randomly distributing the computers to schools is the fairest way to allocate resources in view of this constraint. Second, and more fundamentally, without conducting an experiment, it is not clear which students have been harmed -- those who were given the computers or those who were not.
The Tennessee STAR experiment, which randomly assigned 11,600 K-3 students and their teachers to regular-size classes of about 22 students, regular size classes with a teacher aide, and small classes of about 15 students in the late 1980s demonstrated that large-scale experiments are feasible in education. The experiment indicated that attending a small class increased student test scores, especially for low-income and minority students. The experimental results, however, also suggested that teacher aides yield little academic benefit, and that teachers' educational attainment or experience was only weakly related to student achievement. The Tennessee experiment is far more compelling than all the non-experimental studies that have been done on these issues combined.

We can learn from the Tennessee experiment and design additional experiments to answer more questions about class size. For example, experiments could be conducted to determine if reducing class size at other grade levels, or for fewer years, or by small amounts yield similar benefits. And class-size experiments could be conducted by varying class size across schools, rather than across classes within schools, to determine if there are peer group effects that cut across classes.
Other initiatives in the education field are ripe for experimental evaluation as well. For example, experiments could be designed to evaluate the effectiveness of various teaching methods, teacher selection procedures, tutoring, pre-school education, school-to-work transition programs, summer school, year-round schooling, extended-day schooling, ability grouping, school uniforms, and school size. Another hot area where little research has been conducted involves standardized tests. Experiments could be conducted to determine if teachers teach to the test, and whether that is a bad thing or not, when standardized tests are given more regularly.

A handful of experiments funded by private concerns and foundations are already underway on school vouchers, but they are limited to small geographic areas. These experiments randomly select low-income students who have applied for a voucher to receive around $1,500 if they attend a private school. An experiment in Cleveland found that many parents do not send their children to private school despite being offered a voucher to do so, whereas another experiment just underway in New York found parents much more willing to switch their children to private school if they were offered a voucher. A few years from now, the New York experiment should yield valuable insight on whether students score higher on standardized tests by switching to a private school. It would be informative to perform the same type of experiment in reverse: parents of children in private schools could be randomly selected and offered $1,500 if they send their child to the local public school. The students' achievement could then be compared to the control group that was not selected to see if private schools really do offer advantages.

A tricky issue in the education field is how to measure success, regardless of whether experiments are performed. A medical treatment works if a patient survives or recovers from an ailment. Standardized tests are the most common outcome measure in education, but they are potentially manipulable and subjective. Personally, I would favor casting a broad net, including tests, school completion rates, and outcomes such as success in the labor market, crime, dependency and good citizenship. Many of these outcomes take time to measure, which is why standardized tests are so popular, but tests scores could go up or down without any real change in the outcomes we ultimately care about. Moreover, time is not a barrier to studying medical interventions designed to increase life expectancy. Experiments are not always feasible in education, but there are many more that can and should be conducted. Funds should be provided for long term follow-up of the experiments that already have been conducted. Moreover, we learn a tremendous amount from experiments that is helpful in designing nonexperimental evaluations. For example, the Tennessee STAR experiment indicated that the main benefit of attending a smaller class occurs the very first year a student is in a small class. This in turn calls into question the so called "value added" specification of looking at test score gains, which is widely used in nonexperimental evaluations, because the main gain appears to occur the initial year students are in small classes.

Given the clear benefits of experiments, one may reasonably ask why more are not done. There are several explanations. First, vested interests often oppose experiments. Advocates of a policy are convinced it works and do not want to be delayed by waiting for the evidence. Opponents fear the experiment might show that the policy works, so they too do not want compelling evidence. Second, education schools have focused narrowly on providing their students with teaching tools and educational processes, with little effort devoted to serious empirical study of whether those tools and processes are effective. Third, many fear that experiments on school children are infeasible or inhumane. But these concerns can be allayed; for example, in the Tennessee experiment no one was made worse off because students were either assigned to a class with the normal number of students or a smaller class.

As a practical matter, how should the Department of Education be reinvented to promote controlled experiments? DOE should establish an Office of the Assistant Secretary for Policy, Evaluation and Research (ASPER) that reports directly to the Secretary, and has a budget to fund experiments. Additionally, to change the overall ethic concerning experiments and leverage its influence, the Education Department could partner with other agencies, foundations and research universities to conduct experiments. Researchers should be involved from the beginning, and should agree on the research design ex ante. A similar strategy worked in the Labor Department in the 1970s, and led to an entire industry that has conducted invaluable experiments on job training programs.

The U.S. education system is not so flawed that we need to take radical, untested steps to try to improve it. Standardized test scores are up, and a higher proportion of students than ever before are graduating from high school and going on to college. And this good news has occurred despite the fact that an alarmingly high proportion of students grow up in poverty. We should be careful not to throw the baby out with bath water when it comes to educational reform. Convincing experimental evidence should be required before we make wholesale changes in the education system.

The public's current focus on education may not be a bad thing. Our nation's prosperity hinges on our education system. Human capital -- that is, the education, skills and experience embodied in the workforce -- accounts for an estimated three-quarters of national income. Any step that can improve our school system therefore can have major implications for America's long-term economic well being. Just as the FDA has created the greatest body of medical knowledge that the world has ever seen, the Education Department should lead the way to expand the knowledge base in education.

Alan B. Krueger, a former Chief Economist of the U.S. Department of Labor, is the Bendheim Professor of Economics and Public Affairs at Princeton University. He is currently writing a book entitled Education Matters.