The following was a private letter from Gerhard Casper, presidentof Stanford University, to James Fallows, editor of U.S. News &World Report. With the permission of both, it since has entered thepublic domain.

Mr. James Fallows
Editor
U.S. News & World Report
2400 N Street NW
Washington, DC 20037

Dear Mr. Fallows:

     I appreciate that, as the new editor of U.S. News & WorldReport, you have much to do at this moment. However, it isprecisely because you are the new editor that I write to you,personally.

     I emphasize you, because of your demonstrated willingness toexamine journalism in the same way that journalism examines all otherfacets of society. And I say personally because my letter is foryour consideration, and not a letter to the editor for publication.

     My timing also is related to the recent appearance of the annual U.S.News "America's Best Colleges" rankings. As the president of auniversity that is among the top-ranked universities, I hope I have thestanding to persuade you that much about these rankings -particularly their specious formulas and spurious precision - isutterly misleading. I wish I could forego this letter since, after all,the rankings are only another newspaper story. Alas, alumni, foreignnewspapers, and many others do not bring a sense of perspective to thematter.

     I am extremely skeptical that the quality of a university - anymore than the quality of a magazine - can be measuredstatistically. However, even if it can, the producers of the U.S. News rankings remain far from discovering the method. Let me offer as primafacie evidence two great public universities: the University ofMichigan-Ann Arbor and the University of California-Berkeley. Theseclearly are among the very best universities in America - one couldmake a strong argument for either in the top half-dozen. Yet, in thelast three years, the U.S. News formula has assigned them ranks thatlead many readers to infer that they are second rate: Michigan21-24-24, and Berkeley 23-26-27.

     Such movement itself - while perhaps good for generating attentionand sales - corrodes the credibility of these rankings and yourmagazine itself. Universities change very slowly - in many waysmore slowly than even I would like. Yet, the people behind the U.S.News rankings lead readers to believe either that university qualitypops up and down like politicians in polls, or that last year's rankingswere wrong but this year's are right (until, of course, next year'sprove them wrong). What else is one to make of Harvard's being #1 oneyear and #3 the next, or Northwestern's leaping in a single bound from#13 to #9? And it is not just this year. Could Johns Hopkins be the22nd best national university two years ago, the 10th best last year,and the 15th best this year? Which is correct, that Columbia is #9 (twoyears ago), #15 (last year) or #11 (this year)?

     Knowing that universities - and, in most cases, the statistics theysubmit - change little from one year to the next, I can onlyconclude that what are changing are the formulas the magazine's numbermassagers employ. And, indeed, there is marked evidence of that thisyear.

     In the category "Faculty resources," even though few of us hadsignificant changes in our faculty or student numbers, our class sizes,or our finances, the rankings' producers created a mad scramble in rankorder, for example:

     One component of this category, "Student/faculty ratio," changed equallysharply, and not just in rank order but in what the magazine haspresented as absolute numbers. Again, this is with very little changein our student or faculty counts:

     Then there is "Financial resources," where Stanford dropped from #6 to#9, Harvard from #5 to #7. Our resources did not fall; did otherinstitutions' rise so sharply?

     I infer that, in each case, the formulas were simply changed, withnotification to no one, not even your readers, who are left to assumethat some schools have suddenly soared, others precipitously plummeted.

     One place where a change was made openly was, perhaps, the most openlyabsurd. This is the new category "Value added." I quote the magazine:

Researchers have long sought ways to measure the educational value addedby individual colleges. We believe that we have created such anindicator. Developed in consultation with academic experts, it focuseson the difference between a school's predicted graduation rate -based upon the median or average SAT or ACT scores of its students andits educational expenditures per student - and its actualgraduation rate.

     This passage is correct that such a measure has long been sought. However, like the Holy Grail, no one has found it, certainly not the"we" of this passage. The method employed here is, indeed, theapotheosis of the errors of the creators of these ratings: validquestions are answered with invalid formulas and numbers.

     Let me examine an example in "Value added": The California Institute ofTechnology offers a rigorous and demanding curriculum that undeniablyadds great value to its students. Yet, Caltech is crucified for havinga "predicted" graduation rate of 99% and an actual graduation rate of85%. Did it ever occur to the people who created this "measure" thatmany students do not graduate from Caltech precisely because they findCaltech too rigorous and demanding - that is, adding too much value- for them? Caltech could easily meet the "predicted" graduationrate of 99% by offering a cream-puff curriculum and automatic A's. Would that be adding value? How can the people who came up with thisformula defend graduation rate as a measure of value added? And even ifthey could, precisely how do they manage to combine test scores and"education expenditures" - itself a suspect statistic - topredict a graduation rate?

     Were U.S. News, under your leadership, to walk away from thesemisleading rankings, it would be a powerful display of common sense. Ifear, however, that these rankings and their byproducts have become tooattention-catching for that to happen.

     Could there not, though, at least be a move toward greater honesty with,and service to, your readers by moving away from the false precision? Could you not do away with rank ordering and overall scores, thusadmitting that the method is not nearly that precise and that thedifference between #1 and #2 - indeed, between #1 and #10 -may be statistically insignificant? Could you not, instead of tinkeringto "perfect" the weightings and formulas, question the basic premise? Could you not admit that quality may not be truly quantifiable, and thatsome of the data you use are not even truly available (e.g., many highschools do not report whether their graduates are in the top 10% oftheir class)?

     Parents are confused and looking for guidance on the best choice fortheir particular child and the best investment of their hard-earnedmoney. Your demonstrated record gives me hope that you can begin tolead the way away from football-ranking mentality and toward helping toinform, rather than mislead, your readers.

                                            Sincerely,

                                            GerhardCasper




BUILDING 10 * STANFORD, CALIFORNIA 94305-2060 * (415) 723-2481 * FAX (415) 725-6847