Specialty Rankings
Each fall since 1991, I and other senior intellectual property professors [1] have received ballots from U.S. News & World Report (US News). 1996 ballots listed most U.S. law schools and asked respondents to select up to ten. [2]
The results of such polls are published in a spring issue that gets a great deal of attention. It has been flattering, but not surprising, to see Franklin Pierce Law Center consistently appearing among the top five -- indicating that senior intellectual property faculty at some schools are aware of its intensive and extensive coverage. However, when Pierce Law's ranking slipped in March 1996, a few students and alumni were upset. [3]
That and a related article in a local newspaper [4] prompted a serious look at the 1996 data: [5]
School --- Votes
1. George Washington --- 28
2. Columbia --- 26
3. Stanford --- 22
4. Houston --- 21
5. Pierce Law --- 20
6. Berkeley --- 17
7. NYU --- 16
8. UCLA & Boston University (tied) --- 15
10. John Marshall --- 13
Total votes --- 178
US News has since withheld data and details about its methods, but many aspects of these rankings shed light on their remarkable lack of objective significance. Without exhausting the possibilities, consider, for example:
- Perhaps reflecting honest ignorance about other schools' programs, total 1996 votes for the top ten programs represent only 39% of those that could have been cast. [6] Given how quickly program tallies dwindle, average responding professors may have cast no more than half of their votes.
- Second, most rankings differ on the basis of one or two raw votes. For example, the three-vote difference between Pierce Law (20) and Berkeley (17) might have been found significant after formal analysis, but that is highly improbable.
- Also, only about 60% of respondents recognized GW's IP program -- one of the oldest. This, too, sparks wonder about the information available to even senior law professors on others' programs.
I could continue, but most of this is beside the point: One need not probe deeply to appreciate that it was wholly inappropriate to use data on IP program recognition to support comparative rankings of quality. Thus, the answer to "What did US News 1996 IP program rankings mean?" is clearly: Not much!
TopPostscript. This comment was originally written in 1996. Although I was glad to see Pierce Law top the rankings for several years, during that time my views remained unchanged. Indeed, after being refused raw data since 1996 and having had more time to consider the issues, my skepticism has deepened. For a more recent discussion, see Jon R. Cavicchi and Thomas G. Field, Jr., Evaluating IP Programs (ipFrontline, Feb. 2007)
Notes
1. Association of American Law Schools, Directory of Law Teachers 1995-96, at 1129.
Twenty of the potential 1995 respondents did not teach an "intellectual property" course or seminar during the previous year, but they were nevertheless polled. Conversely, a colleague with about 20 years experience was not polled because he did not update his AALS data sheet; another with nine years experience didn't quite qualify -- and, as shown above, only two votes separated 3d and 5th place!
[2009 addendum. Since this article was originally written (1996), senior professors have the same chance of being polled as their more numerous junior colleagues. In the Fall of 2008, neither I (with 35 years experience) nor any of several IP colleagues received a ballot. That Pierce nevertheless remained in the top ten seems remarkable.]
2. That the number of schools potentially selected was increased is reflected in a 2000 US News statement (its sole disclosure of methodology for specialty rankings): "Specialty rankings: Legal educators identified the 15 schools with the best offerings. The 10 schools receiving the most nominations appear. Schools verified their offerings."
In 1995, respondents were not asked to rank their choices in any way. Yet, the law specialty listings in 1996 America's Best Graduate Schools was preceded by: "Programs ranked best... by faculty experts in these fields." [Emphasis added.]
3. Distress when rankings fall is not surprising. Yet, some apparently take such rankings far too seriously. For example, we interviewed an administrator who was reportedly fired because his previous school dropped but one notch in general rankings!
4. Scott Calvert, Law school guards its niche, Concord (NH) Monitor, May 17, 1996, at 1. Calvert's article shows how even well seasoned faculty may be uninformed about programs at other schools. Intellectual property is a broad subset of the law; see, e.g., Field, The Practical and Legal Fundamentals of Intellectual Property, online here. Copyright faculty, for example, have little reason to know about programs that emphasize patents or trademarks.
5. Bob Morris of US News provided the data to Dan Cahoy, then a Pierce Law student (by phone, May 21, 1996).
6. Only 46 ballots were returned (despite phone follow-ups). Because each professor could have selected up to ten schools, 460 votes were possible -- compared to 178; id.
For further information, see
- Index to Law School Rankings (Internet Legal Resource Guide)
- Rankings Caution and Controversy (University of Illinois at Urbana-Champaign)
