• Home
  • Blogs
  • Ellen Dannin
  • Ranking Colleges – US News & World Report -v- Washington Monthly -v- the U.S. Department of Education

Ranking Colleges – US News & World Report -v- Washington Monthly -v- the U.S. Department of Education

04 Feb 2015 2:06 PM | Deleted user

Ellen Dannin edannin@frii.com

Ellen Dannin writes in the areas of labor and employment, privatization, law, and education.


Summary: The Department of Education is creating its own college ranking system based on access, affordability and performance. How will this college ranking newcomer affect two vintage ranking systems – US News & World Report and Washington Monthly?


Both US News & World Report and Washington Monthly rank schools, but their ranking methods have very different effects on who goes to which college.


How the College Ranking Game Is Played


U.S. News and World Report (USNWR) ranks many things – hospitals, health, money, careers, travel, and cars – but is best known for its college rankings. Indeed, since 1983, when the US News and World Report college ranking system was created, it has heavily influenced who applies to which schools and the behavior of colleges.


For example, USNWR’s rankings have created financial incentives for colleges to offer “merit” scholarships to students who can actually afford to go to college. The goal is to persuade students whose grade point averages and test scores will raise a college’s rank to accept the college’s offer.


These merit scholarships have many pernicious effects. Giving scholarships to students who can afford to attend a college means less money to give applicants need-based scholarships. That financial gap is met by raising tuition. As this cycle continues, the cost of college increases.


The Washington Monthly’s Alternate Ranking System


In 2001, the Washington Monthly (WM) began responding to USNWR’s college rankings by exposing USWNR’s pernicious effects on college education. (More on that subject at this link.)* WM has repeatedly pointed out that USNWR’s rankings fail to assess the quality of education offered by the colleges it assesses.


The public should react to these failures by disqualifying USNWR from continuing to engage in this game that benefits only USNWR. Indeed, given that private and public colleges depend on public money for funding, the public deserves to know how that money is being spent.


WM has repeatedly pointed out that


there are no numbers, no studies, no objective measurements. In a Washington Monthly survey of 50 randomly selected research-university Web sites, only 12 percent clearly posted the six-year graduation rate, the most basic statistical measure of effectiveness. Even fewer offered information about student satisfaction with teaching.


Without solid information on what they will learn, students must make choices based on geography, particular programs, or reputation. . . .


In 2001, in addition to WMs initial essay criticizing USNWR’s rankings, WM created lists of colleges that illustrated the article’s criticism. The lists included  the“Top 20” and “Worst 20,” “How U.S. News’s Top Universities Fared,” “How U.S. News’s Top Liberal Arts Schools Fared,” colleges’ “Peace Corps Enrollment,” and “ROTC Enrollment.”


By 2009, Washington Monthly’s alternative ranking system was in place but has continued to evolve. The basic components of WM’s college ranking system are evidence of:


(1) community service, based on alumni currently serving in the Peace Corps or Reserve Officer Training Corps, relative to the school’s size,


(2) amount of federal work-study money spent on community service projects, based on quality and quantity of research; and


(3) promotion of social mobility, as shown by the school’s commitment to educating lower-income children, based on percentage of Pell Grants.


How Does USNWR Differ from WM?


USNWR’s college rankings are elitist and backward looking. Its focus is on what applicants brought with them from high school – for example, grade point averages and test scores, such as the ACT or SAT. That sort of ranking method takes students as they are and does little to promote a more just and equitable society.


In contrast, WM ranks colleges based on evidence of students’ future promise and on colleges’ support for students who show promise. Put another way, the WM rankings essentially say, “If this sort of student is accepted and is given appropriate support, they will be an asset to our society.”


Or, more concretely,

Imagine, then, what would happen if thousands of schools were suddenly motivated to try to boost their scores on The Washington Monthly College Rankings. They'd start enrolling greater numbers of low-income students and putting great effort into ensuring that these students graduate. They'd encourage more of their students to join the Peace Corps or the military. They'd intensify their focus on producing more Ph.D. graduates in science and engineering. And as a result, we all would benefit from a wealthier, freer, more vibrant, and democratic country


Going to College?


A year ago, the US Department of Education announced that it would create its own college rating system. The question is whether that rating system will follow the USNWR model or that of Washington Monthly?


Meanwhile, the Department of Education has initiated a public discussion on revamping the college admissions process. The Department of Education has set as its goals for higher education: access, affordability, and performance.


In fact, Washington Monthly readers will find much that is familiar in the Education Department's higher education goals. In fact, they appear to be consistent with the standards long used in the WM annual college guides and rankings.


Unlike US News and World Report and similar guides, this one asks not what colleges can do for you, but what colleges are doing for the country. Are they educating low-income students, or just catering to the affluent? Are they improving the quality of their teaching, or ducking accountability for it? Are they trying to become more productive – and if so, why is average tuition rising faster than health care costs? Every year we lavish billions of tax dollars and other public benefits on institutions of higher learning. This guide asks: Are we getting the most for our money?


For decades now, the Washington Monthly has published its own college rankings, while  pointing out obvious problems with USNWR's methodology:


Unfortunately, the highly influential US News & World Report annual guide to "America's Best Colleges" pays scant attention to measures of learning or good educational practices, even as it neatly ranks colleges in long lists of the sort that Americans love. It could be a major part of the solution; instead, it's a problem.


US News' rankings primarily register a school's wealth, reputation, and the achievement of the high-school students it admits.


USNWR has long been aware that it uses a deeply flawed system for assessing colleges’ educational quality. In 1997, USNWR commissioned a study of its methodology by an outside organization, the National Opinion Research Center. The NORC study found, "The principal weakness of the current approach is that the weights used to combine the various measures into an overall rating lack any defensible empirical or theoretical basis."


In contrast, the Washington Monthly based its college ranking system on goals that are likely to provide benefits beyond the students who receive that education:


  • Access, such as the percentage of students receiving Pell grants;
  • Affordability, such as average tuition, scholarships, and loan debt; and
  • Outcomes, such as graduation and transfer rates, graduate earnings, and advanced degrees of college graduates.


Comparing Washington Monthly’s metrics on colleges with those of the U.S. Education Department shows considerable overlap. Both advocate strengthening the performance of colleges and universities in promoting access, ensuring affordability, and improving student outcomes through the design of the college ratings system.


Both aim to:


(1) help colleges and universities measure, benchmark, and continue to improve across the shared principles of access, affordability, and outcomes;


(2) help students and families make informed choices about searching for and selecting a college; and


(3) enable the incentives and accountability structure in the federal student aid program to be properly aligned to these key principles.


(More information on the Education Department’s development of its rating system can be found on its Homeroom blog.)


Who Owns a College Degree?


Who benefits from that education and training? Obviously, the people who get the training and their families do, but it doesn't stop there.


We need to think about who should pay for a college education and why. After all, a college education is among the largest expenses any of us will pay for, yet the benefits of education are not solely confined to the student. In fact, we all benefit from having well-educated doctors, construction workers, teachers and more. The number of professions for which a college education is helpful is increasing: For example, these days apprentice union construction workers receive a large part of their training in college classes.


As I wrote in the Taunton Daily Gazette:


In fact, the training required for the construction trades can equal or exceed the years of education required for a bachelor's degree. The lowest amount of training is two years for laborers, while the training for electricians and some other trades is 5 years or more. Electricians' education includes 900 hours of classroom work and 8000 hours of hands-on training. Just to enter the program requires completion of high school work including Algebra and passing an entry test. Even more math is required along the way. A carpenter's apprenticeship requires 4 years training with 640 hours of classroom instruction and 8000 hours of on-the-job training. Many construction workers earn at least an associates' degree during their training.


The USNWR rankings tell a story of elitism and individualism, but it is a flawed story that ignores the many ways we collectively benefit from having an educated citizenry. The USNWR version of the story overlooks the benefits of education that flow through our communities. Since so much of the costs of post-secondary education are borne by the public, the value of the public's investments deserve to be acknowledged.


The Beneficiaries of USNWR Rankings


Who benefits from the USNWR rankings? We don't have all the data to make that judgment, but it is fair to say that USNWRitself - and its ever growing family of rankings - are the major beneficiaries – not the public.


US News and World Report is a multi-platform publisher of . . . annual guidebooks on Best Colleges, Best Graduate Schools, and Best Hospitals. Focusing on Health, Money, Education, Travel, Cars, and Public Service/Opinion, US News has earned a reputation as the leading provider of service news and information that improves the quality of life of its readers. US News and World Report's signature franchises includes its News You Can Use® brand of journalism and its Best series of consumer guides that include rankings of colleges, graduate schools, hospitals, mutual funds, health plans, and more.


For all its experience with rankings, USNWR's history with college rankings has included troubling behavior. One example: The publication has asked school officials to rate their competitor schools. First, the ratings serve no useful purpose, given that schools are regularly inspected in order to retain their accreditation. Second, the ratings were not only highly subjective; they created a temptation for schools to lower a competitor school's rank. That is, "survey respondents may rate down some schools in order to make their own school look better and schools may try to raise their score on the 'rejection rate' factor by encouraging applications from students who have virtually no chance of being admitted."


What Education Is Needed for a Democracy?


Students from families who have not gone to college are likely to need special guidance to understand how to get the most from a college education. On August 22, 2013, the White House presented a summary of education issues to be addressed. Among them was creating a college rating system that would help families compare schools and help taxpayers judge whether federal investments in financial aid and educational grants are worthwhile.


While certainly helpful to college applicants, changing to such a system would, as Senator Elizabeth Warren might put it, leave blood and teeth on the floor. The question is whose blood and teeth.


Publishing school rankings has been a big moneymaker for USNWR. On the other hand, USNWR has been willing to changing its criteria over the years. Some of the changes have been motivated by Washington Monthly’s criticism concerning USNWR college rankings as focusing in the wrong place.


WMs ranking system advocates assessing the quality of education based on outputs – that is, its rankings are based on evidence of what current students achieve as a result of attending a specific school. Among the outputs that WM’s system assesses are number of students receiving need-basedPell grants, actual graduation rates, research expenditures, and Peace Corps and ROTC participation. Each of these factors promotes outcomes that benefit students and their communities.


In contrast, USNWR's ratings use inputs to rank colleges. Those inputs have included factors such as high school grade point averages and admissions test scores. Those factors predict students’ grades, but they do nothing to expand students’ knowledge and skills.


After more than a decade of Washington Monthly’s criticizing USNWR’s for using inputs for its rankings, USNWR on December 24, 2014, the USNWR announced that it was including Pell grants as a factor in its school rankings. The federal Pell Grant Program provides need-based grants to low-income students to promote access to post-secondary education.


USNWR’s announcement stated:


Currently, the US News Best Colleges rankings methodology incorporates outcome measures such as graduation and retention rates. We also use graduation rate performance, which measures the difference between each school's predicted graduation rate – based on characteristics of the incoming class closely linked to college completion, such as test scores and Pell Grants – and its actual graduation rate. These three outcome factors, in total, count for 30 percent of the rankings and are the most heavily weighted indicators in our methodology.


Waiting for the Education Department's College Rating System


If imitation is the best flattery, the Washington Monthly must be feeling flattered. The Education Department’s A New System of College Ratings – Invitation to Comment outlines  characteristics it wants in a college ranking system. In particular, it wants a system that promotes students’ ability to take advantage of educational opportunities on a more equal footing:


The college ratings system has multiple related purposes. A critical purpose of the ratings system is to recognize institutions that are succeeding at expanding access, maintaining affordability, and ensuring strong student outcomes and setting them apart from institutions that need to improve. By shedding light on key measures, the ratings system will support greater accountability and incentivize schools to make greater progress in these areas of shared priorities, especially at serving and graduating low income and first generation students and holding down the cost of college.


USNWR has dominated the college application process for decades. It will be interesting to see how the government's new initiative affects students' decision-making process and how colleges do or do not revise their application processes.


Is a New Information System for College Applicants Enough?


Whatever ranking system emerges from the Education Department's efforts, can a new ranking system overcome the effects of deep inequality in our schools? It will certainly take time to repair the damage done by the deeply corrupt US News & World Report to our education system.


Meanwhile, there are other educational paths that lead to good jobs, without debt. Those opportunities will be examined in a follow-up story.

Employment Policy Research Network (A member-driven project of the Labor and Employment Relations Association)

121 Labor and Employment Relations Bldg.


121 LER Building

504 East Armory Ave.

Champaign, IL 61820


The EPRN began with generous grants from the Rockefeller, Russell Sage, and Ewing Marion Kauffman Foundations


Powered by Wild Apricot. Try our all-in-one platform for easy membership management