Tracking for Success

At a recent American Enterprise Institute (AEI) conference on “Increasing Accountability in American Higher Education,” panelists argued that the key to increased postsecondary accountability lies with better tracking-systems for student learning outcomes and increasing use of standardized tests.

“For those of you who don’t know it [a student unit record system] basically is a data system maintained at the state or system level which contains one record per student containing information about enrollments, behaviors, and so on,” explained Peter Ewell, Vice President of the National Center for Higher Education Management Systems (NCHEMS).

“There are forty-eight systems in forty-two states; that means that eight states do not have student unit record systems, but they’re mostly small so we cover about 81 percent of the enrollment in [the] country through these kinds of systems,” he said. “Twelve of them contain data on independent institutions, which is a relatively new development and has happened quite quickly.”

He continued, saying that each of these systems “can produce longitudinal data for tracking students” or “creating graduation rates,” while “about half” have “course-level detail” allowing institutions to determine “what courses seem to lead to what patterns of success.”

Dr. Jeffrey Steedle, a measurement scientist at the Council for Aid to Education, mentioned one example of how tracking data led to curricular changes at some colleges.

“And what this pattern has resulted in several schools, we see schools doing things like changing their course credit model to allow for more opportunities to take challenging courses,” he said, continuing, “we see things like the development of first-year seminars that are focusing on critical-thinking and writing skills. We see biweekly professional development on teaching critical thinking skills and additional writing course, writing requirements [in] every major.”

Ewell said that in order to continue data reforms he would like to see more integration of these systems and to “…link state unit record systems together within a region so [that] you can track mobility as students move from state to state and track, essentially, students into the workforce or from K12…”+

The panelists did not discuss the anti-testing movement, which is active in both K-12 and at postsecondary institutions. For example, in the higher ed sector, FairTest advocates for universities and colleges around the U.S. to adopt test-optional admissions policies. “We place special emphasis on eliminating the racial, class, gender, and cultural barriers to equal opportunity posed by standardized tests, and preventing their damage to the quality of education,” states the FairTest website.

The Council for Aid to Education (CAE), which employs Dr. Steedle, focuses on “improving quality and access in higher education” and considers its Collegiate Learning Assessment (CLA) “central to that focus, a national effort to assess the quality of undergraduate education by directly measuring student learning outcomes,” according to the CAE website. The site also states that “[t]o date, over 400 institutions and 165,000 students have participated in the CLA…”

In a January 2007 FairTest newsletter, former University of Nebraska-Lincoln associate professor Chris W. Gallagher condemned the CLA as “solidifying the testing culture in higher education.” He writes,

“Piloting of the CLA has raised a host of concerns, including potential “motivation bias” (and the specter of students sabotaging the results of what is for them a meaningless exercise); the small samples (typically 100 freshmen and 100 seniors); the quality of the prompts (which for example ask students to pretend to write a memo to a fictional boss about buying a private plane); the giving over of instructional time to administer the exam; and the unfairness inherent in counting the results of the exam in courses that don’t teach the content. (This last is one proposed solution to the motivation bias problem; another is to award an iPod to the highest scorer.) And yet, the CLA and other outcomes tests are gaining a foothold in institutions across the country and solidifying the testing culture in higher education.”

Professor Gallagher is now the director of writing programs at Northeastern University.

Not surprisingly, Dr. Steedle said at the conference that he didn’t “think it should be quite as controversial to administer these tests as it is” and expects to “see growth in testing” as the others begin to accept scores’ predictive reliability and recognize “positive changes” that administrative tests have effected in schools to date.

Bethany Stotts is a staff writer at Accuracy in Academia.