Flaws inherent to direct comparisons between CC-student and university-student success

I experienced a visceral and negative reaction to this blog and particularly to this ridiculous statement, “Choosing a two-year college could actually harm students' long-term prospects.” The blog cites (and misinterprets) an excellent study on the topic. The study itself has some potential flaws as well.

First, I’ll discuss the study, then I’ll comment on the blog.

This excellent study by Harvard’s Bridget Terry Long and UC Davis’ Michal Kurlaender starts with the simple fact: Ohio students who begin transfer programs at CCs don’t earn as many bachelor’s degrees as students who start at universities. They attempt to decide whether CC students fail because CCs do a poor job of educating students, or because external risk factors harm students’ chances. They conclude that the former is at fault, harming students’ chances by 14.5%. I humbly disagree.

Retort #1: Long and Kurlaender define ‘success’ too narrowly.

This paper identifies ‘success’ too narrowly for the CC reality. The only possible ‘success’ in this paper is completed bachelor’s degree attainment, therefore relegating any other outcome as “failure.” At a university, success and failure is binary – graduate or don’t. But it isn’t that simple at a CC. Indeed, many positive outcomes for students would be classified as failures by this study. Students can change majors and switch from transfer to career/technical programs. Students can achieve meaningful careers after completing just a “transfer associate degree.” And short of any degree or certificate, students can secure living wage jobs after obtaining high-demand skills.

My tech guy at work is an example of the last ‘positive failure’. He is several classes short of finishing his Information Technology degree. He works full time for my CC’s IT department and make $60k per year. Conversely, the guy that made my latte this morning has a master’s degree in sociology. We could argue about which of these two made wiser academic choices, but I'd be wary about classifying my tech guy as a CC failure.

Certainly, bachelors degree attainment signifies a successful outcome for a CC transfer student. But it’s not the only potential successful outcome.

Retort #2: Long and Kurlaender fail to adequately consider all risk factors which disproportionately affect CC students.

In their paper, Long and Kurlaender acknowledge that selection bias (apples to oranges) is a problem with any study that attempts to compare CC and university students.

“[S]imple comparisons between two-year and four-year students, which suggest students who initially enroll at a community college do far worse, should be treated with caution. Additionally, unobservable differences between students appear to be important.”

As compared to university students, CC students tend to have increased risk factors which preclude them from finishing degrees: low income, first generation, full time jobs, poor academic preparation, etc. This study does a better job than most papers of its type, since it factors, “degree intent information from the college application and details about family income, high school preparation and achievement, and high school type.” It fails to adequately address additional risk factors including: working full time, having dependents, being a parent, attending part-time.

The study attempts to sidestep this research problem by comparing only CC students and “non-selective university” students, arguing that the difference between these students is negligible.

“As discussed above, previous empirical work suggest that community college students enter postsecondary schooling with lower academic credentials on average than their counterparts at four-year institutions (as measured by high school grade point average and ACT scores). However, the differences are not large between students at nonselective four-year universities in Ohio and those at community colleges. Thus, for most of our analysis we focus more explicitly on students at nonselective, four-year institutions as the main comparison group to community college students.”

Perhaps Ohio CC and non-selective university students are similar enough to be considered statistically identical. But in Washington state, such an assumption defies logic. And CC students have been found to be hugely different in this study of Boston’s educational system: “61% of community college enrollees were required to take developmental courses compared to… 25% of enrollees in 4-year institutions.”

Again, the researchers go to great lengths to adjust for these biases, but to a certain extent, we’re still comparing apples to oranges.

Retort #3: The blog entry shows clear design bias; it exaggerates the researchers’ findings and fails to incorporate the researchers’ cautions.

Here’s the ridiculous statement that set me off: “Choosing a two-year college could actually harm students' long-term prospects.” But never fear, by following the blogger’s advice, “students can take advantage of the low-cost classes at community colleges and still make it through to a prestigious degree.” The blogger raises the bar on worthwhile education; a mere bachelor’s degree isn’t enough to avoid failure. To be successful, one must earn a “prestigious degree” that will allow CC students to handle, “upper-class assignments at universities.”

Upper-class indeed.

The researchers warn in the abstract, “The results suggest that straightforward OLS (ordinary least squared) estimates are significantly biased.” In other words, suggesting that CCs harm student prospects is unwarranted and absurd.

Perhaps the author would have been better served by a less prestigious, more practical degree.