The Edge: What Does It Mean if a Big Prize for Student Success Has No Winner?

A foundation was going to award $5 million for an app to improve retention, but it couldn’t find what it was looking for. ADVERTISEMENT [Advertisement]( [logo] [Read this newsletter on the web](. I’m Goldie Blumenstyk, a senior writer at The Chronicle covering innovation in and around academe. Here’s what I’m thinking about this week. What ever happened to that $5-million prize for a student-success app? I never thought it would take more than six years to report to you the results of the College Success Prize [I first covered in 2015](. The delay has a little to do with more-pressing news and, to be candid, my own failure to consistently and insistently follow up. But another part of the story is an unexpected conclusion to this experiment in identifying new ways to promote disadvantaged students’ progress in college. What happened illustrates what can veer off course when an organization launches a splashy contest aimed at finding innovative solutions. Notably, the contest’s sponsors might not be that forthcoming about the results, especially if they fall short of expectations and hype. And that’s too bad, because in the case of the College Success Prize, which was sponsored by the Robin Hood Foundation, the lack of transparency means educators and researchers won’t learn as much as they could from the finalists’ work with community-college students who face academic, financial, and personal challenges in trying to complete their degrees or transfer. To me, that undermines the broad value of the contest. ADVERTISEMENT [Advertisement]( Subscribe to The Chronicle The Chronicle’s award-winning journalism challenges conventional wisdom, holds academic leaders accountable, and empowers you to do your job better — and it’s your support that makes our work possible. [Subscribe Today]( I’m sure that the folks at Robin Hood, a foundation in New York City focused on fighting poverty, had good intentions when they invited organizations to compete for up to $5 million by creating an app that would increase completion rates among students at the City University of New York by at least 15 percentage points above the rate for a control group. Yet once the competition ended, the foundation did not announce the results of the randomized controlled trials with anything near the publicity it had courted in announcing the contest in 2014, or in naming three finalists a year later. Those finalists were Beyond 12, an [organization that promotes student success through online coaching]( and whose founder I recently interviewed for the[Innovation That Matters podcast series]( and another nonprofit called Kinvolved, which is known primarily for its work in K-12 settings fostering student engagement by using text messages to encourage class attendance. The third finalist, the company EAB, ultimately decided not to compete. The experiment ended in August 2018. But Robin Hood never issued a news release announcing the outcome. None of the $5 million was ever awarded. In September 2019, the foundation did post on its website [a two-paragraph statement on “prize results,”]( explaining that neither of the finalists’ interventions had raised graduation rates among students in remedial courses “by an amount statistically significant compared to the control group.” But how would anyone know to look there to find that out? And then, that’s it? Unfortunately, despite an earlier commitment, the foundation has not made public the complete third-party evaluation of the work. A Robin Hood spokesman said that commitment didn’t necessarily include publicizing a proprietary document. "The primary purpose of the report was to facilitate an internal decision in determining a winner of the prize competition, or in this case, to declare there was no winner," the spokesman said. I’ve seen the 59-page evaluation, dated June 2019, under the condition that I can summarize the findings but not publish the document. I’m glad to have that chance, although I’m pretty sure that education researchers better trained to dive into the data (and all those regression analyses) could identify some deeper and potentially useful takeaways if they could see the evaluation, too. But will they? For now, several conclusions from the report, by Abt Associates, did stand out to me. The Beyond 12 and Kinvolved apps weren’t as effective as they could be, it says, because: - Students didn’t use the apps as frequently as the organizations had hoped. - Under the contest’s timetable, the apps couldn’t be tested ahead of time and then tailored to meet students’ preferences or needs. - It might have been misguided to expect that a single-purpose app could have a significant impact on a diverse population of students. In other words, maybe the contest wasn’t designed so thoughtfully to begin with. Maybe it was too steeped in silver-bullet thinking. That’s my conclusion, informed to a degree by what I heard from Alexandra Bernadotte of Beyond 12 and Miriam Altman, co-founder of Kinvolved. While both said they benefited from being finalists, they also hinted at some frustration with the limitations they were under. Kinvolved was accustomed to engaging with administrators at the schools using its products, but with the app it developed for the College Success Prize, Altman said her team wasn’t allowed to connect with administrators at CUNY. The foundation wanted to measure the impact of the tool in isolation, she told me. As for Beyond 12, its digital-coaching app typically relies on personal interventions when called for. But those weren’t allowed in the CUNY experiment either. Still, Bernadotte said she was gratified the contest proved that “for the students who are most at risk of dropping out, an app alone is not enough.” That raises another issue: Even though the contest didn’t demonstrate that a single app could meaningfully raise completion rates — and ultimately promote social mobility — it does seem to have succeeded in reinforcing principles that are foundational to the contestants’ approaches. The foundation said it does not consider the contest or its outcome a disappointment. “As a venture philanthropy, Robin Hood does not wed itself to any one theory of change,” Kevin Thompson, managing director of communications, said in a statement. “We realize that not every idea will be successful, but we owe it to the more than 1.5 million New Yorkers who live in poverty daily to vet and test as many possible solutions to poverty as possible.” And, he added, the results “should not be misconstrued as a reflection on the finalists themselves,” especially given the contest’s “audacious goal.” The foundation remains "optimistic that technological innovations have a place in making a difference in educational outcomes," Thompson said, and it will continue to support those ideas, along with other, more-comprehensive approaches. Six years ago, when I wrote about prizes as a way to promote big goals, I noted the potential for a contest to put too much attention on finding new ideas, rather than highlighting approaches that may actually work but have yet to catch on. Now I realize that there are additional risks, like a timetable and conditions for participants. And perhaps not wanting to embarrass the finalists was the reason the foundation didn’t formally announce the results. Turnover at the foundation could have also been a factor. But Robin Hood officials did say from the start that there might not be a winner. As audacious as its goal was, its follow-through feels like a bit of a letdown — and a missed opportunity for others to learn from the process. Got a tip you’d like to share or a question you’d like me to answer? Let me know, at goldie@chronicle.com. If you have been forwarded this newsletter and would like to see past issues, [find them here](. To receive your own copy, free, register [here](. If you want to follow me on Twitter, [@GoldieStandard]( is my handle. Goldie’s Weekly Picks INFORMATION SECURITY [Cyberattacks Are Spiking. Colleges Are Fighting Back.]( By Katherine Mangan [image] Criminal hackers have taken advantage of the unique circumstances of the Covid-19 pandemic, causing a significant increase in the number and severity of cyberattacks. REOPENING CAMPUS [The Vaccination Dilemma]( By Alexander C. Kafka [image] Should colleges mandate Covid-19 shots? Encourage? Incentivize? All of the above? WORK FORCE [Here’s Who Was Hit Hardest by Higher Ed’s Pandemic-Driven Job Losses]( By Dan Bauman [image] People of color and those with limited labor protections have endured a disproportionate share of the losses. Paid for and Created by LinkedIn [What Higher Ed Marketers Need to Know: The New Adult Learner]( According to insights from LinkedIn, the time is now to engage adult learners. Learn how LinkedIn can help higher education marketers drive enrollment with a full-funnel strategy. Read the article. ADVERTISEMENT [Advertisement]( Job Announcement Vice Provost at Simmons University.[Visit jobs.chronicle.com]( for more details. Paid for and Created by Rowan University [‘A Safety net for all their worries’]( Focused on serving adolescents and adults with intellectual and developmental disabilities, Rowan Integrated Special Needs Center embeds behavioral health and care coordination into primary care, supporting their needs and complex medical conditions. Today's Global Campus Strategies for Reviving International Enrollments and Study Abroad Pandemic travel restrictions cut both ways, causing international enrollments to plummet and limiting study-abroad opportunities. This Chronicle report provides an in-depth look at how the global education experience has changed and offers strategies for assessing and adapting programs to ensure students' exposure to cultural and global diversity. [Order your copy today.]( Job Opportunities [Search the Chronicle's jobs database]( to view the latest jobs in higher education. What did you think of today’s newsletter? [Strongly disliked]( // [It was OK]( // [Loved it](. [logo]( This newsletter was sent to {EMAIL}. [Manage]( your newsletter preferences, [stop receiving]( this email, or [view]( our privacy policy. © 2021 [The Chronicle of Higher Education]( 1255 23rd Street, N.W. Washington, D.C. 20037