image description

KEY FINDINGS

Impacts of the growth mindset program on academic outcomes from the National Study of Learning Mindsets were first published in Nature in 2019. The research team analyzed data from more than 12,000 students attending 65 schools in the NSLM that shared students’ academic records with the study team.

PRIMARY OUTCOMES OF THE GROWTH MINDSET PROGRAM

Positive effects on mindsets. The program reduced the prevalence of self-reported fixed mindsets (the belief that intellectual ability cannot be developed), replicating multiple earlier studies conducted with smaller convenience samples of students. The effects were consistent across all student subgroups.

Positive effects on key academic predictors of high school graduation and college success. The program had benefits for both lower- and higher-achieving students. It improved grades in core academic subjects (mathematics, English language arts, science, and social studies) in 9th grade among previously lower-achieving students. It also increased enrollment in advanced mathematics courses in 10th grade among both higher- and lower-achieving students (this result was obtained in a sub-sample of 41 schools that shared 10th grade enrollment data).

  • Positive effects on grades of lower-achieving students. For students whose grades were below the median in their school, the program improved their GPA in core courses by 0.10 grade points relative to similar students in the control condition. The program also reduced the proportion of these students with a D or F average in their core courses by over 5 percentage points. These effects are substantial when compared to the most successful large-scale, time-consuming, and rigorously evaluated programs with adolescents in the educational research literature, and they are particularly notable given the low cost and time investment of the online program.
  • Effects on grades were related to school factors. Effects were larger in some types of schools and smaller in others. In medium- and lower-performing schools[1] in which the peer climate (the “norms”) supported the pursuit of challenging academic work, the program increased core course GPA by 0.15 points and STEM course GPA by 0.17 points on average among lower-achieving students. In these schools, the intervention also reduced the likelihood of D or F averages in core courses by 8 percentage points among these students. Again, these effects compare very favorably to those documented in rigorous experimental studies of education programs with adolescents (see Kraft 2018), many of which are time and cost intensive.
  • Positive effects on advanced mathematics course-taking were observed among both higher- and lower-achieving students. The program increased students’ likelihood of taking Algebra II or higher in 10th grade by 3 percentage points, elevating advanced course-taking from a base rate of 33% to a rate of 36% (this result was obtained in a sub-sample of 41 schools that shared 10th grade enrollment data). In the highest performing quarter of schools, the program increased the likelihood of taking Algebra II or higher in 10th grade by 4 percentage points.

Findings from the National Study of Learning Mindsets suggest that a carefully developed and tested but brief online growth mindset program can offer a way to improve key academic indicators and promote the pursuit of more challenging coursework as students make the critical transition to high school. The results also suggest that educators in all schools should pay particular attention to cultivating norms that value the pursuit of challenging schoolwork.

___

[1] Medium- and lower-performing schools represent the bottom 75% of schools nationally in terms of performance, as determined by combining several sources of data: within-state rankings based on test scores, average PSAT scores, and AP test-taking data (including the proportion of students at the school who take AP tests and their test scores).

ADDITIONAL DETAILS

It is best practice in the education sciences to compare effects from studies that are similar in terms of target beneficiaries, outcomes, and study design. The growth mindset program in the NSLM was aimed at students entering high school and examined effects on long-term academic outcomes, particularly their GPA and course failures.

The NSLM study team’s primary, pre-registered hypothesis was that there would be an effect of the growth mindset program on previously lower-achieving students’ grades. The program increased the 9th grade core GPA of these students by 0.10 grade points (an effect size of d = 0.10) and reduced the percentage of such students with a D or F average by over 5 percentage points.

One particularly useful specific comparison in secondary contexts is a 2010 evaluation conducted by MDRC and American Institutes for Research of the Enhanced Reading Opportunities (ERO) programs, which were supplemental literacy programs for previously lower-achieving students. The researchers found that the year-long course that students took in place of an elective had an effect of 0.06 grade points (an effect size of d = 0.07) on these students’ core grade point average (GPA) in 9th grade. Such an effect is practically meaningful given that performance in core courses in 9th grade is one of the best predictors of long-term educational outcomes.

In the NSLM, the average treatment effect on core GPA across all students was 0.05 grade points (an effect size of d = 0.05). The program also reduced the percentage of students with a D or F average by 3 percentage points on average.

By way of comparison, a rigorous evaluation of early college high schools found that the program increased the likelihood of students being on-track in 9th grade by 3.7 percentage points on average.

Such effects in education evaluations are rare. Fewer than one out of every six education programs that won scale-up grants from the federal Investing in Innovation Fund (i3) produced statistically significant positive impacts (n = 67). Of the programs that were measured, 0.20 SD and 0.23 SD were the largest two effects observed in a “cohort analysis” of the results of all of the pre-registered, randomized trials evaluating interventions, while the median effect was d = 0.03.

Researchers also compare the intensity and cost of the program. The growth mindset program in the NSLM took less than an hour to complete and such light-touch online programs have been estimated to cost less than $1 per person in previous studies, according to the World Bank. This compares favorably to many educational interventions, which can take many hours to complete and can cost up to thousands of dollars per student.

In his recent paper on interpreting effect sizes, Mindset Scholar Matthew Kraft (unaffiliated with the NSLM) adds that studies that measure short-term outcomes tend to show larger effects than those that are more cumulative, such as those that measure students’ GPAs or graduation rates. He also points to the importance of assessing whether the program could be replicated at scale in ordinary circumstances when comparing effects among studies. The online program in the NSLM was low-cost, brief, and implemented by schools rather than researchers, suggesting that the program is highly scalable. According to Kraft’s framework, the effect of the NSLM growth mindset program on GPA (both the overall average treatment effect and for the previously lower-achieving sub-group) would be classified as a medium effect at low cost with high potential for scalability.

OTHER MSN-SUPPORTED ANALYSES OF THE NSLM DATA

Visit the Mindsets and the Learning Environment Research Portfolio page to learn more about projects by MSN scholars that use the NSLM data.

Visit the NSLM Early Career Fellowship page to learn more about projects currently underway by a cohort of early career scholars using the NSLM data.