Learned Optimism: Probably Better Than Nothing

I was going to read Learned Optimism by Martin Seligman, but got distracted before I even started. The introduction to the second edition mentions a study which showed that the change from pessimism to optimism is at least partly responsible for the prevention of depressive symptoms. I guess if you're reading a self-help book, it's nice to know that the advice is backed up by science. So I was curious and had a look at the cited papers by Jaycox et al. (1994) and Gillham et al. (1995).

Original study

The authors were screening for 10-13 year-old children at risk for depression. The children that matched their criteria and were ready to participate became either part of the treatment group or the control group. The treatment group met once per week over 12 weeks for a 1-1½ hour workshop in groups of 10-12 children, where one of the first three authors of the two papers taught them cognitive and social-problem-solving techniques designed to prevent depressive symptoms (Gillham et al., 1995) with games, group exercises, and homework assignments. The authors call it the Penn Prevention Program. They found a significant reduction of depressive symptoms both immediately after the program and two years later. The effect was no longer significant after three years (Gillham et al., 1999). So far so good.

Unfortunately, the study has two severe design flaws: (1) the assignment to the treatment/control group was not random, and (2) the control group did nothing except for the assessments. Both problems are acknowledged by the authors themselves in the papers' discussion sections. The assignment was not only not random, but treatment and control groups were selected from different schools, so it's possible that some completely unrelated change (a new teacher at the treatment group's school, for example) caused the observed effects. Disregarding this first problem, the second flaw limits the interpretability of the results. Without active controls, we cannot say whether the cognitive and social-problem-solving techniques taught in the workshops caused the observed effects. For all we know, there could be other plausible causes, for example, regular social interactions in a group, possibly initiating friendships that last beyond the duration of the program; peer support by discussing shared problems in the style of a support group; attention of adults, especially for children that have problems at home; or a mere placebo effect. We also don't know how this program compares to alternative activities like joining a sports club, a band, or a study group. Given a random control group, all we could say is that the Penn Prevention Program is better than nothing.

However, this was only a first study, so establishing that a short intervention like this had a significant effect at all can already be a valuable goal and the authors acknowledge the shortcomings themselves. So let's have a look at the research that followed these two papers.

Later studies

The good news is that all the later studies I found had a random control group. The problem of the passive control group, on the other hand, persists, even among the authors of the original study, see Seligman et al. (1999) and Seligman et al. (2007).

Among the studies with active controls, some are not active enough for my taste, for example, those described in Rohde et al. (2018), who have a bibliotherapy group and a brochure control: one group receives a book about cognitive-behavioral (CB) techniques, the other an educational brochure with the title Let's Talk About Depression. It turns out that the bibliotherapy and brochure group are indistinguishable and the in-person workshop group showed a significant effect compared to both bibliotherapy and brochure control. If anything, this suggests that learning about CB concepts had no significant effect, but in-person group meetings did.

Merry et al. (2004) compared a CB program with placebo programs including group art activity, community service, etc. They find a small significant effect of the CB program compared to the control. Another interesting study with a well-designed active control group was conducted by Gillham et al. (2007), following Shatté (1997) and Reivich (1996). They compare a CB program (the Penn Resiliency Program, PRP) with a group intervention designed specifically to control for factors unrelated to CB training such as adult attention, group cohesion, social support, the discussion of depression-relevant topics, sharing thoughts and feelings with peers (the Penn Enhancement Program, PEP). The results are somewhat inconclusive since they differ between schools, but overall PRP doesn't seem significantly better than PEP.

The full sample findings provide little evidence for PRP’s effectiveness or specificity. PRP prevented the onset of elevated symptoms relative to a no-intervention control but not relative to PEP (an intervention that controlled for noncognitive–behavioral therapy specific factors). PRP did not reduce mean levels of depressive symptoms over the follow-up and did not prevent high or clinical levels of symptoms relative to either comparison group.

A somewhat similar study with similar results was performed by Horowitz et al. (2007).

There are also studies that failed to replicate the original study altogether, even with a passive control group (Pattison and Lynd-Stevenson, 2001; Roberts et al., 2003).

Last words

I still haven't read the book (not sure I will) and I'm aware that I only looked at a small selection of related papers, but it's a bit odd that the one study Seligman chose to back up his claims only tries to show that learned optimism is better than nothing.