Selection bias is the principal problem that has bedeviled Alcoholics Anonymous (AA) effectiveness research for decades. Countless studies show that problem drinkers who attend AA have much better outcomes than those who do not, but how does one tell whether that it is an effect of AA or is merely a reflection of those who go to AA being different sorts of people than those who do not? (e.g., more motivated to change, less psychiatrically impaired etc.).

One way to get around this problem is to conduct trials in which people are randomly assigned to interventions designed to encourage AA involvement and then use an instrumental variables analysis (Austin Frakt’s explanation here) to get a selection-bias free estimate of AA’s impact.

My colleagues and I took this approach using 6 datasets on about 2,400 alcohol dependent people participating in randomized clinical trials. Five of the datasets had a similar result, namely that AA significantly increased the likelihood of abstinence at 3 month and 15 month follow-up even when self-selection bias is not a factor. In the sixth dataset, which included a sample with long histories of treatment and AA involvement, further AA participation did not affect outcome one way or the other. This is probably because if AA hadn’t worked for these people over many years of their trying it off and on, it wasn’t likely to start doing so, and if it was working for them, even more AA on top of their high involvement couldn’t help them any more than it already had (i.e., You can’t be sober more than 100% of the time.).

The study is available ungated on line. It’s one of many in the past decade that support AA’s effectiveness.