A new study today on the Oregon Medicaid experiment (they randomly selected some who applied for coverage while denying others; the difference in the groups is the estimated impact of having Medicaid v. being uninsured in Portland, Oregon; they data reported are only from Portland, and not statewide). Such a study design is as good as it gets on internal validity (does x cause y; in this case does Medicaid make people healthier, measured in a variety of ways, as compared to being uninsured). The external validity of such a study-are these findings applicable to other places depends upon how similar other states are to this study population (Portland).
Based on reading the many tweets this afternoon about the study, I was surprised when I actually read the study. Basically, they find that Medicaid coverage:
- reduced depression, improved or maintained self reported quality of life, increased the likelihood of having a usual source of primary care, increased the uptake of several preventive measures such as cholesterol screening, pap smear and mammography, and increased perceptions that patients received quality care. Again, this is the estimated impact of Medicaid as compared to being uninsured.
- Medicaid also greatly reduced the financial burden of health care and nearly eradicated catastrophic health expenditures that they defined as being greater than 30% of family income (or ~$4,500 for family of two at the federal poverty level in Oregon).
There was bad news too. They found, for example, that Medicaid did not:
- decrease blood pressure, reduce cholesterol, or decrease the proportion of diabetics with A1C levels above 6.5%, or reduce the Framingham risk score significantly (a global measure of 10 year heart disease mortality risk). There was also no significant difference in persons reporting no, or mild pain. There were also some prevention screening procedures that those with Medicaid were not more likely to receive (Table 5), for example: fecal-occult blood test, colonoscopy for those age 50+, flu shot. Again, this is the estimated impact of Medicaid as compared to being uninsured.
In terms of health care use/spending, those with Medicaid:
- had more doctor visits, but there were no differences in hospitalizations, ED visits, or outpatient surgery. Persons with Medicaid had an estimated health care expenditure on their behalf of $4,428, or $1,171 more than someone who was uninsured. Keep in mind other parts of Medicaid that were not a part of this study, or the proposed expansions, cost lots more.
Several things of note:
- This study is focused on Portland, and not even all of Oregon (see 1st para of second column on page 1714). The internal validity of the study (does x cause y) is as good as it gets. The external validity (do these results generalize) depends upon how similar Portland, Oregon is to another state thinking of expanding Medicaid.
There are a few things about the study subjects that leap out. First, these are some seriously well controlled diabetics in Portland, with 5.1% of the control group (uninsured) having A1C of 6.5%+; those with Medicaid were not statistically better. I honestly think this must be a typo (Table 2) and instead of 5.1% of diabetics in control group with A1C of 6.5% or greater they meant 5.1% with 6.5% A1C or less (though it is repeated in the text).This is a presentation to the results of an incredibly successful diabetes disease management company in Mississippi that is absolutely thrilled to have 64% of patients with A1C less than 7% after one year (see slide 25); the Oregon study says that 95% have levels of 6.5% or less.* Nationally, between 15-30% of diabetics have A1C of greater than 9.I just don’t see how what is written can be true, or if it is, the country should be studying diabetes care in Portland. Update 11:50pm on 5/1; again at 5/2 8:00am: This is a A1C of everyone on the study and not only diabetics. I was thinking in terms of diabetics in what I wrote above and now strikethrough that text; that was my error. This link says prevalence of diabetes over age 20 is 11.3%…but what is needed for comparison to see the similarity of the study population is prevalence between 18-64….going to bed will get after that tomorrow.and this one also has 11.3% prevalence for age 20+, but 26.9% for age 65+. I don’t think the A1C prevalence of 6.5% or greater is that different from the national average, but I can’t find a precise age comparison for the study population (ages 18-64).- This sample of patients went to the doctor a lot. The uninsured reported an average of 5.5 office visits annually, with those covered by Medicaid having 2.7 more per year, on average. Using this CDC report (Table 10, No. 252, page 115), I found that nationally 69% of persons age 18-44 in the US in 2010 had between 0 and 3 office visits and only 19% had 4-9 visits; for the age group 45-64, it was 57.7% with 0-3 and 25% with 4-9. Yet the mean of the uninsured group in this study is 5.5 visits annually, with those covered by Medicaid having an average of 2.7 more.
The bottom line is that this is a well done study with high internal validity, and it finds some positive health impacts of Medicaid on health as compared to being uninsured. It also finds some areas in which health is no better (or worse) for those covered by Medicaid. The study unambiguously shows that Medicaid reduces the financial burden of health care for beneficiaries. There are some aspects of the study findings that may reduce the applicability of the findings to other parts of the country, but that doesn’t mean it isn’t an important study. It is just another piece of the puzzle.
However, for all the tweeters saying this study showed Medicaid is utterly flawed and there is no way anyone should expand Medicaid coverage I can only think they didn’t actually read the study. More tomorrow on the political and policy ramifications of this study and how people talked about it within 5 minutes of its release.
*Disclosure: I am a consultant for Diabetes Care Group, Inc. and have done cost estimations for them based on their clinical A1C results; some of them are excerpted in that publicly available slide deck.
cross posted at freeforall
Once Oregon had decided to expand Medicaid by lottery, the naturally controlled trial was sensible and ethical. Would it have been ethical to set up the whole thing just for the trial? I’m not so sure. Denying care to people for the advancement of knowledge was the rationale for Tuskegee. And the initial policy_driven expansion by lottery looks fishy too. If you have x million dollars for an expansion of medical care, equity suggests an equal distribution.
However, I can see a case for lottery policy experiments in fields where there’s more genuine a priori uncertainty about the benefits, as in educational interventions.
One thing this study probably shows isn’t that Medicaid is flawed, but that our current system of medical intervention is flawed. I bet the findings of no improvement in cholesterol, blood pressure, etc. would be matched for the same cohort with private insurance.
Actually, the study didn’t show that Medicare failed to reduce blood pressure, reduce cholesterol, etc. What it showed was a non-significant change with a very limited sample size, and it’s very important to be specific about that (otherwise you get headlines like this one from today: “More Bad News for Obamacare: Study Finds Medicaid Has No Effect on Measured Health Outcomes”)
“…our power to detect changes in health was limited by the relatively small numbers of patients with these conditions; indeed, the only condition in which we detected improvements was depression, which was by far the most prevalent of the four conditions examined. The 95% confidence intervals for many of the estimates of effects on individual physical health measures were wide enough to include changes that would be considered clinically significant — such as a 7.16-percentage-point reduction in the prevalence of hypertension. Moreover, although we did not find a significant change in glycated hemoglobin levels, the point estimate of the decrease we observed is consistent with that which would be expected on the basis of our estimated increase in the use of medication for diabetes.”
There’s a good analysis here: http://theincidentaleconomist.com/wordpress/oregon-and-medicaid-and-evidence-and-chill-people/
unfortunately, an inability to understand statistics and statistical reasoning is prevalent throughout the population. the remarkable misinterpretive framing of the results of this study demonstrate that as well as i’ve seen in a while.
I think the most striking finding of the study is that 4.4% of the Medicaid eligible group had diabetes diagnosed during the two year period. That seems huge to me (Ok so I had no idea how common diabetes is).
The study didn’t contain proof that current treatment of diabetes reduces the fraction of diabetics with A1C > 6.5% but, I mean, we don’t need a new study to know that. The statistically significantly higher rate of diagnosis of diabetes combined with information from other studies (and decades of clinical practice) seems to me to imply that access to Medicaid causes fewer people to have highly glycosylated hemaglobin.
Now glycosylated hemaglobin is not mortality or morbidity. For some reason the step from glycosylated hemaglobin (or high blood cholesterol or high blood pressure) to morbidity and mortality was taken as known, but the step from treatment to low glycosylation, cholesterol or pressure wasn’t.
By my back of the envelope calculation the p-level of the difference in rates of diagnosis of diabetes (listed as <0.1% in table 2 reproduced by Kevin Drum) is also less than ten to the minus 27th which I think is called an octillionth I mean a billionth of a billionth of a billionth.
The general view is that the study does not contain proof that Medicaid causes improved health, but really, does anyone doubt that it is better to have one's diabetes diagnosed. That is the question left open by the study to the extent it was open last year which is not at all.
I’m a bit surprised that no one seems tobe drawing the obvious inference-health care in general might not be as effective as we think. These findings might not be a Medicaid thing at all.
There are two points that I think are not being discussed enough.
First of all, a rather important aspect of Medicaid is that it is an anti-poverty program, as Matt Yglesias notes. Keeping fluctuations in income down (to which the cost of medical care is a major contributor) is a rather important factor (halting or mitigating the slow economic death spiral of debt, depression, divorce, and delinquency) and in that respect, the program seems to have had pretty good success.
Second, the results are all about chronic conditions [1], but the far more important benefit would be enabling the treatment of acute conditions (especially insofar as they prevent them turning into chronic ones) and preventive care. Like, preventing people from dying to a tooth abscesses because they can’t afford antibiotics [2]. Household accidents are common; burns and broken bones need to be treated, not managed (unless someone is advocating DIY tetanus shots, and even then you’d have to pay for the vaccine).
[1] It is also worth noting that “not significant” did mean “not statistically significant”. There was actually a noticeable decrease, so it doesn’t disprove the hypothesis that Medicaid is beneficial for chronic conditions; the data just wasn’t strong enough to serve as a statistically robust corroboration. Even if not, as Sebastian H notes, this would likely be a question of how effective treatment/management of these conditions is in general, assuming the results are not specific to poor patients (in which case, the anti-poverty aspect would become even more critical).
[2] Another scary problem is that even people with health insurance may lack dental insurance, including for even the most basic dental care.