When older patients seek out health care, they are unwittingly enrolling in an experiment: Will medical procedures that have been proved effective mainly on the young also help the elderly?
That’s the first sentence in the editorial that my colleague Dr. Donna Zulman and I have in the New York Times today. We discuss the exclusion of older patients from many medical research studies and how it both compromises quality of care and is socially unjust. It’s a specific example of the more general problem I have discussed before: The people who are enrolled in the research studies that guide the provision of health care are markedly different from real-world patients on many dimensions.
When I was a member of an National Institutes of Health (NIH) grant review section, I saw many proposals that restricted study enrollment to people under the age of 55 or 60 or 62 or 65. I would always ask “If a patient otherwise enrollable was one day over your upper age limit, how would you justify excluding them to their face?”. The dominant response by grant applicants was interesting:
“I don’t know”.
What I began to see is that many exclusion criteria in research studies are simply cut and pastes from prior grant proposals and no one really knows where they came from or wants to defend them when asked. Authors of grant proposals debated many of the suggestions I made as a reviewer (which is appropriate) but never in my four years on the committee did someone argue that it was important that no one over the age of 55 or 60 or 62 or 65 be allowed to enroll. Exclusion of older people is simply a bad habit/tradition, with no strong defenders or even much thinking behind it, which is why I think NIH would encounter little resistance if it adopted the elderly patient-friendly reforms Donna and I propose.
The unnecessary exclusion of many patients from medical research is a serious ethical and clinical challenge to the future of health care. With welcome support from The Greenwall Foundation, I personally intend to make this problem a major focus of my work in the coming years.
Suppose that it is thought a medication or treatment may have a different affect on the elderly, however defined (& I qualify according to at least one of ages included in the OP). Then including them in the trial may well raise statistical issues that would involve a fair amount of expense to resolve properly. If they are included in the sample any measured average response is quite possible a composite of very distinct responses. If the sample size (and expense) is not increased substantially from what was proposed when the plan was to exclude older individuals from the trial, then the result may well be a finding of no statistically significant effect because of the increased variation from including them. Either include older participants and substantially increase the size of the trial or conduct a separate trial altogether, perhaps after performing it on the younger sample. Either way it is more costly. Will the NIH fund this?
The first question of course is how do we know that age is the correct variable to exclude on to promote homogeneity in treatment response? I am always struck by the assumption that even before we have demonstrated a main effect, we already know some interactions! Some modeling work we have done in our center shows that many of the most common exclusion criteria *reduce* statistical power because researchers have guessed wrong on what to exclude on.
Also, it is not at all clear that highly restrictive studies are less expensive because they have longer recruitment periods and often miss their sample recruitment target. It can be cheaper to drop exclusions and enroll more people, using the added power to compensate for any increase in heterogeneity in treatment response.
If the investigator thinks that age is an effect modifier, then there is time to plan for a stratified analysis of the outcome data in the study protocol. As long as the interaction is specified at the start of the study, problems with data mining should not arise.
http://www.ncbi.nlm.nih.gov/pubmed/17113277 has a nice example of a related phenomenon; the vast majority of community asthmatics would not have met inclusion criteria for the randomized trials which were used to write the Global Initiative for Asthma guidelines.
Keith: I wonder if you can shed light on a puzzling phenomenon. Trials that are preregistered at clinicaltrials.gov specify the primary outcome, but never seem to specify the minimal clinically important difference that they are looking for. You know what they want to measure, but you do not know how big a treatment effect they think is meaningful. Is this a rather odd omission or does it make sense?
Wow, what an amazing study thanks. This is a cross-disease problem. Mark Hlatky and his colleagues have shown that over 90% of CVD patients could not have enrolled in the major observational studies in the field. http://www.ncbi.nlm.nih.gov/pubmed/6396793
I did not realize that you didn't have to register an expected outcome size in clinical trials.gov. I would assume you could derive the expectation by working backwards from the target sample size and an assumption about statistical power. On the other hand, a number of trials miss their recruitment target (often because they have too many exclusion criteria) and some of them still find an effect because it was larger than they expected. Maybe's that's why the expected effect does not have to be pre-registered, but I am just guessing.
There is obviously merit in your point, but on the other hand be using primarily younger, arguably healthier people with less complicated medical histories the researchers may hope to increase the statistical power of their studies, and to avoid accusations that they are preying on the vulnerable elderly as they conduct their studies.
Still, even assuming you're completely correct: how would you propose to address the situation? I am powerfully reminded of the current, utterly asinine course the NIH has decided upon with respect to the gender of animals and even cell lines used in basic research. The latter is beyond risible to the genuinely offensive: there are huge problems of idiosyncrasy with the various cell lines commonly used in laboratory research, and focusing on their gender is a ludicrously misguided missing of the forest for the trees; but the former takes a genuinely important notion and, as an official and officious response, attempts to damage most science in the country.
I listened to an interview with the director of the NIH's office on women's health yesterday, and she started with a similar, even more glaring problem than they one you mention (profound under-representation of female lab animals, even when studying complaints that more often affect women than men), and started with the quite sensible proposition that we ought to pay attention to the possibility that we're overlooking or overstating possible effects by ignoring the differing physiologies of females and males. So far, so good. But then, and in her own self-damning words, she declared that henceforth all studies will be strongly pressed to include a mix of females and males, and to make it possible to break the data down by gender. This is not addressing the underlying, thoroughly valid complaint; instead, it adds a layer of complexity to each experiment and reduces the statistical power available, and adds work for the researcher, all without adequate compensation. Do you imagine the grants will be increased so that the researchers will have adequate sample sizes, separately, of female and of male animals? If so, you imagine vividly, and unreasonably. A dictate that subject gender must be tracked, and announced, and considered, and should be varied from experiment to experiment, would have been a valuable corrective. Instead, as official policy the NIH has now declared that all studies must generate muddled data with inadequate sample sizes.
Please: prove you're smarter than these half-bright people who saw a genuine problem and in response threaten to burn down their own house. Don't just point out that we're overlooking an important category of patient as research is performed, propose a sensible, effective, non-destructive set of reforms to solve the problem.
Please: prove you're smarter than these half-bright people who saw a genuine problem and in response threaten to burn down their own house.
Wow Warren, this is the most condescending thing I have seen you write. Out of character for you but nonetheless not much of an invitation to conversation.