If you were a neuroscientist and discovered that there were, on average, measurable anatomical differences between (1) the brains of 20 young right-handed people who smoke an average of a little more than 11 joints per week and had been consuming cannabis for a little more than 6 years and (2) the brains of an age-sex-chirality-matched group, and that those differences persisted even after controlling for alcohol consumption and were accentuated among those who use more cannabis, you’d probably say, “Hmmm … that’s interesting. I wonder what it means.”
After all, it might mean:
1. That using cannabis at that level causes changes in the brain.
2. That something else correlated with cannabis use - for example, use of other illicit drugs - causes changes in the brain.
3. That something about having that kind of brain makes cannabis use more attractive to people to have it than it is to people who don’t.
4. That the brain differences and the cannabis-use differences between the two groups are the product of some unknown third factor.
If, on further study, it were to turn out that the differences were the result of cannabis use, then you might ask, “What are the consequences - if any - of those differences?” After all, various learning tasks are known to change brain anatomy: London cabbies, who are required to learn London in astonishing detail, have larger-than-normal anterior and posterior hippocampi, and visual artists also have brains that are structurally different from non-artists.
Now, if you’d started out with already-known measured deficits in cannabis users and found brain changes independently known to lead to such deficits, that last question wouldn’t be so hard to answer. Of course, that’s hard, because the most recent meta-analysis found “no evidence for enduring negative effects of cannabis use” on neurcognitive function. (The Dunedin 8-IQ-point finding is about people with chronic substance use disorder.)
Overall then, if you were that neuroscientist, you’d write a paper saying “We studied cannabis users and non-users and found the following brain differences. Here’s the next study we plan to do, addressing the questions of causation and possible impact.”
That’s assuming that your goal was informing your readers about the content of your findings. If instead you wanted to score points in the culture wars, push your political agenda, and perhaps please your sponsors at the National Institute on Drug Abuse and the Office of National Drug Control policy, you’d behave differently.
First, you’d assume that all measured differences represent “abnormalities” among cannabis users: for example, both increased gray-matter density in some brain regions and decreased gray-matter density in other brain regions. You’d ignore the fact that your study hadn’t found any functional differences between the two groups. You’d use the loaded term “abnormalities” rather than the neutral term “differences” in the title of your paper. Second, even if the journal reviewers made you put in some weasel-words about not being able to make any causal inference (because the study was purely correlational, with neither a controlled intervenion nor a longitudinal design) you’d still put out a press release that makes it sound as if you’d found something very scary, “a strong challenge to the idea that casual marijuana use isn’t associated with bad consequences,â€: (remember, no actual bad consequences of any kind were found). Pretending that the findings to “casual” cannabis user would require that you gloss over how extreme your sample was: an average age of onset of just over 15 (very young exposure is known to be correlated with higher risks) and cannabis use of a minimum of a joint a week and an average of 11 joints a week. (The median cannabis user consumes once a month; once a week - the minimum in this study - puts someone in the top quartile, while 11 joints a week would put someone in the top 15%.) Instead, you’d describe your findings as applying to “recreational” or “light-to-moderate” cannabis use. And just in case anyone had missed the point, you’d claim that “further study of marijuana effects are needed to help inform discussion about the legalization of marijuana.” (And of course you’d get away with it, though an alcohol researcher who tried to put in a plug for higher alcohol taxes in a paper on alcohol-related violence would almost certainly get slapped down.)
It’s entirely possible, though not yet demonstrated, that chronic heavy cannabis use causes undesirable changes in brain structure and function. Even if it doesn’t, spending a good chunk of your waking hours zonked seems to me like a bad idea no matter what the zonking agent is, and that’s true in spades for adolescents, who may be unable to make up missed opportunities for both formal and social learning.
So I agree entirely that preventing a large increase in the number of people at the top of the cannabis-use distributionm and the number of mid-teens initiating to more than very occasional use are important policy goals, and that, as we go about legalizing cannabis, we ought to design the tax and regulation systems to serve those goals. I’ve made myself unpopular among extreme legalization advocates - not to mention the people hoping to cash in on an increase in substance use disorder - by saying so, and by making fun of the What-Me-Worry approach to the problem (typified by the claim that legalization will reduce underage use because legal vendors won’t sell to kids). But the way this study was written up demonstrates that neither side in the culture war has a monopoly on b.s., and I think we’re all entitled to complain especially bitterly when the b.s. is supported by our tax dollars.
[See also Maiz Szalavitz and Jacob Sullum. Sullum points to Lior Pachter, a computational biologist, who gets down in the statistical weeds before concluding that this is “the worst paper I’ve read all year.”]
Update The lead author on the paper seems to be innocent of any political purpose. The scare quotes in the press release come from the last-named author, who is also the most senior person on the team.
Thank you for providing the final exam for my statistics students.
I would expect that the progressive pot movement has concluded that the science is settled and pot is benign. If you disagree, then be prepared to be issued the "hater" and "antiScience" tags. Wear them proudly.
Accept, Cope , Obey
I wonder how much longer the heroic London taxi test can last, since GPS has made it obsolete. Its splendid name - "The Knowledge" - will pass into history.
I lamented this in these pages
https://thesamefacts.com/2012/03/psychology-well-…
And all it takes is that initial élan of implied disaster for media outlets to run in to the endzone, especially with marijuana policy such a hit-magnet these days. It’s fun watch these stories bounce around the news-o-sphere, with about two and a half days between the paper’s publication and CNN running the headline “Casual Marijuana Use May Damage Brain.”
A polity just slightly more literate in research methods and statistical analysis (read: better bullshit detectors) would go a long way toward re-populating the middle ground in these policy debates, and ultimately producing better policy. But I’m not holding my breath.
Lior Pachter, cited by Sullum, says at the very end of that post that scientists should be sanctioned for making public statements which contradict the content of their papers, and cites as a precedent the case against Scott Harkonen, linking to an article in the Washington Post.
(This is part 1; comments are apparently limited by length and more will follow.)
Part 2:
The WaPo article reports that Harkonen’s conviction was for wire fraud based on a press release in which he told potential investors that a clinical trial had demonstrated a survival benefit for his company’s drug Actimmune in patients with mild to moderate cases of idiopathic pulmonary fibrosis (IPF). The facts in that case are complex, but Harkonen, acting as CEO of InterMune, represented Actimmune as having demonstrated effectiveness in these patients, based on a (very likely flawed) post hoc subgroup analysis from a larger clinical trial which had failed to show a benefit (that is, it had a p value greater than 0.05) in a population with mild, moderate, and severe IPF combined. Harkonen skewed the findings of that clinical trial, and such post hoc analyses do not “demonstrate” anything, however valuable they may be for generating hypotheses for further research. As it turned out, a later clinical trial in 2009 did not show any benefit and was stopped at the second interim analysis with a conclusion that the drug cannot be recommended for treatment of IPF.
Part 3:
Harkonen had done something in his press release which happens regrettably often in the published medical literature; he had done an analysis which had not been planned at the beginning of the study, had found an answer he liked, and had reported that answer as if it meant that the drug worked in the selected subgroup of patients. For this he was convicted of a felony in federal court.
Part 4:
The government wanted Harkonen sentenced to ten years in prison, but he was sentenced to six months house arrest. It is not clear what kind of sanction Pachter has in mind for scientists who engage in dubious statistical gamesmanship, but there is something going on here which warrants comment (hint, hint) for an RBC post by one of the regulars. In the financial collapse of 2008, there were CEOs of large banking and investment firms whose frauds have never been brought to court, much less conviction and confinement. These CEOs are still very rich, still enjoy membership in good standing in the social class to which they belonged before they ruined their firms and the wider economy, and in many cases remain working in the same industry. Harkonen, on the other hand, is finished as a researcher and possibly as a physician; his petition for certiorari was rejected by the Supreme Court in December of last year.
Does anyone out there have a sense of outrage at the double standard here?
In the financial collapse of 2008, there were CEOs of large banking and investment firms whose frauds have never been brought to court, much less conviction and confinement.
I'm going to say the same thing I do every time someone alleges this: I want names; I want the specific acts they committed that meet the legal definition of fraud and not just the common usage definition; and I want to know what admissible evidence you have that ties the specific individual to those specific acts.
People spout off on this without having the slightest idea what it takes to actually convict someone of fraud. In fact, fraud in this context is actually very narrow and has high hurdles to meet. It requires that the charged individual explicitly intended to engage in the specific acts that constitute fraud or that they were grossly negligent, another term that in a legal setting is a lot narrower than most people seem to understand, and not just that they were in charge of a company in which some people engaged in fraud. The acts must be material, yet another term that people don't understand; it means not only that the amount of harm caused is sufficiently high but also that absent the fraudulent act the injured party likely wouldn't have engaged in the transaction and there are a lot of instances in which they would have done so even without the dishonest behavior (often for reasons of what did and didn't qualify as Tier 1 capital). And you need to have that hard evidence tying the individual to all of that.
So it's not a double standard. It's that Harkonen engaged in behavior that met a narrow definition and that he was stupid enough to do so leaving a paper trail behind him to tie him to that behavior.
Fair question; I rely upon Charles Ferguson’s book Predator Nation, which does supply some specifics regarding allegations of securities fraud. On page 190 of that book he cites Rule 10b-5 under the authority of the SEC granted by Congress by the 1934 Securities Exchange Act, which covers intentional misstatements or omissions sufficiently material to affect an investor’s assessment of a situation resulting in actual losses. As an example, Ferguson writes about the Hudson and Timberwolf synthetic CDOs sold by Goldman Sachs, which was the focus of a Senate investigation led by Carl Levin, who called as witnesses Goldman’s head mortgage trader Dan Sparks, CFO David Viniar, and CEO Lloyd Blankfein. Goldman misled a number of institutional investors such as foreign banks and public pension funds when unloading Timberwolf assets known by Goldman executives, as documented by internal e-mails, to be toxic.
I do rely upon the expertise of authors like Ferguson in these matters; unlike the role of statistical significance testing in clinical trials, I have insufficient knowledge of my own to assess the facts independently.
I have not read Predator Nation but I have not seen any evidence in any of those actions that would build a credible case of fraud against any corporate officers in the deals you're discussing. And the Abacus case, which in many ways underlies those, is one in which I was surprised that the SEC managed to get a settlement at all because I thought they would have a hard time meeting the materiality standard in court. This is one of the deals I was referring to in which I think it's likely the injured party, in that case a German Landesbank, would have engaged in the transaction even had they known all of the details because they weren't really looking for a safe investment; they were looking for something with a high rate of return that slipped through a loophole in German bank regulations as to what qualified as Tier 1 capital.
The second issue is that of values, in which misconduct results in opprobrium from colleagues and banishment from the industry. Dr. John Darsee was an academic superstar at Emory and later at Harvard, where he published several studies in the field of cardiology while still a fellow; the New England Journal of Medicine published two papers on cardiomyopathy in 1979 and 1981 for which he was the first author. In 1983 the papers were retracted after it was shown that the data had been fabricated. Darsee was summarily dismissed from Harvard and even had his licence to practice medicine revoked by the State of New York in 1984, not for anything he had done to patients under his care but for his fraudulent research published earlier.
Darsee was never prosecuted for fraud, but the NIH banished him from research for 10 years and demanded the return of grant money which had sponsored his work. He ended up practicing internal medicine in Indiana but has never shown his face in the world of medical research again.
Contrast this with the retention of status and reputation enjoyed by the financial incompetents under whose leadership major financial firms collapsed into bankruptcy, and these firms were rescued by taxpayers under TARP and other programs. Different standards suggest differences in morals.
This I will certainly agree with. The financial industry is impervious to shame, much to it's, uhm, shame.
define Zonked