I built software for clinical trials for about a decade, and in that window, it was pretty obvious to me that the inherent conservatism and resistance to change of the majors (and likely the FDA) was driving a lot of unneeded costs. Better regulatory policy (especially around IRBs) and more effective auditors who could push for quality improvements rather than box checking would make things safer and cheaper.
Aug 14, 2023·edited Aug 14, 2023Liked by Ruxandra Teslo
I wonder whether the first plot depicting Eroom's Law actually tells us much. Is the number of new drugs the correct numerator? Suppose we invented a single drug that cured cancer; by that measure, R&D efficiency would plummet. Of course, if we adjusted for clinical benefit (in QALYs, perhaps?) per new drug, perhaps it would show an even more disturbing drop in R&D efficiency. But that's the sort of evidence that would convince me that things are as bad as people say. Furthermore, measuring trends in clinical benefit would probably be quite hard, precisely because clinical benefit, like drug profit, is fat-tailed. Just as Taleb showed that the fat-tailed distribution of war casualties makes it hard to infer whether the world has become less violent over time, I would expect that the fat-tailed distribution of drug profits would make it hard to have tight confidence intervals around an estimate of R&D efficiency over time.
However, I think if you did QALYs it wouldn’t look much different. A lot of the newly approved drugs are in oncology (for historical reasons related to more investment in this area +prioritisation of cancer) and anecdotally those bring quite incremental benefits.
Overall I see no reason to believe that the drugs we make today increase lifespan by more units than those in the past (although there is a cumulative effect such that cancer survival rates are higher?
Indeed... Another thing that occurred to me is that, contra my argument above, the fat-tailed distribution of outcomes for each new drug should probably make us focus more, not less, on the number of new drugs each year. If we can't really predict the impact of a drug until after it's been approved and reaches patients, then we should just maximize the number of "at-bats", instead of trying to maximize the blockbuster-to-approval ratio.
I built software for clinical trials for about a decade, and in that window, it was pretty obvious to me that the inherent conservatism and resistance to change of the majors (and likely the FDA) was driving a lot of unneeded costs. Better regulatory policy (especially around IRBs) and more effective auditors who could push for quality improvements rather than box checking would make things safer and cheaper.
Yes, I can believe this. Alex Tabarrok writes abt this issue
I wonder whether the first plot depicting Eroom's Law actually tells us much. Is the number of new drugs the correct numerator? Suppose we invented a single drug that cured cancer; by that measure, R&D efficiency would plummet. Of course, if we adjusted for clinical benefit (in QALYs, perhaps?) per new drug, perhaps it would show an even more disturbing drop in R&D efficiency. But that's the sort of evidence that would convince me that things are as bad as people say. Furthermore, measuring trends in clinical benefit would probably be quite hard, precisely because clinical benefit, like drug profit, is fat-tailed. Just as Taleb showed that the fat-tailed distribution of war casualties makes it hard to infer whether the world has become less violent over time, I would expect that the fat-tailed distribution of drug profits would make it hard to have tight confidence intervals around an estimate of R&D efficiency over time.
That’s a good point.
However, I think if you did QALYs it wouldn’t look much different. A lot of the newly approved drugs are in oncology (for historical reasons related to more investment in this area +prioritisation of cancer) and anecdotally those bring quite incremental benefits.
Overall I see no reason to believe that the drugs we make today increase lifespan by more units than those in the past (although there is a cumulative effect such that cancer survival rates are higher?
Indeed... Another thing that occurred to me is that, contra my argument above, the fat-tailed distribution of outcomes for each new drug should probably make us focus more, not less, on the number of new drugs each year. If we can't really predict the impact of a drug until after it's been approved and reaches patients, then we should just maximize the number of "at-bats", instead of trying to maximize the blockbuster-to-approval ratio.
The outcomes are not outcomes as in outcomes for patients. They represent revenue