In which I argue that universities do a unique type of research that cannot easily be replaced by markets and give concrete examples as well as empirical evidence
Excellent piece. Is the question whether or not private markets can *replace* research but: 1) Whether we are getting enough output relative to the $ we spend in academia 2) Whether that output is aligned with human flourishing 3) How the incentive structures from funding sources (primarily NIH/NSF) affect research directions and probably most importantly 4) How much of the science in academia is reaching regular people. Biotech, particularly drug development, is an anomalous case. I'm not sure it's good that biotech companies have outsourced R&D to academia and early stage startups. An industry with healthy profits ought to be pumping those profits back into R&D for a flywheel. If they are not, one should inquire why.
I worry most about (4) and believe a good chunk of the problem is the technology transfer offices at universities, who see their mission as trying to protect IP rather than maximize its value in society.
Finally, in contrast to bio, software progress has been driven largely by private labs and corporations. DeepMind and OpenAI and Meta have pushed the state of the art in AI forward considerably. The best AI and and software researchers are in universities, they are in the companies themselves.
Check out my last Substack and we should have a chat :)
I think the part I disagree with is not that academia has been important and useful for some advances, e.g. CRISPR.
It's that a much better system can exist that is hard for us to currently imagine because we are so invested in the status quo.
I don't think it's hard to imagine a big pharma company spending a few billion a year on fundamental research. They already spend so much.
Nor is it difficult to imagine philanthropists stepping in to fund the most important basic research. E.g. the Gates foundation.
But I think that's just the starting line of science without public funding. Similar to how we've invented specific structures for startups and an ecosystem that includes VC's and angels, why wouldn't further structures evolve for science?
If many parties benefit over time from fundamental research, then isn't it just a coordination problem that we can solve with mutually beneficial contracts? This is after all, the motivation behind patents: incentivize someone to publish their innovation in exchange for monopoly rights for a number of years.
But even obvious for-profit structures will solve for advances like CRISPR. Think: research labs that spin-off a series of startups. VC's can invest in the labs which then invest in the spinoff companies.
Then, you say, what about the sleeper hits in academia, the basic science that takes a very long time and eventually makes a big impact.
Well, the decision of whether this is worth funding is literally a profit-maximizing calculation: do we get better returns to capital by investing in this basic research which eventually pays off in 50 years or by investing in the next-best alternative. Academia never does any such calculation and frequently wastes everyone's time.
And that's my next point: Academia sucks up all the energy and prestige for science and prevents better alternatives. Ambitious researchers join and then are way less impactful than they would be, because they actually don't get the opportunity to do the curiosity-focused research that they want. They have to grind out insignificant papers that adhere to current trends. Risky, innovative ideas are shut down or not funded because committees and established scientists with a reputation to preserve are not acting with the right incentives.
I think this is a really interesting debate. I am still very bullish that removing public funding for science would make it so much healthier, innovative, and cost-efficient. It would also save us from the negative externalities of bad ideas produced by a lot of public funding.
> Ambitious researchers join and then are way less impactful than they would be, because they actually don't get the opportunity to do the curiosity-focused research that they want. They have to grind out insignificant papers that adhere to current trends
My preferred method is increasing diversity in ways of funding science (FROs, academia, stuff like ARIA etc). I am writing a post on this and the sort of broader, theoretical considerations around this that is going to come out tomorrow or early next week and addresses exactly your point around conformism.
>Well, the decision of whether this is worth funding is literally a profit-maximizing calculation: do we get better returns to capital by investing in this basic research which eventually pays off in 50 years or by investing in the next-best alternative. Academia never does any such calculation and frequently wastes everyone's time.
I understand the consideration around this. The problem is, biotech is itself an enormously inefficient industry. I think the problem is that long time to product (and consequently feedback from the market) means that human incentives for career promotion are not very aligned to producing something that actually works. Biotech has a lot of dumb market failures. You should check this out this post I wrote about Roivant that goes into this a bit:
As an additional example, we do not have good animal models for a lot of diseases, because it's basically in no one's interest to produce them. I think the solution to this is an FRO (I hope to write about this too in the future)
hmm what I will add though is that this stuff might be different in Life Sciences vs other industries. There is this new paper by Arora et al that I need to review in a future post that claims the opposite of the paper I reviewed here (that academic research is bad etc in a manner similar to what you describe). Notably., they say this is not the case for Life Sciences. So a lot of my intuitions that I draw upon (I never rely purely on results from papers on Econ or Social Science - I always try to match them up to experience) might be somewhat domain specific.
I see, yeah that's a shame. So much high-quality writing is tucked away into various Substacks.
We'll try to not throttle Substack links on our site, at least!
I'm not sure how to market it. Maybe put the link in your Twitter Bio and post screenshots of posts? Or get other Substacks to link to you (perhaps in exchange for you linking to them?) Submit your articles to hacker news??
I’m convinced that the private market can’t completely replace public research but entirely unconvinced that curiosity led research should be the primary recipient for public and philanthropic research funding. For every Jennifer doudna, there are hundreds of scientists doing research on something marginal that doesn’t even have a plausible pathway to human welfare. Of course it’s true that every big unexpected breakthrough came through the current model we have but that’s the only model we’ve really tried.
My intuition is that funders trying to maximize human welfare can be directionally right about which broad buckets are promising even if we don’t know exactly which breakthrough will pay off. And in so far as that’s true, we should have more FROs or problem focussed funding. I might be wrong ofc but I don’t think “interest” or scientific curiosity is as s inelastic to changes in funding and prestige and we should use that elasticity to direct Human Resources a bit more strategically. To be clear, I’m not a fan of centralized planning in general but you’re right that markets are insufficient but so is trusting scientific instinct
I think FROs are a good idea and they complement blue sky research pretty well. and we have too few of them. But I do not think they can completely replace curiosity driven research. Mostly because "you don't know what you don't know"
Frankly, the problem is that we allow a lot of low quality research from mediocre researchers, that is neither fundamental nor directly applied.
I must vehemently and respectfully disagree. You are making two egregious assumptions. First, why do you think bureaucrats (God forbid politicians influenced by lobbyists) with no skin in the game (spending other people's money) will make better decisions about what to fund than people who are genuinely interested and donating their own money? Look at how much is currently wasted just on ludicrous gender and race "research". Second and more importantly, you have a dearth of imagination about what people would do absent the government extraction of wealth from them. Look at how much science progressed before government funded research.
Well, I hope that was respectful (don't really care about the vehemence:)
Excuse my impertinence for returning to the scene of the crime, but in case I didn't make myself clear: almost all the problems with academia - administrative bloat, useless departments and research, political activism, ideological imbalance of employees, cancel culture, fear of ideas(!), etc. - can be traced to government funding and other interference. Cut the funding and all that will be minimized. Follow the money!
Bravo, great piece! Biological and natural sciences seem to especially benefit from the freedom of exploration because there's still a lot of room for discovery (curiosity-driven research, as you mention). To extend your piece to other disciplines: public funding is even more relevant for fields where we've already discovered most foundational things, but in which the freedom of exploration is also very important. Here, I'm thinking of the humanities and mathematics, where I sense that generating truly new, original knowledge or insights about the fundamental questions is especially difficult. (Of course, it's also harder for funders to assess which research projects bear more potential. And more difficult for mathematicians and humanists to get funding for their projects and research careers.)
Over time I've become convinced that we need a publicly funded employee training program for similar reasons. Employers are increasingly reluctant to invest in their employees because it's expensive and the employees could quit after being trained. Companies often hire away employees after they got training at a different company rather than pay to train the employees they already have. We publicly fund a lot of academia to train future employees but only up to a certain point.
Academic research does and still have a role to play, though yes I've also encountered the mass exodus out of academic from my PhD contemporaries (1/13 of us are in "academic research" in the form of non-profit research institute). I would add a distinction here: the type of research that is "academic", with uncertain and long payoffs, and more fundamental understandings of e.g. biology, is still critical and not well served by private sector. But the current form of that work, e.g. in Universities, does not seem to be doing so well, with all the mis-aligned incentives and bureaucracy. Other types of institutions (see https://spec.tech/) may help fill some gaps
I do not know if it's feasible to create a more complex control system. My approach to this is that you should give a bunch of smart people freedom to explore the directions they find important, even if it does not seem to lead to anything practical at first. The only requirement should be that it's high quality. Imo we should cut away funding for a lot of low quality research but this is easier said than done...
Companies are (obviously) inventivised to maximise profit, but academia isn't very inventivised to maximise fundamental research. Academia seems to be maximising for bureaucracy and government funding more than it is maximising (significant) research output. I have no idea how to create a system that inventivised fundamental research, but it is crucial to solving the crisis in academia.
agreed. The metascience people are thinking about a better system... not sure if they will succeed at coming up with a better system but fingers crossed
Excellent piece. Is the question whether or not private markets can *replace* research but: 1) Whether we are getting enough output relative to the $ we spend in academia 2) Whether that output is aligned with human flourishing 3) How the incentive structures from funding sources (primarily NIH/NSF) affect research directions and probably most importantly 4) How much of the science in academia is reaching regular people. Biotech, particularly drug development, is an anomalous case. I'm not sure it's good that biotech companies have outsourced R&D to academia and early stage startups. An industry with healthy profits ought to be pumping those profits back into R&D for a flywheel. If they are not, one should inquire why.
I worry most about (4) and believe a good chunk of the problem is the technology transfer offices at universities, who see their mission as trying to protect IP rather than maximize its value in society.
Finally, in contrast to bio, software progress has been driven largely by private labs and corporations. DeepMind and OpenAI and Meta have pushed the state of the art in AI forward considerably. The best AI and and software researchers are in universities, they are in the companies themselves.
Check out my last Substack and we should have a chat :)
This was a good read, thanks!
I think the part I disagree with is not that academia has been important and useful for some advances, e.g. CRISPR.
It's that a much better system can exist that is hard for us to currently imagine because we are so invested in the status quo.
I don't think it's hard to imagine a big pharma company spending a few billion a year on fundamental research. They already spend so much.
Nor is it difficult to imagine philanthropists stepping in to fund the most important basic research. E.g. the Gates foundation.
But I think that's just the starting line of science without public funding. Similar to how we've invented specific structures for startups and an ecosystem that includes VC's and angels, why wouldn't further structures evolve for science?
If many parties benefit over time from fundamental research, then isn't it just a coordination problem that we can solve with mutually beneficial contracts? This is after all, the motivation behind patents: incentivize someone to publish their innovation in exchange for monopoly rights for a number of years.
But even obvious for-profit structures will solve for advances like CRISPR. Think: research labs that spin-off a series of startups. VC's can invest in the labs which then invest in the spinoff companies.
Then, you say, what about the sleeper hits in academia, the basic science that takes a very long time and eventually makes a big impact.
Well, the decision of whether this is worth funding is literally a profit-maximizing calculation: do we get better returns to capital by investing in this basic research which eventually pays off in 50 years or by investing in the next-best alternative. Academia never does any such calculation and frequently wastes everyone's time.
And that's my next point: Academia sucks up all the energy and prestige for science and prevents better alternatives. Ambitious researchers join and then are way less impactful than they would be, because they actually don't get the opportunity to do the curiosity-focused research that they want. They have to grind out insignificant papers that adhere to current trends. Risky, innovative ideas are shut down or not funded because committees and established scientists with a reputation to preserve are not acting with the right incentives.
You can read more discussion in my market on whether Michael Nielsen will agree by 2030 that private-only funding for science is better than the status quo: https://manifold.markets/JamesGrugett/will-michael-nielsen-agree-by-2030
I think this is a really interesting debate. I am still very bullish that removing public funding for science would make it so much healthier, innovative, and cost-efficient. It would also save us from the negative externalities of bad ideas produced by a lot of public funding.
Thank you.
> Ambitious researchers join and then are way less impactful than they would be, because they actually don't get the opportunity to do the curiosity-focused research that they want. They have to grind out insignificant papers that adhere to current trends
My preferred method is increasing diversity in ways of funding science (FROs, academia, stuff like ARIA etc). I am writing a post on this and the sort of broader, theoretical considerations around this that is going to come out tomorrow or early next week and addresses exactly your point around conformism.
>Well, the decision of whether this is worth funding is literally a profit-maximizing calculation: do we get better returns to capital by investing in this basic research which eventually pays off in 50 years or by investing in the next-best alternative. Academia never does any such calculation and frequently wastes everyone's time.
I understand the consideration around this. The problem is, biotech is itself an enormously inefficient industry. I think the problem is that long time to product (and consequently feedback from the market) means that human incentives for career promotion are not very aligned to producing something that actually works. Biotech has a lot of dumb market failures. You should check this out this post I wrote about Roivant that goes into this a bit:
https://www.writingruxandrabio.com/p/what-does-ramaswamys-roivant-do
Also the work of Jack Scannell: https://pubmed.ncbi.nlm.nih.gov/36195754/ and some others analysing bias in pharma R&D: https://www.nature.com/articles/d41573-022-00157-4
As an additional example, we do not have good animal models for a lot of diseases, because it's basically in no one's interest to produce them. I think the solution to this is an FRO (I hope to write about this too in the future)
hmm what I will add though is that this stuff might be different in Life Sciences vs other industries. There is this new paper by Arora et al that I need to review in a future post that claims the opposite of the paper I reviewed here (that academic research is bad etc in a manner similar to what you describe). Notably., they say this is not the case for Life Sciences. So a lot of my intuitions that I draw upon (I never rely purely on results from papers on Econ or Social Science - I always try to match them up to experience) might be somewhat domain specific.
Btw, how did you find this post?
My cofounder posted it in my market as a comment! https://manifold.markets/JamesGrugett/will-michael-nielsen-agree-by-2030#avfh81lxn46
I'm not sure how he found this post. Sounds like he was reading through your Substack.
aw, nice! I am curious how to market my substack (X is now throttling links)
Oh my, have you seen this 😂: https://manifold.markets/stochasticcockatoo/will-ruxandra-teslo-and-jose-luis-r
😂😂😂😂
I see, yeah that's a shame. So much high-quality writing is tucked away into various Substacks.
We'll try to not throttle Substack links on our site, at least!
I'm not sure how to market it. Maybe put the link in your Twitter Bio and post screenshots of posts? Or get other Substacks to link to you (perhaps in exchange for you linking to them?) Submit your articles to hacker news??
I don't have any great ideas unfortunately.
Thanks! am doing various of the things you mentioned. I was just explaining why I asked you where you found the post! (Doing my market research)
I’m convinced that the private market can’t completely replace public research but entirely unconvinced that curiosity led research should be the primary recipient for public and philanthropic research funding. For every Jennifer doudna, there are hundreds of scientists doing research on something marginal that doesn’t even have a plausible pathway to human welfare. Of course it’s true that every big unexpected breakthrough came through the current model we have but that’s the only model we’ve really tried.
My intuition is that funders trying to maximize human welfare can be directionally right about which broad buckets are promising even if we don’t know exactly which breakthrough will pay off. And in so far as that’s true, we should have more FROs or problem focussed funding. I might be wrong ofc but I don’t think “interest” or scientific curiosity is as s inelastic to changes in funding and prestige and we should use that elasticity to direct Human Resources a bit more strategically. To be clear, I’m not a fan of centralized planning in general but you’re right that markets are insufficient but so is trusting scientific instinct
I think FROs are a good idea and they complement blue sky research pretty well. and we have too few of them. But I do not think they can completely replace curiosity driven research. Mostly because "you don't know what you don't know"
Frankly, the problem is that we allow a lot of low quality research from mediocre researchers, that is neither fundamental nor directly applied.
I must vehemently and respectfully disagree. You are making two egregious assumptions. First, why do you think bureaucrats (God forbid politicians influenced by lobbyists) with no skin in the game (spending other people's money) will make better decisions about what to fund than people who are genuinely interested and donating their own money? Look at how much is currently wasted just on ludicrous gender and race "research". Second and more importantly, you have a dearth of imagination about what people would do absent the government extraction of wealth from them. Look at how much science progressed before government funded research.
Well, I hope that was respectful (don't really care about the vehemence:)
Excuse my impertinence for returning to the scene of the crime, but in case I didn't make myself clear: almost all the problems with academia - administrative bloat, useless departments and research, political activism, ideological imbalance of employees, cancel culture, fear of ideas(!), etc. - can be traced to government funding and other interference. Cut the funding and all that will be minimized. Follow the money!
Bravo, great piece! Biological and natural sciences seem to especially benefit from the freedom of exploration because there's still a lot of room for discovery (curiosity-driven research, as you mention). To extend your piece to other disciplines: public funding is even more relevant for fields where we've already discovered most foundational things, but in which the freedom of exploration is also very important. Here, I'm thinking of the humanities and mathematics, where I sense that generating truly new, original knowledge or insights about the fundamental questions is especially difficult. (Of course, it's also harder for funders to assess which research projects bear more potential. And more difficult for mathematicians and humanists to get funding for their projects and research careers.)
Over time I've become convinced that we need a publicly funded employee training program for similar reasons. Employers are increasingly reluctant to invest in their employees because it's expensive and the employees could quit after being trained. Companies often hire away employees after they got training at a different company rather than pay to train the employees they already have. We publicly fund a lot of academia to train future employees but only up to a certain point.
Academic research does and still have a role to play, though yes I've also encountered the mass exodus out of academic from my PhD contemporaries (1/13 of us are in "academic research" in the form of non-profit research institute). I would add a distinction here: the type of research that is "academic", with uncertain and long payoffs, and more fundamental understandings of e.g. biology, is still critical and not well served by private sector. But the current form of that work, e.g. in Universities, does not seem to be doing so well, with all the mis-aligned incentives and bureaucracy. Other types of institutions (see https://spec.tech/) may help fill some gaps
I do not know if it's feasible to create a more complex control system. My approach to this is that you should give a bunch of smart people freedom to explore the directions they find important, even if it does not seem to lead to anything practical at first. The only requirement should be that it's high quality. Imo we should cut away funding for a lot of low quality research but this is easier said than done...
Companies are (obviously) inventivised to maximise profit, but academia isn't very inventivised to maximise fundamental research. Academia seems to be maximising for bureaucracy and government funding more than it is maximising (significant) research output. I have no idea how to create a system that inventivised fundamental research, but it is crucial to solving the crisis in academia.
agreed. The metascience people are thinking about a better system... not sure if they will succeed at coming up with a better system but fingers crossed