On Lumina and the need for human trial data
In which I analyse the Lumina data and have to face the inner conflict of supporting self-experimentation but also wanting actual trial data
One of the descriptors of my blog is “anti-safetyist” — I think in general our society is too risk-averse, with regulatory ratchets everywhere threatening to stifle innovation. Through reading Alex Tabarrok I have become convinced of the failures of the FDA, the need for allowing organ donor markets and so on. I’ve talked to Jake Seliger about his struggles trying to get enrolled in trials and facing administrative hurdles at every step. But I am also an empiricist. As a PhD student in Biology and an analyst of biotech companies, I have learned one thing : what looks good on paper seldom ends up having real-world success. In science, encountering failure is more the rule than the exception. To determine if a new intervention works well in humans, I need to see data from randomized trials in … humans! Not data from mice, not data from in vitro experiments, not fancy diagrams or nice stories about the product or whatever else. No matter how neat. To those of you not in biology, this might sound unnecessary, but hear me out: the failure rate of translating results from animal models to humans stands at ~92%. At the other end of the spectrum, GLP-1 agonists, considered miracle drugs, were initially developed as a treatment for diabetes, and their weight-loss activity discovered by accident. There is now accumulating evidence they might be effective against a host of seemingly unrelated diseases, something no one expected until they were tried in actual humans. All this to say biology is unpredictable.
These two conflicting impulses stirred some turmoil in my heart upon initially learning about Lumina, a product supposed to prevent cavities (forever?) that swiftly gained popularity among rationalists.
Overall, Scott Alexander approached this quite cautiously — he has been very careful to not make exaggerated claims. He also framed this as a community effort with the final goal of potentially getting attention from the FDA and actually organizing some high quality trials. But others have been markedly less so: when Lumina was released, twitter was full of claims that this treatment is likely to “cure caries with only one application”. Some, like
, say that if Scott Alexander told him to jump off a bridge, he would consider it, so Scott’s endorsement of this product is enough for him to trust it. Meanwhile, Scott politely thanks him but also says “no, no, no, I actually had not taken this myself when I wrote original post, I give it only a 30% probability to work as a normal caries treatment, not even a cure.”In my opinion, many of these endorsements failed to properly explain the caveats of the animal model data that we do have. I am not interested in adjudicating where this type of information comes from: whether it was influencers themselves who decided to describe it as such or the company. What I do want is to provide some clarity into the strength of evidence we have at the moment — at least based on what I have seen so far from the existing endorsements.
Rat data is not human data
The idea behind Lumina is neat, simple and promising. Lumina is a form of replacement therapy: the idea is to displace the naturally occurring cariogenic species S. mutans with a genetically modified version of it that produces the more benign ethanol instead of lactic acid. Lactic acid is a substance that is thought to be behind much of the cariogenic impacts of S. mutans. The optimized strain would have strong colonizing potential, ensuring its persistence (with the company claiming a one-time application is enough for a lifetime), but would be much less pathogenic. I will not further dwell on this, as Scott Alexander himself has a quite detailed description of the mechanism. Some might shudder at the thought of allowing a genetically modified organism to colonize one’s mouth. This is, in my opinion, not a very valid concern. There is little reason to believe genetic modification in itself is dangerous. My main gripe is that the experimental evidence so far comes almost entirely from studies using animal models, and small sample size studies at that.
One of the biggest problems in biomedical research is the tools we have to model disease and test interventions are inadequate: whether in vitro systems or in vivo animal models, they fail to accurately reproduce human physiology. The human data that we do have before clinical trials suffers from being purely observational. This has resulted in misleading conclusions, such as widely publicized findings suggesting low doses of alcohol may be beneficial—claims later debunked through research designed to tease out causation. For these reasons, the definitive evaluation of any new therapy is the randomized control trial (RCT) in humans. These trials serve as a critical "moment of truth," with about 90% of drugs that pass preclinical stages failing during clinical testing and billions wasted in the development process. To improve the odds of success, one can use informed strategies—for instance, drugs developed based on human genetic data are more than twice as likely to be successful compared to those that aren't. This approach leverages the experiment conducted by nature on millions of humans: if genetic variants are significantly associated with a disease, the proteins they encode are more likely to be causal contributors rather than merely correlated. But even with such improved decision-making tools, the odds are extremely low.
Although there is evidence that incentives within pharma companies can be misaligned and thus contribute to lower than optimal success rates, if one believes at all in the efficient market hypothesis, one has to acknowledge therapeutic discovery is an incredibly hard problem, and that’s not just because bad academics have conspired to halt innovation. So, when judging the Lumina results, one has to keep in mind these dismal success rates as a sort of baseline and the inherent difficulties of deriving any strong conclusions from animal research. Now, one could argue that this product is different because the mouth is a somewhat more isolated system from the rest of the body and caries formation is an easier to understand process than more complex diseases like cancer. But equally, one could say the opposite: from what I have seen, caries are a relatively underresearched topic and in my attempt to understand the literature, I was shocked by just how little information there was compared to other diseases. In any case, endless debates can take place on this point, so it’s better to just look at the data.
The main interventional, in vivo data supporting the efficacy of the specific strain Lumina uses comes from a 2000 paper1 in which rats were infected with a supercolonizing and non lactate producing strain of S. mutans, BCS3-L1, and compared to rats infected with a strain of S. mutans that is supercolonizing but lactate producing (JH1140) as well as control rats2. All rats were fed a caries-inducing diet. The table below best summarises the results of the experiment, although I could not find the sample size anwhere3 in the paper.
Now, there are two things that popped out to me from this table. Firstly, and contrary to some claims, rats treated with BCS3-L1 do indeed develop caries, albeit fewer than those infected with the highly pathogenic JH1140 strain. This suggests that BCS3-L1 is not fully protective against caries, even in a highly constrained experimental model. The persistence of caries in BCS3-L1 infected rats could be due to other, non lactate-dependent cariogenic mechanisms employed by S. mutans, such as polysaccharide synthesis (which can contribute to biofilm creation), or the presence of other cariogenic bacterial species.
The other important caveat is that BCS3-L1 develops about as many caries as the Control mice, which have not been treated with any strain (the values for Control are a bit higher, but it’s not significant.) Now, what does this experiment mean overall? It shows that a strain of lab rats which are deliberately infected with a highly colonizing and known to be pathogenic strain of S mutans and are fed a cariogenic diet get twice as many cavities as the rats infected with the modified version of the bacteria. But also, that Control rats do not show significantly more caries than BCS3-L1 infected ones. Now, how you interpret these results depends a lot on whether you think the steady-state human mouth is better modelled by the JH1140 infected rat mouth or the control rat mouth. On one hand, we do know human mouths are often colonised by S. mutans strains (albeit not the JH1140 strain4), so you could argue JH1140 infected rats model human mouths better. But equally, not many people are kept in a lab for most of their lives and then get high concentrations of bacteria aggressively rubbed on their gums. Also, not many humans are rats, so there is that difference too.
The difficulty of accurately capturing the relevant mechanisms of a disease in a model organism is not to be held against the researchers. This is common across biology. Indeed, a lack of biologically accurate animal models is considered to be one of the prime reasons for therapeutic failure. The difficulties of inducing caries specifically in rats are conveyed by the following paragraph from a review on animal models of caries:
When maintained on natural diets the rat does not normally develop caries. Some strains of rat are highly resistant to the onset of the disease in any dietary circumstances, and it is apparent that genetic factors play an important part in caries susceptibility in the rat. Nevertheless certain strains of caries-susceptible animals have been-bred and are available for the study of the disease.
Given the intraspecies heterogeneity in caries formation and the fact that this is all dependent on strain of rat, I think caution when extending such findings to humans should be exercised. This is especially true given that, from I have seen, the sample
There are other issues too, mostly pertaining to how this has been communicated by those promoting the treatment. For once, there seems to be some confusion among some as to exactly what strain is being used in the product (a pretty big problem.) Most people, including Scott Alexander, seem to agree the strain used in the final product is BCS3-L1, but Yishan, one of the more prominent promoters of this treatment, as well as an investor in the company, seems to refer to another strain, A2JM, when he argues that the Lumina product is more “genetically stable” than wild-type bacteria. Then, there is also the claim that the strain would colonize the human mouth for a lifetime without the need for reapplication. Again, this seems needlessly optimistic and the only evidence I have seen for this is hearsay in the vein of “Professor Hillmann found it to be very persistent in the mouths of volunteers.”
In vitro studies are not enough
At this point, some might quibble that we do have human in vitro data: that is data from human teeth in a lab. Needless to say, isolated teeth in a lab are not like teeth in a human body. The tooth in a human body is connected to an immune system. It’s also part of an oral cavity that is home to a myriad of other bacteria apart from S. mutans. Some of them beneficial, some of them not.
The discovery that S. mutans is prevalent in atherosclerotic plaques strikingly illustrates the interconnectedness of bodily systems. Research using mouse models of atherosclerosis as well as epidemiological evidence from humans has identified a potential causative role for S. mutans in the progression of these plaques. Additionally there is strong evidence that these bacteria are implicated in endocarditis. If you took the Lumina S. mutans strain and are panicked — don’t be! My point was more to showcase why teeth in a lab are not teeth in a human.
Luckily, BCS3-L1 seems to be derived from the serotype that is least capable of affecting blood vessels, at least according to in vitro studies - serotype c (caveat: even if serotype c is not involved in atherogenesis in the exact way tested in the study above, it could still be triggering inflammation.) Overall, this is a version of it specifically engineered for increased colonisation potential, so I would have tested this strain in a mouse model at least, specifically when it comes to its atherogenic potential. This would be mainly for safety reasons, but also, who knows? Maybe it actually has a beneficial effect on heart disease, by displacing potentially more pathogenic serotype k, in which case it could plausibly be thought to help with heart disease too. The point is we do not know, until we get at least some experimental validation.
At this point, one might ask what is the overall risk-benefit assessment for an individual? Overall, while I don't buy into the “one application and you’ll be cavity free forever” claims, I think it has a reasonable chance of being helpful against caries. What I am most doubtful of is its “lifelong persistence” claims. There are a lot of things that could happen throughout a lifetime and the certainty of this assertion is not really believable. However, I can believe that for some people it could be unusually persistent. It’s likely that the impact will be person-specific, as well. To optimize the risk benefit ratio, one could consider whether they are susceptible to caries (family, medical history.) Another good idea is testing oneself for the presence of S. mutans: ~10% of adults do not have detectable S. mutans in their saliva (for children this percentage is much higher.) Even among adults who do have S. mutans, the amount of bacteria varies in a way that seems to be correlated with caries incidence, so the exact amount seems like another important variable to consider. I personally would not get this product, simply because I have never had issues with caries and do not expect to have given my family history. However, I am at risk of periodontosis, or inflammation of the gum, at least as big of a concern for oral health as caries. Although S. mutans has also been implicated in periodontosis, the data is much less clear than for caries, with other species seemingly being more important. There is also a small chance that the new strain would disturb the natural microbiome in a way that would cause even more harm. What’s more, I already have to do all sorts of stuff to prevent periodontosis, so I expect all these measures to be more than enough for my already not very prone to caries mouth.
Science is not an act of faith
In his book, Michael Strevens presents a compelling argument that the inception of modern science can be attributed to a singular, seemingly irrational idea:“strip away all previous knowledge - such as theological, metaphysical or political beliefs - and channel unprecedented energy into observation and experiment.” Strevens refers to this scientific principle as the iron rule of explanation. He shows that this rule, by being uncompromisingly rigid, helps transcend individual biases and inexorably guides humanity towards understanding nature. This is something that we pretty much take for granted now, but it has not always been obvious. As Trevor Klee points out, past generations have had to fight to establish it. Aristotle combined philosophy with his pursuit of science, while even early modern scientists were still trying to fit their theories into accepted religious paradigms. And all of these pursuits hindered them, delaying scientific progress by millennia! The idea that the ultimate test for one’s ideas has to be found in empirical validation, no matter how aesthetically or conceptually pleasing they are otherwise, is one of those many extremely unnatural norms that humans seem to have adopted during the Enlightenment and to which we can attribute much of our success. The insistence on randomized human clinical trials is a direct application of this broader norm.
This norm is highly artificial and for this reason, fragile. Human nature compels us to believe in ideas endorsed by our peers or social groups, making the subjective seem objectively true. Besides that, humans want to believe and are drawn to aesthetics and social proof. Empirical reality is often much less fascinating and more disappointing than stories and grandiose narratives. One of the things that I have noticed during this whole Lumina situation is the prevalence of claims based on hearsay: “my friend said that they felt better after this treatment” or “based on stories I have heard of experiments.” There is even some hint of religious sentiment, especially in Hanania’s post about trusting Scott’s judgement above all else, like a sort of entity that exists above mere mortal endeavours like clinical trials. Although he is one of the very few people to openly admit it, I think this rationale is quite widespread. I have specifically refused to engage with such evidence. While potentially valuable in narrow cases, this should not become the norm in a science- driven community. I perceive the strict adherence to the “data from clinical trials in humans” or, at the very least, “data that’s more than hearsay” criterion as analogous to the staunch defense of free speech norms. Both practices are artificial constructs that have emerged relatively recently in human history and go against our instincts. Moreover, they share a susceptibility to slippery slopes, where initial deviations can lead to greater and more frequent compromises. If standing up for this norm makes me boring and not sufficiently excited, then so be it.
And here I am turning to what this means in a broader context. In the sections above, I discussed whether taking Lumina makes sense from an individual perspective. But for a community, it has implications beyond that. I do think that FDA takes too long and is too much on the side of safety. But the response to that should be to push for easier enrolment into randomized human trials, or organize your own, not to eschew them altogether. It does not seem that there will be such trials for Lumina, so unfortunately all the causal effects of this therapy will be hard to disentangle correctly. Precisely because I support self-experimentation, I believe this is an issue. Scott has actually specifically mentioned in his post that he hopes people taking this treatment would maybe pressure the FDA into reconsidering the issue, but I wonder whether FDA needs to be involved at all and whether Lumina could just organise a trial themselves. This could be an interesting experiment that would actually combine the daring of self-experimentation with the rigour of Science, and show “The Establishment” things can be done quicker, but also in a legit way.
This is the main paper cited in further studies as well as in the more optimistic write-ups of the evidence.
A related experiment in the paper had a sample size of 10.
