when i got tenure earlier, i thought that would change how i work and live. it was true, but it wasn’t because of tenure but because of my thyroid cancer (see https://kyunghyuncho.me/sharing-some-good-news-and-some-bad-news/ if you’re curious.) when i was promoted to become a full professor, i thought that would change how i work and live, but to be frank, it didn’t. though, i started to think about what i should be able to think about, now that i have become a full professor with tenure, implying (at least in my mind) that i have an obligation not only to carry on as usual but also to think out of box. for the past year and counting, this has been inconvenient because my status quo isn’t bad at all but also interesting because i feel like i am perhaps at the right stage of my career to not only think carefully but also think somewhat outrageously (though, to be strict, whatever i think of or execute tends to be too little and often too late always.) anyhow, i thought i’d share one such thought an airport (SFO) while waiting to board my flight in … let’s see .. 25 minutes.
although i have always been on the side of benefiting from the society, making me difficult to see many corners of the society where people suffer from injustice and oblivion, a few experiences, including thyroid cancer as well as ramsey hunt syndrome and a few more, during the past decade or so, have made me think a lot about healthcare. on top of these personal experiences, my research collaborations with my good colleagues at NYU Langone, including my own former postdocs, give me many opportunities to think of the current healthcare system as well as how the ongoing technological innovation may affect and change the healthcare system in the future.
healthcare is an interesting beast. it’s a complex system in which no one makes money (or at least everyone must pretend to make no money) and everyone suffers due to resource constraints (if no one suffers at all, we are probably in a situation without any resource constraints.) it’s a very difficult system for anyone to make any outrageous claim, unless they are totally oblivious of the healthcare system or have ulterior motives. in addition to its critical and sensitive nature, this makes it difficult to change how things work in healthcare; there is largely only downside for anyone to try to change the existing system. but, being a tenured full professor, i feel like i can and should think of some outrageous ideas about healthcare and share it with a broader audience; no ulterior motive, but my self-righteous feeling of obligation.
i want to start from this premise; “no decision in healthcare is certain.” when a doctor sees a patient, the doctor tries their best to obtain as much information about the patient as possible, combine this information with their best knowledge, often based on our incomplete understanding of physics, biology, chemistry, human physiology and environmental factors, and come up with some diagnosis. once this diagnosis based on incomplete information, the doctor must come up with some treatment plan, including doing nothing, based on the imperfect knowledge of treatment options and their impacts. in fact, this is perhaps the worst decision making process you can imagine in that the amount of known (observed) information is significantly smaller than that of unknown (unobserved) information. despite the amazing rate at which we are making progress in all these directions, there are so much that will never be revealed to us for doctors to make low-uncertainty, or highly certain, diagnosis as well as accompanying treatment plan. this applies to every single stage in overall healthcare, including all clinicians, pharmacists, insurance claim handlers, insurance underwriters, clinical trial designers, regulatory agencies, etc.
in order to assist in this process, our society over the past 70 years has adopted the idea of clinical trials, or more technically randomized controlled trials. the basic idea is relatively straightforward; in order to measure the causal effect of a particular treatment (which can be a combination of treatments) on the outcome (that is, whether the treatment successfully alleviated symptoms), we must recruit participants from a generally representative population, split them into the treatment (ones who receive the actual potential treatment) and control (ones who receive sugar pills) groups and see if the former exhibit more favourable outcomes on average. by now, thanks to the recent covid-19 pandemic, everyone who’s been reading newspapers during the past four years knows about this procedure. once such a randomized controlled trial is successful (i.e., the treatment group resulted in a “significantly” more favourable outcome than the control group did,) we let doctors consider this treatment as a serious option to consider prescribing to patients who are reasonably similar to the participants to this study, ask insurance companies to reimburse the cost of such treatment and use taxpayers’ money to support those insurance companies, in order to ensure that the society continues to function better than before.
it all sounds great, except that there are a few issues. you probably could see it immediately that this protocol is extremely costly and potentially extremely harmful to a lot of people. for every treatment or treatment combination, we must recruit enough participants to draw a statistically meaningful conclusion. we cannot use this protocol for rare diseases, since we will likely never find enough people with the condition to satisfy statistical constraints. we cannot use this protocol for highly experimental treatments, because they may end up accelerating or causing other side effects to participants. such a study can only be run on diseases for which we can relatively quickly check their progressions, which is often not the case for many sophisticated diseases (for instance, if we want to test a drug for aging-related diseases, we may need to wait several decades to check their effect.) even more critically, these studies are only valid among the population from which the participants were drawn, and if anything, we all now that the world never stops changing; what we thought worked will likely won’t work for another population as well as for the same population in several years. we already know of this issue thanks to various seasonal infectious diseases, such as seasonal flu and now sadly coronavirus. finally, these constraints effectively prevent us from rigorously studying the effect of a complicated treatment combination, largely due to the issue of curse of dimensionality (or simply read it as “exponential growth isn’t good.”)
what are issues here? the major issue here is the assumption that such a “treatment effect” is stationary; that is, once it’s estimated correctly, we believe this would continue to be the correct treatment effect. this is certainly not the case, since both the population drifts and the effect of the treatment drifts (think of ever-evolving variants of sars-cov-2 as well as seasonal flu.) furthermore, it is extremely limiting to have to recruit a large group of participants for each and every treatment combination, in order for us to make confident conclusions on those treatments’ effects on outcomes.
now, here comes an outrageous (though, statistically and algorithmically sane) idea. bear with me.
we must exploit (or benefit from) inherent uncertainty in every decision made in healthcare. when a doctor is faced with two seemingly undistinguishable (from the perspective of an expected outcome), we can essentially run a part of a clinical trial by choosing one of these two treatment options at random. we are effectively assigning this patient into one of these two treatment options, enabling us to break any confounding between this pair of treatment options and the outcome. of course, doing so once doesn’t get us anything beyond giving this particular doctor the peace of mind that their choice was justified whatever the outcome ends up being.
but, what if we ask every clinician in the entire healthcare system to follow this? every time, any clinician is faced with a situation where there are more than one decision options (either diagnostic or treatment), they simply record those reasonable options in their mind and let a random number generator (could very well be their phones) break the tie. for instance, a patient comes to a clinic with mild headache. the doctor at the clinic can either recommend the patient to take some tylenol or just take a rest, since it may very well be nothing but headache induced by working too hard. the doctor doesn’t choose one themself, since both have the same level of uncertainty, and lets their random number generator make the decision.
of course, sometimes, one option is known to be significantly better than the other option “on average”. even in this case, we must acknowledge that this is “on average” and does not apply to every patient. in fact, what we know about online learning, bandits and reinforcement learning tells us that we should not simply take the best option “on average”, as this may result in a significantly higher regret (i won’t go into details here, but a regret is literally a regret for choosing a worse option in hindsight.) rather, we want clinicians to choose the option proportionally to our beliefs on the effectiveness of these treatments. for this, we can ask each clinician to assess (roughly) what their belief one option is better than another before letting a random number generator pick one accordingly.
perhaps most importantly, in this scheme, we are _not_ losing anything relative to the conventional practice where we ask each clinician to pick one option at their best to “do no harm”, since most of these treatment options are equally (in)effective and because we simply don’t know which option is the best one. in fact, we will likely result in a higher benefit overall, since we avoid any incorrect option being selected over and over across the whole population. and, this is not a hypothetical scenario and has sadly occurred multiple times in humanity’s modern history.
on the other hand, we gain a lot from this practice. as long as we record these alternative options and randomly selected choices, we end up effectively running the largest-scale and longest-running clinical trials for all available treatment options. by combing through this vast amount of data, we would be able to (almost) accurately estimate causal effects of various treatment options on a diverse set of outcomes offline. if we adopt this protocol at the society level, these estimates will be valid for the entire society (or at least those that are in the mainstream healthcare system, which is not 100% in many places, including US). furthermore, these estimates can be continuously updated to be up-to-date, since we can easily use latest data to re-estimate them. this will allow us to even discover the (positive) effect of existing treatments (or their combinations) on a diverse set of outcomes that we never anticipated in advance. funnily, this is not a fresh idea at all, but has been studied extensively under the names of online learning, bandits and reinforcement learning.
since we cannot expect any clinician to consider all possible treatment options even collectively, this will nevertheless result in somewhat biased estimate of treatment effects. the major bias would come from the fact that no clinician can recommend a yet-unknown treatment. we would thus still continue to run clinical trials for new medicines and treatment options (such as novel surgical techniques, etc.) it will nevertheless be much better than simply looking correlations between various treatment options that were not randomized and outcomes or expecting large-scale, long-term clinical trials for all possible treatment combinations. the former is simply wrong in most cases, and the latter is sadly untenable in conventional healthcare, where it is all about optimization under severe resource constraints.
i can expand on this further and talk about it on and on, but i just boarded the plane, and they are about to close the doors to start taxing and take off toward NYC. so, i’m cutting it short here, but i’d love to hear your thoughts on this; too naive? outrageous? simply misinformed? share your thoughts!