Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and EHR for FREE!

FDA Under Pressure To Deliver Clinical Decision Support Guidelines

Posted on November 10, 2016 I Written By

Anne Zieger is veteran healthcare consultant and analyst with 20 years of industry experience. Zieger formerly served as editor-in-chief of FierceHealthcare.com and her commentaries have appeared in dozens of international business publications, including Forbes, Business Week and Information Week. She has also contributed content to hundreds of healthcare and health IT organizations, including several Fortune 500 companies. Contact her at @ziegerhealth on Twitter or visit her site at Zieger Healthcare.

The world of clinical decision support technologies may change soon, as the FDA may soon be releasing guidelines on how it will regulate such technology. According to a new report in Politico, the agency has been working on such guidelines since 2011, but it’s not clear what standards it will use to establish these rules.

Software vendors in the CDS business are getting antsy. Early this year, a broad-based group known as the Clinical Decision Support Coalition made headlines when it challenged the agency to clarify the scope of CDS software it will regulate, as well as what it will require from any software that does fall under its authority.

At the time, the group released a survey which found that one-third of CDS developers were abandoning CDS product development due to uncertainty around FDA regulations. Of CDS developers that were moving ahead despite the uncertainty, the only two-thirds were seeing significant delays in development, and 20% of that group were seeing delays of greater than one year.

The delay has caught the attention of Congress, where Sens. Orrin Hatch (R-Utah) and Michael Bennet (D-Colo.) have filed the Medical Electronic Data Technology Enhancement for Consumers’ Health Act, legislation designed to resolve open questions around CDS software, but the problem still remains.

The FDA has had a research project in place since late 2014 which is creating and evaluating a CDS system for safe and appropriate use of antibiotics. The researcher-developed system generates alerts when a provider prescribes an antibiotic that poses a risk of serious cardiac adverse events for specific patients. Two of the 26 hospitals in the Banner Health network are participating in the study, one of which will use the system and the other which will not. The results aren’t due until April of next year.

It’s hard to say what’s holding the FDA up in this case, particularly given that the agency itself has put CDS guidance on his list of priority projects. But it could be a simple case of too much work and too few staff members to get the job done. As of late last year, the agency was planning to fill three new senior health scientist positions focused on digital health, a move which could at least help it keep up with the flood of new health technologies flooding in from all sides, but how many hours can they work?

The truth is, I’d submit, that health IT may be moving too quickly for the FDA to keep up with it. While it can throw new staff members at the problem, it could be that it needs an entirely new regulatory process to deal with emerging technology such as digital health and mobile device-based tools; after all, it seems to be challenged by dealing with CDS, which is hardly a new idea.

Correlations and Research Results: Do They Match Up? (Part 2 of 2)

Posted on May 27, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

The previous part of this article described the benefits of big data analysis, along with some of the formal, inherent risks of using it. We’ll go even more into the problems of real-life use now.

More hidden bias

Jeffrey Skopek pointed out that correlations can perpetuate bias as much as they undermine it. Everything in data analysis is affected by bias, ranging from what we choose to examine and what data we collect to who participates, what tests we run, and how we interpret results.

The potential for seemingly objective data analysis to create (or at least perpetuate) discrimination on the basis of race and other criteria was highlighted recently by a Bloomberg article on Amazon Price deliveries. Nobody thinks that any Amazon.com manager anywhere said, “Let’s not deliver Amazon Prime packages to black neighborhoods.” But that was the natural outcome of depending on data about purchases, incomes, or whatever other data was crunched by the company to produce decisions about deliveries. (Amazon.com quickly promised to eliminate the disparity.)

At the conference, Sarah Malanga went over the comparable disparities and harms that big data can cause in health care. Think of all the ways modern researchers interact with potential subjects over mobile devices, and how much data is collected from such devices for data analytics. Such data is used to recruit subjects, to design studies, to check compliance with treatment, and for epidemiology and the new Precision Medicine movement.

In all the same ways that the old, the young, the poor, the rural, ethnic minorities, and women can be left out of commerce, they can be left out of health data as well–with even worse impacts on their lives. Malanga reeled out some statistics:

  • 20% of Americans don’t go on the Internet at all.

  • 57% of African-Americans don’t have Internet connections at home.

  • 70% of Americans over 65 don’t have a smart phone.

Those are just examples of ways that collecting data may miss important populations. Often, those populations are sicker than the people we reach with big data, so they need more help while receiving less.

The use of electronic health records, too, is still limited to certain populations in certain regions. Thus, some patients may take a lot of medications but not have “medication histories” available to research. Ameet Sarpatwari said that the exclusion of some populations from research make post-approval research even more important; there we can find correlations that were missed during trials.

A crucial source of well-balanced health data is the All Payer Claims Databases that 18 states have set up to collect data across the board. But a glitch in employment law, highlighted by Carmel Shachar, releases self-funding employers from sending their health data to the databases. This will most likely take a fix from Congress. Unless they do so, researchers and public health will lack the comprehensive data they need to improve health outcomes, and the 12 states that have started their own APCD projects may abandon them.

Other rectifications cited by Malanga include an NIH requirement for studies funded by it to include women and minorities–a requirement Malanga would like other funders to adopt–and the FCC’s Lifeline program, which helps more low-income people get phone and Internet connections.

A recent article at the popular TechCrunch technology site suggests that the inscrutability of big data analytics is intrinsic to artificial intelligence. We must understand where computers outstrip our intuitive ability to understand correlations.

Correlations and Research Results: Do They Match Up? (Part 1 of 2)

Posted on May 26, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Eight years ago, a widely discussed issue of WIRED Magazine proclaimed cockily that current methods of scientific inquiry, dating back to Galileo, were becoming obsolete in the age of big data. Running controlled experiments on limited samples just have too many limitations and take too long. Instead, we will take any data we have conveniently at hand–purchasing habits for consumers, cell phone records for everybody, Internet-of-Things data generated in the natural world–and run statistical methods over them to find correlations.

Correlations were spotlighted at the annual conference of the Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics at Harvard Law School. Although the speakers expressed a healthy respect for big data techniques, they pinpointed their limitations and affirmed the need for human intelligence in choosing what to research, as well as how to use the results.

Petrie-Flom annual 2016 conference

Petrie-Flom annual 2016 conference

A word from our administration

A new White House report also warns that “it is a mistake to assume [that big data techniques] are objective simply because they are data-driven.” The report highlights the risks of inherent discrimination in the use of big data, including:

  • Incomplete and incorrect data (particularly common in credit rating scores)

  • “Unintentional perpetuation and promotion of historical biases,”

  • Poorly designed algorithmic matches

  • “Personaliziaton and recommendation services that narrow instead of expand user options”

  • Assuming that correlation means causation

The report recommends “bias mitigation” (page 10) and “algorithmic systems accountability” (page 23) to overcome some of these distortions, and refers to a larger FTC report that lays out the legal terrain.

Like the WIRED articles mentioned earlier, this gives us some background for discussions of big data in health care.

Putting the promise of analytical research under the microscope

Conference speaker Tal Zarsky offered both fulsome praise and specific cautions regarding correlations. As the WIRED Magazine issue suggested, modern big data analysis can find new correlations between genetics, disease, cures, and side effects. The analysis can find them much cheaper and faster than randomized clinical trials. This can lead to more cures, and has the other salutory effect of opening a way for small, minimally funded start-up companies to enter health care. Jeffrey Senger even suggested that, if analytics such as those used by IBM Watson are good enough, doing diagnoses without them may constitute malpractice.

W. Nicholson Price, II focused on the danger of the FDA placing too many strict limits on the use of big data for developing drugs and other treatments. Instead of making data analysts back up everything with expensive, time-consuming clinical trials, he suggested that the FDA could set up models for the proper use of analytics and check that tools and practices meet requirements.

One of exciting impacts of correlations is that they bypass our assumptions and can uncover associations we never would have expected. The poster child for this effect is the notorious beer-and-diapers connection found by one retailer. This story has many nuances that tend to get lost in the retelling, but perhaps the most important point to note is that a retailer can depend on a correlation without having to ascertain the cause. In health, we feel much more comfortable knowing the cause of the correlation. Price called this aspect of big data search “black box” medicine.” Saying that something works, without knowing why, raises a whole list of ethical concerns.

A correlation stomach pain and disease can’t tell us whether the stomach pain led to the disease, the disease caused the stomach pain, or both are symptoms of a third underlying condition. Causation can make a big difference in health care. It can warn us to avoid a treatment that works 90% of the time (we’d like to know who the other 10% of patients are before they get a treatment that fails). It can help uncover side effects and other long-term effects–and perhaps valuable off-label uses as well.

Zarsky laid out several reasons why a correlation might be wrong.

  • It may reflect errors in the collected data. Good statisticians control for error through techniques such as discarding outliers, but if the original data contains enough apples, the barrel will go rotten.

  • Even if the correlation is accurate for the collected data, it may not be accurate in the larger population. The correlation could be a fluke, or the statistical sample could be unrepresentative of the larger world.

Zarsky suggests using correlations as a starting point for research, but backing them up by further randomized trials or by mathematical proofs that the correlation is correct.

Isaac Kohane described, from the clinical side, some of the pros and cons of using big data. For instance, data collection helps us see that choosing a gender for intersex patients right after birth produces a huge amount of misery, because the doctor guesses wrong half the time. However, he also cited times when data collection can be confusing for the reasons listed by Zarsky and others.

Senger pointed out that after drugs and medical devices are released into the field, data collected on patients can teach developers more about risks and benefits. But this also runs into the classic risks of big data. For instance, if a patient dies, did the drug or device contribute to death? Or did he just succumb to other causes?

We already have enough to make us puzzle over whether we can use big data at all–but there’s still more, as the next part of this article will describe.

Healthcare Consent and its Discontents (Part 3 of 3)

Posted on May 18, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

The previous section of this article rated the pros and cons of new approaches to patient consent and control over data. Here we’ll look at emerging risks.

Privacy solidarity

Genetics present new ethical challenges–not just in the opportunity to change genes, but even just when sequencing them. These risks affect not only the individual: other members of her family and ethnic group can face discrimination thanks to genetic weaknesses revealed. Isaac Kohane said that the average person has 40 genetic markers indicating susceptibility to some disease or other. Furthermore, we sometimes disagree on what we consider a diseased condition.

Big data, particularly with genomic input, can lead to group harms, so Brent Mittelstadt called for moving beyond an individual view of privacy. Groups also have privacy needs (a topic I explored back in 1998). It’s not enough for an individual to consider the effect of releasing data on his own future, but on the future of family members, members of his racial group, etc. Similarly, Barbara Evans said we have to move from self-consciousness to social consciousness. But US and European laws consider privacy and data protection only on the basis of the individual.

The re-identification bogey man

A good many references were made at the conference to the increased risk of re-identifying patients from supposedly de-identified data. Headlines are made when some researcher manages to uncover a person who thought himself anonymous (and who database curators thought was anonymous when they released their data sets). In a study conducted by a team that included speaker Catherine M. Hammack, experts admitted that there is eventually a near 100% probability of re-identifying each person’s health data. The culprit in all this is burgeoning set of data collected from people as they purchase items and services, post seemingly benign news about themselves on social media, and otherwise participate in modern life.

I think the casual predictions of the end of anonymity we hear so often are unnecessarily alarmist. The field of anonymity has progressed a great deal since Latanya Sweeney famously re-identified a patient record for Governor William Weld of Massachusetts. Re-identifications carried out since then, by Sweeney and others, have taken advantage of data that was not anonymized (people just released it with an intuitive assumption that they could not be re-identified) or that was improperly anonymized, not using recommended methods.

Unfortunately, the “safe harbor” in HIPAA (designed precisely for medical sites lacking the skills to de-identify data properly) enshrines bad practices. Still, in a HIPAA challenge cited by Ameet Sarpatwari,only two of 15,000 individuals were re-identified. The mosaic effect is still more of a theoretical weakness, not an immediate threat.

I may be biased, because I edited a book on anonymization, but I would offer two challenges to people who cavalierly dismiss anonymization as a useful protection. First, if we threw up our hands and gave up on anonymization, we couldn’t even carry out a census, which is mandated in the U.S. Constitution.

Second, anonymization is comparable to encryption. We all know that computer speeds are increasing, just as are the sophistication of re-identification attacks. The first provides a near-guarantee that, eventually, our current encrypted conversations will be decrypted. The second, similarly, guarantees that anonymized data will eventually be re-identified. But we all still visit encrypted web sites and use encryption for communications. Why can’t we similarly use the best in anonymization?

A new article in the Journal of the American Medical Association exposes a gap between what doctors consider adequate consent and what’s meaningful for patients, blaming “professional indifference” and “organizational inertia” for the problem. In research, the “reasonable-patient standard” is even harder to define and achieve.

Patient consent doesn’t have to go away. But it’s getting harder and harder for patients to anticipate the uses of their data, or even to understand what data is being used to match and measure them. However, precisely because we don’t know how data will be used or how patients can tolerate it, I believe that incremental steps would be most useful in teasing out what will work for future research projects.

Healthcare Consent and its Discontents (Part 2 of 3)

Posted on May 17, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

The previous section of this article laid out what is wrong with informed consent today. We’ll continue now to look at possible remedies.

Could we benefit from more opportunities for consent?

Donna Gitter said that the Common Rule governing research might be updated to cover de-identified data as well as personally identifiable information. The impact of this on research, of course, would be incalculable. But it might lead to more participation in research, because 72% of patients say they would like to be asked for permission before their data is shared even in de-identified form. Many researchers, such as conference speaker Liza Dawson, would rather give researchers the right to share de-identified data without consent, but put protections in place.

To link multiple data sets, according to speaker Barbara Evans, we need an iron-clad method of ensuring that the data for a single individual is accurately linked. This requirement butts up against the American reluctance to assign a single ID to a patient. The reluctance is well-founded, because tracking individuals throughout their lives can lead to all kinds of seamy abuses.

One solution would be to give each individual control over a repository where all of her data would go. That solution implies that the individual would also control each release of the data. A lot of data sets could easily vanish from the world of research, as individuals die and successors lose interest in their data. We must also remember that public health requires the collection of certain types of data even if consent is not given.

Another popular reform envisioned by health care technologists, mentioned by Evans, is a market for health information. This scenario is part of a larger movement known as Vendor Relationship Management, which I covered several years ago. There is no doubt that individuals generate thousands of dollars worth of information, in health care records and elsewhere. Speaker Margaret Foster Riley claimed that the data collected from your loyalty card by the grocer is worth more than the money you spend there.

So researchers could offer incentives to share information instead of informed consent. Individuals would probably hire brokers to check that the requested uses conform to the individuals’ ethics, and that the price offered is fair.

Giving individuals control and haggling over data makes it harder, unfortunately, for researchers to assemble useful databases. First of all, modern statistical techniques (which fish for correlations) need huge data sets. Even more troubling is that partial data sets are likely to be skewed demographically. Perhaps only people who need some extra cash will contribute their data. Or perhaps only highly-educated people. Someone can get left out.

These problems exist even today, because our clinical trials and insurance records are skewed by income, race, age, and gender. Theoretically, it could get even worse if we eliminate the waiver that lets researchers release de-identified data without patient consent. Disparities in data sets and research were heavily covered at the Petrie-Flom conference, as I discuss in a companion article.

Privacy, discrimination, and other legal regimes

Several speakers pointed out that informed consent loses much of its significance when multiple data sets can be combined. The mosaic effect adds another layer of uncertainty about what will happen to data and what people are consenting to when they release it.

Nicolas Terry pointed out that American law tends to address privacy on a sector-by-sector basis, having one law for health records, another for student records, and so forth. He seemed to indicate that the European data protection regime, which is comprehensive, would be more appropriate nowadays where the boundary between health data and other forms of data is getting blurred. Sharona Hoffman said that employers and insurers can judge applicants’ health on the basis of such unexpected data sources as purchases at bicycle stores, voting records (healthy people have more energy to get involved in politics), and credit scores.

Mobile apps notoriously bring new leaks to personal data. Mobile operating systems fastidiously divide up access rights and require apps to request these rights during installation, but most of us just click Accept for everything, including things the apps have no right to need, such as our contacts and calendar. After all, there’s no way to deny an app one specific access right while still installing it.

And lots of these apps abuse their access to data. So we remain in a contradictory situation where certain types of data (such as data entered by doctors into records) are strongly protected, and other types that are at least as sensitive lack minimal protections. Although the app developers are free to collect and sell our information, they often promise to aggregate and de-identify it, putting them at the same level as traditional researchers. But no one requires the app developers to be complete and accurate.

To make employers and insurers pause before seeking out personal information, Hoffman suggested requiring that data brokers, and those who purchase their data, to publish the rules and techniques they employ to make use of the data. She pointed to the precedent of medical tests for employment and insurance coverage, where such disclosure is necessary. But I’m sure this proposal would be fought so heavily, by those who currently carry out their data spelunking under cover of darkness, that we’d never get it into law unless some overwhelming scandal prompted extreme action. Adrian Gropper called for regulations requiring transparency in every use of health data, and for the use of open source algorithms.

Several speakers pointed out that privacy laws, which tend to cover the distribution of data, can be supplemented by laws regarding the use of data, such as anti-discrimination and consumer protection laws. For instance, Hoffman suggested extending the Americans with Disabilities Act to cover people with heightened risk of suffering from a disability in the future. The Genetic Information Nondiscrimination Act (GINA) of 2008 offers a precedent. Universal health insurance coverage won’t solve the problem, Hoffman said, because businesses may still fear the lost work time and need for workplace accommodations that spring from health problems.

Many researchers are not sure whether their use of big data–such as “data exhaust” generated by people in everyday activities–would be permitted under the Common Rule. In a particularly wonky presentation (even for this conference) Laura Odwazny suggested that the Common Rule could permit the use of data exhaust because the risks it presents are no greater than “daily life risks,” which are the keystone for applying the Common Rule.

The final section of this article will look toward emerging risks that we are just beginning to understand.

Healthcare Consent and its Discontents (Part 1 of 3)

Posted on May 16, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Not only is informed consent a joke flippantly perpetrated on patients; I expect that it has inspired numerous other institutions to shield themselves from the legal consequences of misbehavior by offering similar click-through “terms of service.” We now have a society where powerful forces can wring from the rest of us the few rights we have with a click. So it’s great to see informed consent reconsidered from the ground up at the annual conference of the Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics at Harvard Law School.

Petrie-Flom annual 2016 conference

Petrie-Flom annual 2016 conference

By no means did the speakers and audience at this conference agree on what should be done to fix informed consent (only that it needs fixing). The question of informed consent opens up a rich dialog about the goals of medical research, the relationship between researchers and patients, and what doctors have a right to do. It also raises questions for developers and users of electronic health records, such as:

  • Is it ethical to save all available data on a person?

  • If consent practices get more complex, how are the person’s wishes represented in the record?

  • If preferences for the data released get more complex, can we segment and isolate different types of data?

  • Can we find and notify patients of research results that might affect them, if they choose to be notified?

  • Can we make patient matching and identification more robust?

  • Can we make anonymization more robust?

A few of these topics came up at the conference. The rest of this article summarizes the legal and ethical topics discussed there.

The end of an era: IRBs under attack

The annoying and opaque informed consent forms we all have to sign go back to the 1970s and the creation of Institutional Review Boards (IRBs). Before that lay the wild-west era of patient relationships documented in Rebecca Skloot’s famous Immortal Life of Henrietta Lacks.

IRBs were launched in a very different age, based on assumptions that are already being frayed and will probably no longer hold at all a few years from now:

  • Assumption: Research and treatment are two different activities. Challenge: Now they are being combined in many institutions, and the ideal of a “learning heath system” will make them inextricable.

  • Assumption: Each research project takes place within the walls of a single institution, governed by its IRB. Challenge: Modern research increasingly involves multiple institutions with different governance, as I have reported before.

  • Assumption: A research project is a time-limited activity, lasting generally only about a year. Challenge: Modern research can be longitudinal and combine data sets that go back decades.

  • Assumption: The purpose for which data is collected can be specified by the research project. Challenge: Big data generally runs off of data collected for other purposes, and often has unclear goals.

  • Assumption: Inclusion criteria for each project are narrow. Challenge: Big data ranges over widely different sets of people, often included arbitrarily in data sets.

  • Assumption: Rules are based on phenotypal data: diagnoses, behavior, etc. Challenge: Genetics introduces a whole new set of risks and requirements, including the “right not to know” if testing turns up an individual’s predisposition to disease.

  • Assumption: The risks of research are limited to the individuals who participate. Challenge: As we shall see, big data affects groups as well as individuals.

  • Assumption: Properly de-identified data has an acceptably low risk of being re-identified. Challenge: Privacy researchers are increasingly discovering new risks from combining multiple data sources, a trend called the “mosaic effect.” I will dissect the immediacy of this risk later in the article.

Now that we have a cornucopia of problems, let’s look at possible ways forward.

Chinese menu consent

In the Internet age, many hope, we can provide individuals with a wider range of ethical decisions than the binary, thumbs-up-thumbs-down choice thrust before them by an informed consent form.

What if you could let your specimens or test results be used only for cancer research, or stipulate that they not be used for stem cell research, or even ask for your contributions to be withdrawn from experiments that could lead to discrimination on the basis of race? The appeal of such fine-grained consent springs from our growing realization that (as in the Henrietta Lacks case) our specimens and data may travel far. What if a future government decides to genetically erase certain racial or gender traits? Eugenics is not a theoretical risk; it has been pursued before, and not just by Nazis.

As Catherine M. Hammack said, we cannot anticipate future uses for medical research–especially in the fast-evolving area of genetics, whose possibilities alternate between exciting and terrifying–so a lot of individuals would like to draw their own lines in the sand.

I don’t personally believe we could implement such personalized ethical statements. It’s a problem of ontology. Someone has to list all the potential restrictions individuals may want to impose–and the list has to be updated globally at all research sites when someone adds a new restriction. Then we need to explain the list and how to use it to patients signing up for research. Researchers must finally be trained in the ontology so they can gauge whether a particular use meets the requirements laid down by the patient, possibly decades earlier. This is not a technological problem and isn’t amenable to a technological solution.

More options for consent and control over data will appear in the next part of this article.

Another Quality Initiative Ahead of Its Time, From California

Posted on March 21, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

When people go to get health care–or any other activity–we evaluate it for both cost and quality. But health care regulators have to recognize when the ingredients for quality assessment are missing. Otherwise, assessing quality becomes like the drunk who famously looked for his key under the lamplight instead of where the key actually lay. And sadly, as I read a March 4 draft of a California initiative to rate health care insurance, I find that once again the foundations for assessing quality are not in place, and we are chasing lamplights rather than the keys that will unlock better care.

The initiative I’ll discuss in this article comes out of Covered California, one of the Unites States’ 13 state-based marketplaces for health insurance mandated by the ACA. (All the other states use a federal marketplace or some hybrid solution.) As the country’s biggest state–and one known for progressive experiments–California is worth following to see how adept they are at promoting the universally acknowledged Triple Aim of health care.

An overview of health care quality

There’s no dearth of quality measurement efforts in health care–I gave a partial overview in another article. The Covered California draft cites many of these efforts and advises insurers to hook up with them.

Alas–there are problems with all the quality control efforts:

  • Problems with gathering accurate data (and as we’ll see in California’s case, problems with the overhead and bureaucracy created by this gathering)

  • Problems finding measures that reflect actual improvements in outcomes

  • Problems separating things doctors can control (such as follow-up phone calls) with things they can’t (lack of social supports or means of getting treatment)

  • Problems turning insights into programs that improve care.

But the biggest problem in health care quality, I believe, is the intractable variety of patients. How can you say that a particular patient with a particular combination of congestive heart failure, high blood pressure, and diabetes should improve by a certain amount over a certain period of time? How can you guess how many office visits it will take to achieve a change, how many pills, how many hospitalizations? How much should an insurer pay for this treatment?

The more sophisticated payers stratify patients, classifying them by the seriousness of their conditions. And of course, doctors have learned how to game that system. A cleverly designed study by the prestigious National Bureau of Economic Research has uncovered upcoding in the U.S.’s largest quality-based reimbursement program, Medicare Advantage. They demonstrate that doctors are gaming the system in two ways. First, as the use of Medicare Advantage goes up, so do the diagnosed risk levels of patients. Second, patients who transition from private insurance into Medicare Advantage show higher risk not seen in fee-for-service Medicare.

I don’t see any fixes in the Covered California draft to the problem of upcoding. Probably, like most government reimbursement programs, California will slap on some weighting factor that rewards hospitals with higher numbers of poor and underprivileged patients. But this is a crude measure and is often suspected of underestimating the extra costs these patients bring.

A look at the Covered California draft

Covered California certainly understands what the health care field needs, and one has to be impressed with the sheer reach and comprehensiveness of their quality plan. Among other things, they take on:

  • Patient involvement and access to records (how the providers hated that in the federal Meaningful Use requirements!)

  • Racial, ethnic, and gender disparities

  • Electronic record interoperability

  • Preventive health and wellness services

  • Mental and behavioral health

  • Pharmaceutical costs

  • Telemedicine

If there are any pet initiatives of healthcare reformers that didn’t make it into the Covered California plan, I certainly am having trouble finding them.

Being so extensive, the plan suffers from two more burdens. First, the reporting requirements are enormous–I would imagine that insurers and providers would balk simply at that. The requirements are burdensome partly because Covered California doesn’t seem to trust that the major thrust of health reform–paying for outcomes instead of for individual services–will provide an incentive for providers to do other good things. They haven’t forgotten value-based reimbursement (it’s in section 8.02, page 33), but they also insist on detailed reporting about patient engagement, identifying high-risk patients, and reducing overuse through choosing treatments wisely. All those things should happen on their own if insurers and clinicians adopt payments for outcomes.

Second, many of the mandates are vague. It’s not always clear what Covered California is looking for–let alone how the reporting requirements will contribute to positive change. For instance, how will insurers be evaluated in their use of behavioral health, and how will that use be mapped to meeting the goals of the Triple Aim?

Is rescue on the horizon?

According to a news report, the Covered California plan is “drawing heavy fire from medical providers and insurers.” I’m not surprised, given all the weaknesses I found, but I’m disappointed that their objections (as stated in the article) come from the worst possible motivation: they don’t like its call for transparent pricing. Hiding the padding of costs by major hospitals, the cozy payer/provider deals, and the widespread disparities unrelated to quality doesn’t put providers and insurers on the moral high ground.

To me, the true problem is that the health care field has not learned yet how to measure quality and cost effectiveness. There’s hope, though, with the Precision Medicine initiative that recently celebrated its first anniversary. Although analytical firms seem to be focusing on processing genomic information from patients–a high-tech and lucrative undertaking, but one that offers small gains–the real benefit would come if we “correlate activity, physiological measures and environmental exposures with health outcomes.” Those sources of patient variation account for most of the variability in care and in outcomes. Capture that, and quality will be measurable.

Randomized Clinical Trial Validates BaseHealth’s Predictive Analytics

Posted on March 11, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

One of the pressing concerns in health care is the validity of medical and health apps. Because health is a 24-hour-a-day, 365-day-a-year concern, people can theoretically overcome many of their health problems by employing apps that track, measure, report, and encourage them in good behavior. But which ones work? Doctors are understandably reluctant to recommend apps–and insurers to cover them–without validation.

So I’ve been looking at the scattered app developers who have managed to find the time and money for randomized clinical studies. One recent article covered two studies showing the value of a platform that provided the basis for Twine Health. Today I’ll look at BaseHealth, whose service and API I covered last year.

BaseHealth’s risk assessment platform is used by doctors and health coaches to create customized patient health plans. According to CEO Prakash Menon, “Five to seven people out of 1,000, for instance, will develop Type II diabetes each year. Our service allows a provider to focus on those five to seven.” The study that forms the basis for my article describes BaseHealth’s service as “based on an individual’s comprehensive information, including lifestyle, personal information, and family history; genetic information (genotyping or full genome sequencing data), if provided, is included for cumulative assessment.” (p. 1) BaseHealth has trouble integrating EHR data, because transport protocols have been standardized but semantics (what field is used to record each bit of information) have not.

BaseHealth analytics are based on clinical studies whose validity seems secure: they check, for instance, whether the studies are reproducible, whether their sample sizes are adequate, whether the proper statistical techniques were used, etc. To determine each patient’s risk, BaseHealth takes into account factors that the patient can’t control (such as family history) as well as factors that he can. These are all familiar: cholesterol, BMI, smoking, physical activity, etc.

Let’s turn to the study that I read for this article. The basic question the study tries to answer is, “How well does BaseHealth predict that a particular patient might develop a particular health condition?” This is not really feasible for a study, however, because the risk factors leading to diabetes or lung cancer can take decades to develop. So instead, the study’s authors took a shortcut: they asked interviewers to take family histories and other data that the authors called “life information” without telling the interviewers what conditions the patients had. Then they ran the BaseHealth analytics and compared results to the patients actual, current conditions based on their medical histories. They examined the success of risk assignment for three conditions: coronary artery disease (CAD), Type 2 diabetes (T2), and hypertension (HTN).

The patients chosen for the study had high degrees of illness: “43% of the patients had an established diagnosis of CAD, 22% with a diagnosis of T2D and 70% with a diagnosis of HTN.” BaseHealth identified even more patients as being at risk: 74.6% for CAD, 66.7% for T2D, and 77% for HTN. It makes sense that the BaseHealth predictions were greater than actual incidence of the diseases, because BaseHealth is warning of potential future disease as well.

BaseHealth assigned each patient to a percentile chance of getting the disease. For instance, some patients were considered 50-75% likely to develop CAD.

The study used 99 patients, 12 of whom had to be dropped from the study. Although a larger sample would be better, results were still impressive.

The study found a “robust correlation” between BaseHealth’s predictions and the patients’ medical histories. The higher the risk, the more BaseHealth was likely to match the actual medical history. Most important, BaseHealth had no false negatives. If it said a patient’s risk of developing a disease was less than 5%, the patient didn’t have the disease. This is important because you don’t want a filter to leave out any at-risk patients.

I have a number of questions about the article: how patients break down by age, race, and other demographics, for instance. There was also an intervention phase in the study: some patients took successful measures to reduce their risk factors. But the relationship of this intervention to BaseHealth, however, was not explored in the study.

Although not as good as a longitudinal study with a large patient base, the BaseHealth study should be useful to doctors and insurers. It shows that clinical research of apps is feasible. Menon says that a second study is underway with a larger group of subjects, looking at risk of stroke, breast cancer, colorectal cancer, and gout, in addition to the three diseases from the first study. A comparison of the two studies will be interesting.

Randomized Controlled Trials and Longitudinal Analysis for Health Apps at Twine Health (Part 2 of 2)

Posted on February 18, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

The previous section of this article described the efforts of Dr. John Moore of Twine Health to rigorously demonstrate the effectiveness of a digital health treatment platform. As Moore puts it, Twine Health sought out two of the most effective treatment programs in the country–both Harvard’s diabetes treatment and MGH’s hypertension treatment are much more effective than the standard care found around the country–and then used their most effective programs for the control group of patients. The control group used face-to-face visits, phone calls, and text messages to keep in touch with their coaches and discuss their care plans.

The CollaboRhythm treatment worked markedly better than these exemplary programs. In the diabetes trial, they achieved a 3.2% reduction in diabetic patients’ A1C levels over three months (the control group achieved 2.0%). In the hypertension trial, 100% of patients reached a controlled blood pressure of less than 140/90 and the average reduction in blood pressure was 26mmHg (the control group had an average 16mmHg reduction and fewer than one-third of the patients went down less than 140/90).

What clinical studies can and cannot ensure

I see a few limitations with these clinical studies:

  • The digital program being tested combines several different intervention, as described before: reminders, messaging, virtual interactions, reports, and so on. Experiments show that all these things work together. But one can’t help wondering: what if you took out some time-consuming interaction? Could the platform be just as successful? But testing all the options would lead to a combinatorial explosion of tests.

    It’s important that interventions by coaches started out daily but decreased over the course of the study as the patient became more familiar and comfortable with the behavior called for in the care plans. The decrease in support required from the human coach suggests that the benefits are sustainable, because the subjects are demonstrating they can do more and more for themselves.

  • Outcomes were measured over short time frames. This is a perennial problem with clinical studies, and was noted as a problem in the papers. The researchers will contact subjects in about a year to see whether the benefits found in the studies were sustained. Even one year, although a good period to watch to see whether people bounce back to old behaviors, isn’t long enough to really tell the course of chronic illness. On the other hand, so many other life events intrude over time that it’s unfair to blame one intervention for what happens after a year.

  • Despite the short time frame for outcomes, the studies took years to set up, complete, and publish. This is another property of research practice that adds to its costs and slows down the dissemination of best practices through the medical field. The time frames involved explain why the researchers’ original Media Lab app was used for studies, even though they are now running a company on a totally different platform built on the same principles.

  • These studies also harbor all the well-known questions of external validity faced by all studies on human subjects. What if the populations at these Boston hospitals are unrepresentative of other areas? What if an element of self-selection skewed the results?

Bonnie Feldman, DDS, MBA, who went from dentistry to Wall Street and then to consulting in digital health, comments, “Creating an evidence base requires a delicate balancing act, as you describe, when technology is changing rapidly. Right now, chronic disease, especially autoimmune disease is affecting more young adults than ever before. These patients are in desperate need of new tools to support their self-care efforts. Twine’s early studies validate these important advances.”

Later research at Twine Health

Dr. Moore and his colleagues took stock of the tech landscape since the development of CollaboRhythm–for instance, the iPhone and its imitators had come out in the meantime–and developed a whole new platform on the principles of CollaboRhythm. Twine Health, of which Moore is co-founder and CEO, offers a platform based on these principles to more than 1,000 patients. The company expects to expand this number ten-fold in 2016. In addition to diabetes and hypertension, Twine Health’s platform is used for a wide range of conditions, such as depression, cholesterol control, fitness, and diet.

With a large cohort of patients to draw on, Twine Health can do more of the “big data” analysis that’s popular in the health care field. They don’t sponsor randomized trials like the two studies cited early, but they can compare patients’ progress to what they were doing before using Twine Health, as well as to patients who don’t use Twine Health. Moore says that results are positive and lasting, and that costs for treatment drop one-half to two-thirds.

Clinical studies bring the best scientific methods we know to validating health care apps. They are being found among a small but growing number of app developers. We still don’t know what the relation will be between randomized trials and the longitudinal analysis currently conducted by Twine Health; both seem of vital importance and they will probably complement each other. This is the path that developers have to take if they are to make a difference in health care.

Randomized Controlled Trials and Longitudinal Analysis for Health Apps at Twine Health (Part 1 of 2)

Posted on February 17, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Walking into a restaurant or a bus is enough to see that any experience delivered through a mobile device is likely to have an enthusiastic uptake. In health care, the challenge is to find experiences that make a positive difference in people’s lives–and proving it.

Of course, science has a time-tested method for demonstrating the truth of a proposition: randomized tests. Reproducibility is a big problem, admittedly, and science has been shaken by the string of errors and outright frauds perpetrated in scientific journals. Still, knowledge advances bit by bit through this process, and the goal of every responsible app developer in the health care space is the blessing offered by a successful test.

Consumer apps versus clinical apps

Most of the 165,000 health apps will probably always be labeled “consumer” apps and be sold without the expense of testing. They occupy the same place in the health care field as the thousands of untested dietary supplements and stem cell injection therapies whose promise is purely anecdotal. Consumer anger over ill-considered claims have led to lawsuits against the Fitbit device manufacturer and Lumosity mental fitness app, leading to questions about the suitability of digital fitness apps for medical care plans.

The impenetrability of consumer apps to objective judgment comes through in a recent study from the Journal of Medical Internet Research (JMIR) that asked mHealth experts to review a number of apps. The authors found very little agreement about what makes a good app, thus suggesting that quality cannot be judged reliably, a theme in another recent article of mine. One might easily anticipate that subjective measures would produce wide variations in judgment. But in fact, many subjective measures produced more agreement (although not really strong agreement) than more “objective” measures such as effectiveness. If I am reading the data right, one of the measures found to be most unreliable was one of the most “objective”: whether an app has been tested for effectiveness.

Designing studies for these apps is an uncertain art. Sometimes a study may show that you don’t know what to measure or aren’t running the study long enough. These possible explanations–gentler than the obvious concern that maybe fitness devices don’t achieve their goals–swirl about the failure of the Scripps “Wired for Health” study.

The Twine Health randomized controlled trials

I won’t talk any more about consumer apps here, though–instead I’ll concentrate on apps meant for serious clinical use. What can randomized testing do for these?

Twine Health and MIT’s Media Lab took the leap into rigorous testing with two leading Boston-area partners in the health care field: a diabetes case study with the Joslin Diabetes Center and a hypertension case study with Massachusetts General Hospital. Both studies compared a digital platform for monitoring and guiding patients with pre-existing tools such as face-to-face visits and email. Both demonstrated better results through the digital platform–but certain built-in limitations of randomized studies leave open questions.

When Dr. John Moore decided to switch fields and concentrate on the user experience, he obtained a PhD at the Media Lab and helped develop an app called CollaboRhythm. He then used it for the two studies described in the papers, while founding and becoming CEO of Twine Health. CollaboRhythm is a pretty comprehensive platform, offering:

  • The ability to store a care plan and make it clear to the user through visualizations.

  • Patient self-tracking to report taking medications and resulting changes in vital signs, such as glycemic levels.

  • Visualizations showing the patient her medication adherence.

  • Reminders when to take medication and do other aspects of treatment, such as checking blood pressure.

  • Inferences about diet and exercise patterns based on reported data, shown to the patient.

  • Support from a human coach through secure text messages and virtual visits using audio, video, and shared screen control.

  • Decision support based on reported vital statistics and behaviors. For instance, when diabetic patients reported following their regimen but their glycemic levels were getting out of control, the app could suggest medication changes to the care team.

The collection of tools is not haphazard, but closely follows the modern model of digital health laid out by the head of Partners Connected Health, Joseph Kvedar, in his book The Internet of Healthy Things (which I reviewed at length). As in Kvedar’s model, the CollaboRhythm interventions rested on convenient digital technologies, put patients’ care into their own hands, and offered positive encouragement backed up by clinical staff.

As an example of the patient empowerment, the app designers deliberately chose not to send the patient an alarm if she forgets her medication. Instead, the patient is expected to learn and adopt responsibility over time by seeing the results of her actions in the visualizations. In exit interviews, some patients expressed appreciation for being asked to take responsibility for their own health.

The papers talk of situated learning, a classic education philosophy that teaches behavior in the context where the person has to practice the behavior, instead of an artificial classroom or lab setting. Technology can bring learning into the home, making it stick.

There is also some complex talk of the relative costs and time commitments between the digital interventions and the traditional ones. One important finding is that app users expressed significantly better feelings about the digital intervention. They became more conscious of their health and appreciated being able to be part of decisions such as changing insulin levels.

So how well does this treatment work? I’ll explore that tomorrow in the next section of this article, along with strengths and weaknesses of the studies.