Randomized Controlled Trials and Longitudinal Analysis for Health Apps at Twine Health (Part 2 of 2)

Posted on February 18, 2016 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

The previous section of this article described the efforts of Dr. John Moore of Twine Health to rigorously demonstrate the effectiveness of a digital health treatment platform. As Moore puts it, Twine Health sought out two of the most effective treatment programs in the country–both Harvard’s diabetes treatment and MGH’s hypertension treatment are much more effective than the standard care found around the country–and then used their most effective programs for the control group of patients. The control group used face-to-face visits, phone calls, and text messages to keep in touch with their coaches and discuss their care plans.

The CollaboRhythm treatment worked markedly better than these exemplary programs. In the diabetes trial, they achieved a 3.2% reduction in diabetic patients’ A1C levels over three months (the control group achieved 2.0%). In the hypertension trial, 100% of patients reached a controlled blood pressure of less than 140/90 and the average reduction in blood pressure was 26mmHg (the control group had an average 16mmHg reduction and fewer than one-third of the patients went down less than 140/90).

What clinical studies can and cannot ensure

I see a few limitations with these clinical studies:

  • The digital program being tested combines several different intervention, as described before: reminders, messaging, virtual interactions, reports, and so on. Experiments show that all these things work together. But one can’t help wondering: what if you took out some time-consuming interaction? Could the platform be just as successful? But testing all the options would lead to a combinatorial explosion of tests.

    It’s important that interventions by coaches started out daily but decreased over the course of the study as the patient became more familiar and comfortable with the behavior called for in the care plans. The decrease in support required from the human coach suggests that the benefits are sustainable, because the subjects are demonstrating they can do more and more for themselves.

  • Outcomes were measured over short time frames. This is a perennial problem with clinical studies, and was noted as a problem in the papers. The researchers will contact subjects in about a year to see whether the benefits found in the studies were sustained. Even one year, although a good period to watch to see whether people bounce back to old behaviors, isn’t long enough to really tell the course of chronic illness. On the other hand, so many other life events intrude over time that it’s unfair to blame one intervention for what happens after a year.

  • Despite the short time frame for outcomes, the studies took years to set up, complete, and publish. This is another property of research practice that adds to its costs and slows down the dissemination of best practices through the medical field. The time frames involved explain why the researchers’ original Media Lab app was used for studies, even though they are now running a company on a totally different platform built on the same principles.

  • These studies also harbor all the well-known questions of external validity faced by all studies on human subjects. What if the populations at these Boston hospitals are unrepresentative of other areas? What if an element of self-selection skewed the results?

Bonnie Feldman, DDS, MBA, who went from dentistry to Wall Street and then to consulting in digital health, comments, “Creating an evidence base requires a delicate balancing act, as you describe, when technology is changing rapidly. Right now, chronic disease, especially autoimmune disease is affecting more young adults than ever before. These patients are in desperate need of new tools to support their self-care efforts. Twine’s early studies validate these important advances.”

Later research at Twine Health

Dr. Moore and his colleagues took stock of the tech landscape since the development of CollaboRhythm–for instance, the iPhone and its imitators had come out in the meantime–and developed a whole new platform on the principles of CollaboRhythm. Twine Health, of which Moore is co-founder and CEO, offers a platform based on these principles to more than 1,000 patients. The company expects to expand this number ten-fold in 2016. In addition to diabetes and hypertension, Twine Health’s platform is used for a wide range of conditions, such as depression, cholesterol control, fitness, and diet.

With a large cohort of patients to draw on, Twine Health can do more of the “big data” analysis that’s popular in the health care field. They don’t sponsor randomized trials like the two studies cited early, but they can compare patients’ progress to what they were doing before using Twine Health, as well as to patients who don’t use Twine Health. Moore says that results are positive and lasting, and that costs for treatment drop one-half to two-thirds.

Clinical studies bring the best scientific methods we know to validating health care apps. They are being found among a small but growing number of app developers. We still don’t know what the relation will be between randomized trials and the longitudinal analysis currently conducted by Twine Health; both seem of vital importance and they will probably complement each other. This is the path that developers have to take if they are to make a difference in health care.