Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and EHR for FREE!

Clinical Decision Support Should Be Open Source

Posted on January 26, 2015 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Clinical decision support is a long-standing occupant of the medical setting. It got in the door with electronic medical records, and has recently received a facelift under the term “evidence based medicine.” We are told that CDS or EBM is becoming fine-tuned and energized through powerful analytics that pick up the increasing number of patient and public health data sets out in the field. But how does the clinician know that the advice given for a treatment or test is well-founded?

Most experts reaffirm that the final word lies with the physician–that each patient is unique, and thus no canned set of rules can substitute for the care that the physician must give to a patient’s particular conditions (such as a compromised heart or a history of suicidal ideation) and the sustained attention that the physician must give to the effects of treatment. Still, when the industry gives a platform to futurists such as Vinod Khosla who suggest that CDS can become more reliable than a physician’s judgment, we have to start demanding a lot more reliability from the computer.

It’s worth stopping a moment to consider the various inputs to CDS. Traditionally, it was based on the results of randomized, double-blind clinical trials. But these have come under scrutiny in recent years for numerous failings: the questionable validity of extending the results found on selected test subjects to a broader population, problems reproducing results for as many as three quarters of the studies, and of course the bias among pharma companies and journals alike for studies showing positive impacts.

More recently, treatment recommendations are being generated from “big data,” which trawl through real-life patient experiences instead of trying to isolate a phenomenon in the lab. These can turn up excellent nuggets of unexpected impacts–such as Vioxx’s famous fatalities–but suffer also from the biases of the researches designing the algorithms, difficulties collecting accurate data, the risk of making invalid correlations, and the risk of inappropriately attributing causation.

A third kind of computerized intervention has recently been heralded: IBM’s Watson. However, Watson does not constitute CDS (at least not in the demo I saw at HIMSS a couple years ago). Rather, Watson just does the work every clinician would ideally do but doesn’t have time for: it consults thousands of clinical studies to find potential diagnoses relevant to the symptoms and history being reported, and ranks these diagnoses by probability. Both of those activities hijack a bit of the clinician’s human judgment, but they do not actually offer recommendations.

So there are clear and present justifications for demanding that CDS vendors demonstrate its reliability. We don’t really know what goes into CDS and how it works. Meanwhile, doctors are getting sick and tired of bearing the liability for all the tools they use, and the burden of their malpractice insurance is becoming a factor in doctors leaving the field. The doctors deserve some transparency and auditing, and so do the patients who ultimately incorporate the benefits and risks of CDS into their bodies.

CDS, like other aspects of the electronic health records into which it is embedded, has never been regulated or subjected to public safety tests and audits. The argument trotted out by EHR vendors–like every industry–when opposing regulation is that it will slow down innovation. But economic arguments have fuzzy boundaries–one can always find another consideration that can reverse the argument. In an industry that people can’t trust, regulation can provide a firm floor on which a new market can be built, and the assurance that CDS is working properly can open up the space for companies to do more of it and charge for it.

Still, there seems to be a pendulum swing away from regulation at present. The FDA has never regulated electronic health records as it has other medical software, and has been carving out classes of medical devices that require little oversight. When it took up EHR safety last year, the FDA asked merely for vendors to participate voluntarily in a “safety center.”

The prerequisite for gauging CDS’s reliability is transparency. Specifically, two aspects should be open:

  • The vendor must specify which studies, or analytics and data sets, went into the recommendation process.

  • The code carrying out the recommendation process must be openly published.

These fundamentals are just the start of of the medical industry’s responsibilities. Independent researchers must evaluate the sources revealed in the first step and determine whether they are the best available choices. Programmers must check the code in the second step for accuracy. These grueling activities should be funded by the clinical institutions that ultimately use the CDS, so that they are on a firm financial basis and free from bias.

The requirement for transparent studies raises the question of open access to medical journals, which is still rare. But that is a complex issue in the fields of research and publishing that I can’t cover here.

Finally, an independent service has to collect reports of CDS failures and make them public, like the FDA Adverse Event Reporting System (FAERS) for drugs, and the FDA’s Manufacturer and User Facility Device Experience (MAUDE) for medical devices.

These requirements are reasonably light-weight, although instituting them will seem like a major upheaval to industries accustomed to working in the dark. What the requirements can do, though, is put CDS on the scientific basis it never has had, and push forward the industry more than any “big data” can do.

Ebola Lapse in Dallas Offers Few Lessons, Except About Our Over-reliance on Technology

Posted on October 8, 2014 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site (http://oreilly.com/) and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Of all the EHR problems encountered daily across the country, the only one to hit the major news outlets was a non-story about a missed Ebola diagnosis in Dallas, Texas. Before being retracted, the hospital’s claim of an Epic failure launched a slew of commentary in the health IT field. These swirled through my head last night as I tried to find a lesson in the incident.

The facts seem to be as follows. A 42-year-old man named Thomas Eric Duncan arrived from Liberia and checked in to the emergency room at Texas Health Presbyterian Hospital Dallas complaining of symptoms consistent with an Ebola diagnosis. He told the admitting nurse he had come from Liberia, and the nurse entered the data into the Epic EHR.

The purpose of recording the patient’s travel history, however, seemed to be simply to determine the need for immunizations, so the EHR kept it within a nurse’s section of the data (which the hospital called a “workflow”) and did not display it to the doctor. The doctor sent Duncan home, where he came into contact with about 100 people who were potentially infected. His symptoms worsened and he returned to the hospital two days later, where he was finally diagnosed correctly and admitted.

Late night musing #1: If Texas Health Presbyterian Hospital Dallas can’t diagnose a case of Ebola, why do they think they can treat one? The hospital has won numerous awards, including one for patient safety–I guess you’re safe once you’re admitted.

Meanwhile, the city of Dallas waited several extra days to clean up infected sheets and other belongings from the Duncan home. In Africa, such detritis are recognized as a major source of new Ebola infections.

Late night musing #2: Does this reflect the competence of public health officials in this country? Maybe we should turn the job over to the Secret Service.

It’s really a shame that the national press jumped on the hospital’s announcement that the EHR was the source of the problem. Commenters criticized the hospital right away, asking why the nurse didn’t simply tell the doctor, and why the doctor didn’t ask on his own.

Finally, the hospital backed off from blaming Epic, thus making the hospital look even stupider and more guilty than it already appeared. Nevertheless, EHRs at some hospitals may be designed to flag warning signals.

Clearly, there are many layers to this health care failure. I don’t blame the nurse, or even the doctor. ERs are always busy, and the nurse might never have known who would see the patient or even be in the ER when the doctor finally saw him.

But I do find a small lesson in the brief appearance of the EHR as a pivotal character in the story. The nurse thought he or she was doing their job just by entering the data into the EHR, and the doctor thought he was doing his job by reading it. The EHR had loomed as a magical solution to health care workflow–in the minds of hospital administrators, if not the ER staff.

Maybe if the nurse knew that the travel history was for the purpose of immunizations, he or she would not have relied on the EHR to use that information for diagnosis. Besides showing the need for training, some of my colleagues suggest that this problem calls for FDA regulation of EHR interfaces. They also suggest that systems use good user interface design to highlight important information (which would require a definition of what’s “important”) or at least allow searches for critical elements of the record.

Late night musing #3: Behind this also lies the mindlessness of much data collected by EHRs. I’m sure the nurse knew whether the unfortunate Mr. Duncan was a smoker and whether he suffered from depression, because regulations require these things to be recorded. Travel history became just another one of these automatic requirements to be tossed into the EHR and forgotten.

My story also concerns the musings of other health IT commentators, who suggested that EHRs be better integrated into “workflows”–as if every clinician follows a mechanical path of treatment and the EHR can figure out what it is.

Another thoughtful posting calls for integrating infectious diseaess into clinical decision support. But as my colleague Sandra Raup (R.D., J.D., M.P.H.) points out, CDS depends on a long history of clinical data collection. One can’t instantly add a new disease.

It might have been useful for some international health organization to realize, when the Ebola outbreak began to spread, that it would eventually break out of central Africa, and then to provide an app to hospitals around the world for checking symptoms and travel history. There is certainly a creative role for health IT to play.

I think the messiness of the Texas Health Presbyterian Hospital Dallas story shows why EHR failures, numerous as they are, don’t get reported in the press. There are just too many complicating factors. The EHR is partly configured by the clinic’s staff, who thereby become responsible for some of its decisions. The EHR failure usually comes when the staff is under stress, when they have communication problems, when the patient’s condition is rare. Ascribing blame becomes a tangled mess; one must start designing systems with multiple, redundant points to catch failures that can fall through the cracks.

So one level, this is just another sad story of humanity’s tendency to trust too much in its technology, a story that ranges from the flight of Icarus to the sail of the Titanic and the failure of the Fukushima Daiichi nuclear power plant. On other, it’s a familiar story of a systemic problem leading to what’s sometimes called a “normal failure.” Not much new to learn, but lots of work to do. Clinicians have to evaluate EHRs and know how the data is used, a more open system in all directions.

101 Tips to Make Your EMR and EHR More Useful – EHR Tips 21-25

Posted on November 8, 2011 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

Time for the next entry covering Shawn Riley’s list of 101 Tips to Make your EMR and EHR More Useful. I met someone at a conference who commented that they liked this series of posts. I hope you’re all enjoying the series as well.

25. Care coordination is much easier in an EMR and should be evaluated to be used
The idea of care coordination has never been more important in the history of healthcare. It’s the future of healthcare (at least in the US). Whether they end up being called ACOs or some other term, the switch to needing to coordinate care in order to improve the health of a population is happening as we speak. Luckily, EMR software is a great way to facilitate this care coordination.

24. Take advantage of E-Health tools
I actually think that this is a big call to EMR vendors to integrate their EMR software with the various e-health tools out there today. EHR vendors that think they can create every e-Health tool a doctor could want are going to be left behind by those systems which support the most popular consumer health tools on the market. However, that’s not to say that doctors can’t do their part. Start getting your patient using the e-health tools that will benefit them as a patient and then start requesting that your EMR vendor support the tools you’re using.

23. Make certain all caregivers know that logs are kept for any system overrides
Don’t hide the fact that everything is logged. Let everyone know that whatever is done on the system is logged. While some may see this as big brother watching them, most will realize that the logs are a protection for them. They log exactly what was done and said and who did it.

I remember one time there was some problem in our EMR system. I can’t remember the specific issue. Well, it was brought up in our staff meeting and the director said, whoever made this mistake is going to be providing breakfast for the whole staff. I went into the logs to see who’d accessed the patient to do the offending task. Little did the director (who was also a practicing provider a few times a week) know that she was the offending party. Everyone in the clinic enjoyed a nice breakfast that week.

22. Give caregivers the ability to override the system when necessary
Mistakes happen in documentation in an EMR. We’re all imperfect human beings (except for my wife) who make mistakes. So, you need an option and likely a process for how and who can make corrections to what was done in the EMR. Just be sure that everything that’s “overwritten” is logged and the reason for the change is well documented.

21. Develop a root cause analysis process for the EMR
I’m not that familiar with root cause analysis processes, so I’ll just share what Shawn says about it:

You very likely already have a root cause analysis model for your practice. You will need to adopt that model to the EMR. If you don’t, you will create a likelihood for the same errors to continually repeat. The EMR process is different than a usual root cause analysis. You will need to take into account interfaces, security roles, single sign on, and several other things beyond the “simple” human process.

If you want to see my analysis of the other 101 EMR and EHR tips, I’ll be updating this page with my 101 EMR and EHR tips analysis. So, click on that link to see the other EMR tips.

101 Tips to Make Your EMR and EHR More Useful – EHR Tips 26-30

Posted on October 28, 2011 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of InfluentialNetworks.com and Physia.com. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

Time for the next entry covering Shawn Riley’s list of 101 Tips to Make your EMR and EHR More Useful. I met someone at a conference who commented that they liked this series of posts. I hope you’re all enjoying the series as well.

30. Remember that the EMR is only part of the safety problem
Remember that the EMR is just a tool. How you use that tool still matters. How you manage that tool matters. How you implement that tool matters. Safety is a result of great processes and that doesn’t change when you implement an EMR. In fact, I’d say it’s even more important. The same applies to bad clinical workflows. EMR won’t solve those bad workflows either. You can try to do a redesign of the workflows with the EMR implementation, but that often doesn’t go over well.

29. Errors should be easily reportable
To be honest, I’m not sure exactly which errors Shawn is talking about. I think I’ll take a different spin on it than what he intended and talk about the errors or issues that someone has using an EMR. This is particularly important when you first implement an EMR. You should want to know the errors that are occurring regularly so you can fix them. Make it easy for them to report them and provide proper encouragement and/or rewards for reporting errors they have with the system. Ignorance is not bliss…it always catches up to you eventually.

28. Use data to show both individual and system safety metrics
The key component that Shawn is describing here is the ability to report on various cross sections of data (individual vs system). If you can’t chop up your data to really know what’s going on in your system, then you’re not going to be able to really pinpoint the issues that users are having. Maybe it’s only one person who’s bringing down the average for the entire hospital. You don’t want to make sweeping changes to the system that annoy the majority of users when all you really needed to do was address the issues of an individual or small group of individuals.

27. Record management in the EMR is just as important as in paper
You thought HIM was done when you got the EMR. Wrong! Their role is still very important. Granted, it changes pretty dramatically, but in the clinics I’ve worked in the records management people were able to do a much more effective job improving the patient record in the EMR. Many of the things they did they never had time to do cause they were too busy pulling and filing paper charts.

26. Evaluate decision support tools for a fit to your needs
I believe that the clinical decision support tools are going to be the thing that changes the most over the next 5-10 years. You should definitely see how the clinical decision support tools they have available fit into your environment, but also spend as much time seeing what they’ve implemented and what their road map and method of implementing new clinical decision support tools is so you know where they’re going to be with their tools and product in five years.

If you want to see my analysis of the other 101 EMR and EHR tips, I’ll be updating this page with my 101 EMR and EHR tips analysis. So, click on that link to see the other EMR tips.