Regular reader, Carl Bergman from EHR Selector, attended the recent NIST EMR Usability Conference and sent over the following guest post on what was said. Thanks Carl for sharing your experience with us.
A year ago last June I attended NIST’s (National Institute of Standards and Technology) conference on EMR/EHRs usability. [See Carl's post on the NIST EHR Usability Conference from 2011.] It was a mixed bag. There were several excellent presentations on the fundamentals of usability, how to analyze an EMR and where the field was headed. Unfortunately, NIST’s staff took a narrow view confining their work to EMR error conditions and assiduously avoiding interface, workflow and clinical setting issues. It was odd that an agency that prided itself on redesigning nuclear control rooms after Three Mile Island or the design of airplane cockpits would ignore EMR user interfaces.
New Approach: New Protocol
At this year’s conference at NIST headquarters in Gaithersburg, MD, the past was not prolog. Last week’s conference focus was on a comprehensive EMR usability protocol, NISTIR 7804, that NIST produced last February. (For a good synopsis, see Katherine Rourke’s Design Errors That Cause Patient Harm per NIST.) NIST’s staff pulled together a notable group of speakers on patient safety in general and implementing the protocol in particular. (NIST is posting the presentations here.)
The protocol, designed to review an EMR, is not a trivial undertaking since it has about 180 line item questions. It asks, for example, if the EMR:
- Keeps patient identities distinct from each other? That is, does the system prevent one record from writing over another or erroneously sharing data elements?
- Lays out pages in a consistent manner using color, icons and links identically?
- Uses measurements consistently? That is, if weight is entered in pounds and ounces in one place, do they show that way in other places?
- Displays fields fully rather than being truncated?
- Sorts logically based on the subject?
- Show dosages, etc., with all needed information on the page?
- Displays multipage entries or lookups with proper navigation choices?
- Has error messages that state what is wrong and how to cure the problem?
- Accommodates different levels of user knowledge? That is, does it have extended help for novice users, refresher information for occasional users and short cuts for experienced users?
Developers Present in Force
If NIST’s major intent was to get developer attention, they succeeded. Of the hundred or so attendees, about 20 percent were from major systems. 3m, Allscripts, Athenahealth, Centricity, McKesson, NextGen, etc., each had one or more representatives present. Others present included Kaiser, HIMSS, Medstar, First Choice, ACP, Columbia, etc.
Unfortunately, there is no way to know developer reaction to the protocol. The conference had no comment session. I don’t know if this was by design or if time just ran out. NIST staff did indicate that next year the conference would be two days rather than one. However, a year is a long time to wait for reactions. This is especially pertinent since NIST is not a regulatory agency. Its protocols are strictly voluntary and depend on vendor acceptance.
What NIST did do is offer several presentations that emphasized how fragile patient safety can be in an HIT world. One breakout session used an actual, unnamed product’s screen that had dozens of misleading or ambiguous fields. For example, the screen’s fields cut off drug names, used red to indicate several different findings and used a pop up that blocked a view of a pertinent entry.
In another more broadly based patient safety presentation, University of Pennsylvania’s voluble Ross Koppel drove home how common elements in EMRs such as blood pressure – he’s found 40 different ways to show it so far – are subject to many formats for capture and display. Moreover, if you think EMRs have problems, Koppel shows how bar codes and work arounds can play havoc with workflow and patient safety.
Wanted: One Good Policy Compass
For those of us possessed of an EMR design demon, it was both a good chance to wonder out loud just what it all meant and where, if anywhere, things were headed. Sadly, the most common answer was who knows? There were some common points:
- It’s better to have NIST’s protocol than not.
- You can forget the FDA playing a bigger role. It’s under funded and over worked.
- HIMSS will wait for the industry and the industry has shown no hurry.
- EMR adverse incident reporting would be great, but who would do it and how open would it be?
In short, if you’re shopping for an EMR, regardless of your size, don’t count on anyone handing you a usability report on an EMR anytime soon. Moreover, don’t try to run NIST’s protocol on your own unless you have full access to the proposed EMR, lots of time on your hands and a good grasp of the protocols details.
There are some things you can do. You can ask potential vendors questions such as these:
- Have they run the NIST protocol and what did they do as a result?
- If not NIST, do they have a written usability protocol and, if so, can you see it? How have they implemented it?
- Have they tested their EMR’s usability with outside, independent users? What were the results?
- Have they used any interface designers?
- What usability changes do they plan?
There is no guarantee that you’ll get a great product, but it could mean that you get one that doesn’t bite your patients or you.