Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and EHR for FREE!

Reply to Dr. Jacob Reider on NIST Dissects Workflow: Is Anyone Biting?

One comment on my latest post, NIST Dissects Workflow: Is Anyone Biting?, deserves a more than casual reply.

Here’s the comment from Jacob Reider (Note: Dr. Reider is ONC’s Acting Principle Deputy National Coordinator and Chief Medical Officer. He has made major contributions to the HIT field and is one of its significant advocates.)

Carl, ONC’s UCD requirement references ISO 9241–11, ISO 13407, ISO 16982, NISTIR 7741, ISO/IEC 62366 and ISO 9241–210 as appropriate UCD processes.

We also require summative testing as defined in NISTIR 7742.

Might “Refuses to incorporate NIST recommendations” be a bit of an overstatement?

We solicited public comment in our proposed rule for 2015 certification and would welcome specific suggestions for how we can/should improve user experience of health IT products for efficiency and safety.

Dr. Reider, thank you for your comment – it certainly falls into the category of you never know who’s reading.

Let’stake a look at your last comment first, “Might ‘Refuses to incorporate NIST recommendations’ be a bit of an overstatement?”

Obviously, I don’t think so, but I am not alone.

I based my comment on ONC’s statement in its rule making that refers to NIST’s usability protocols. It says:

While valid and reliable usability measurements exist, including those specified in NISTIR 7804 “Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records,” (21) we are concerned that it would be inappropriate at this juncture for ONC to seek to measure EHR technology in this way.

Sounds like a rejection to me, however, don’t take my word. Here’s the AMA’s response to this decision. First, they demur and quote ONC:

We disagree with ONC’s assertion in the Version 2014 final rule that, “[w]hile valid and reliable usability measurements exist, including those specified in NISTIR 7804 ‘‘Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records,’’ we expressed that it would be inappropriate for ONC to seek to measure EHR technology in this way.”

It then says:

To the contrary, we believe that it is incumbent upon ONC to include more robust usability criteria in the certification process.  The incentive program has certainly spurred aggressive EHR uptake but has done so through an artificial and non-traditional marketplace.  As a consumer, the physician’s choice of products is limited not only by those EHRs that are certified but also by the constraint that all of these products are driven by federal criteria.  The AMA made several detailed recommendations for improving Version 2014 certification in our Stage 2 comment letter, which were not adopted, but still hold true, and we recommend ONC consider them for the next version.  Testimony of AMA’s Health IT Policy Committee’s Workgroups on Certification/Adoption and Implementation, July 23, 2013, pp. 5-6

I recognize that ONC says that it may consider the protocols in the future. Nevertheless, I think the plain English term rejected fits.

In the first part of his statement, Dr. Reider cites several ISO standards. With the exception of the Summative Testing, all of these have been referred to, but none have been adopted. Reference to a standard is not sufficient for its inclusion under the operation of the federal Administrative Procedure Act, which governs all federal agency rulemaking. In other words, these standards are important, but ONC simply calls them out for attention, nothing more.

I think two factors are at work in ONC’s reluctance to include the NIST usability protocols. The first is that the vendors are adamantly opposed to having them mandated. However, I believe there is a way around that objection.

As I have argued before, ONC could tell vendors that their products will be subject to a TURF based review of their product for compliance and that the results would be made public. That would give users a way to judge a product for suitability to their purpose on a uniform basis. Thus, users looking at the results could determine for themselves whether or not one or more non compliance was important to them, but at least they would have a common way to look at candidate EHRs, something they cannot do now , nor under ONC’s proposed approach.

The other factor is more complex and goes to the nature of ONC’s mission. ONC is both the advocate and the standards maker for HIT. In that, it is similar to the FAA, which is vested with both promoting and regulating US aviation.

It’s well established that the FAA’s dual role is a major problem. It’s hard to be a cheerleader for an industry and make it toe the line.

With the FAA, its dual mandate is exacerbated when the highly respected NTSB investigates an incident and makes recommendations. The FAA, acting as industry friend, often defers NTSB’s authoritative and reasonable recommendations to the public’s determent.

I believe that something similar is going on with ONC. NIST’s relationship to ONC is roughly analogous to that of the NTSB’s to the FAA.

NIST is not an investigative agency, but it is the federal government’s standards and operations authority. It isn’t infallible. However, ONC dismissing NIST’s usability protocols, in one word, inappropriate. It did this without explanation or analysis (at least none that they’ve shared). In my view, that’s really inappropriate.

ONC has a problem. It’s operating the way it was intended, but that’s not what patients and practioners need. To continue the aviation analogy, ONC needs to straighten up and fly right.

March 31, 2014 I Written By

When Carl Bergman isn't rooting for the Washington Nationals or searching for a Steeler bar, he’s Managing Partner of EHRSelector.com, a free service for matching users and EHRs. For the last dozen years, he’s concentrated on EHR consulting and writing. He spent the 80s and 90s as an itinerant project manger doing his small part for the dot com bubble. Prior to that, Bergman served a ten year stretch in the District of Columbia government as a policy and fiscal analyst.

NIST Dissects Workflow: Is Anyone Biting?

Psst. Hey, Buddy, wanna see an EHR, visit’s workflow? Here it is, thanks to the National Institutes of Standards and Technology’s (NIST) new report, NISTIR 7988, Integrating Electronic Health Records into Clinical Workflow, etc.

Returning Patient Ambulatory Workflow NIST

What It Represents

NIST wants to make EHRs usable and useful. It first took aim at patient safety EHR functions that endangered, confused users or were error prone. To counter these, it developed and recommended EHR usability protocols.

Now, in an extensive report, it’s tackled EHR workflow to determine where problems occur. The result is a comprehensive work with significant findings and recommendations. The question is: Is anyone listening?

NIST’s Analytical Approach

NIST decided to create a typical workflow by interviewing knowledgeable physicians, who it calls Subject Matter Experts, SMEs. The physicians had different specialties and used different EHRs, though who they were, NIST doesn’t say.

From their discussions, NIST’s analysts created the above chart, NIST’s Figure 2. NIST’s authors recognize that actual workflows will vary based on setting, sequences, staffing, etc., but that it provides a useful way to look at these issues.

What They Did With It

Working with their physicians, NIST’s analysts broke down the workflow into three sections: before, during and after the visit. Then, they broke down, or decomposed, each of those sections, like opening nested Russian dolls. For example, they segmented the physician’s encounter, below, and once again, broke each down into its functions.

Returning Patient, Physician Encounter - NIST

What They Found

It was at this stage the analysts found significant variations among the EHRs used by their physicians,

[T]here appeared to be high variation in whether and how the EHR was used during this period, how extensive each of the activities typically were for each SME, different based upon the type of patient, how complex the patient was, context of how busy the day was, and other factors. NSTIR 7988, p 18.

Despite these differences, the physicians identified two issues that crossed their EHRs:

  • Working Diagnoses. The physicians wanted systems that let them create a working diagnosis and modify it as they worked until they made a final diagnosis. Similarly, they wanted to be able to back up and make changes as needed, something current systems make hard.
  • Multiple Diagnoses. Diagnoses usually involve multiple causes, not single factors. They wanted their EHRs to support this.

These types of issues aren’t new to those familiar with EHR problems. What’s new is NIST, as an independent, scientific organization, defining, cataloguing and explaining them and their consequences.

What They Recommended

From this work, NIST’s analysts developed extensive and persuasive recommendations, in three categories:

  • EHR Functions
  • System Settings, and
  • System Supports

EHR Functions

NIST’s recommends reducing practitioner workload, while increasing their options and supports. For example, they suggest:

  • Workload Projections. Give practitioners a way to see their patient workloads in advance, so they can plan their work more effectively
  • Notes to Self. Let users create reminder notes about upcoming visit issues or to highlight significant ,patient information. These would be analogous to their hand written notes they used to put on paper charts.
  • Single Page Summaries. Create single page labs summaries rather than making users plow through long reports for new information.
  • Single Page Discharge Summaries. Eliminate excessive boiler plate with more intelligent and useful discharge sheets.
  • Highlight Time Critical Information. Segregate time critical information. Often they get mixed in with other notices where they may be overlooked or hard to find.
  • Allow Time Pressure Overrides. When time is critical, EHRs should allow skipping certain functions.
  • System Settings

NIST recommendations echo the familiar litany of issues that characterize poor implementations:

  • Allow Patient Eye Contact. Exam room designs should put the doctor and patient in a comfortable, direct relationship with the computer as a support.
  • Login Simplification. Allow continuous logins or otherwise cut down on constant login and outs.

System Supports

The physicians recognized they often caused workflow bottlenecks. NIST recommended off loading work to medical assistants, nurse practitioners, physician assistants, etc.. For example, physician assistants could draft predicted orders for routine situations for the physician to review and approve.

Progress Note Frustrations

In the thorny area of clinical documentation, the report details physician frustration with their EHRs. All experienced excessive or missing options, click option hell, excessive output, puzzling terms, etc. These were compounded by time consuming system steps that did not aid in diagnosis or solving patient problems. The report discusses their attempts at improving documentation:

Several of the SMEs had attempted and then abandoned strategies to increase the efficiency of documentation. One SME reported that copying and pasting and “smart text” where typing commands generate auto-text had a “vigilance problem.” The issue was that it would be too easy to put the wrong or outdated information in or in the wrong place and not detect it, and then someone later, including himself, could act on it not realizing that it was incorrect.

One physician described an attempt to use automated speech recognition for dictation for a patient with scleritis, which is inflammation of the white of the eye. He stopped using the software when what was documented in the note was “squirrel actress.”

Another SME described that colleagues relied upon medical assistants to draft the note and then completed it, but they did not like that approach because it was too tempting to rely upon what was typed without reviewing it, and he felt the medical knowledge level was not high enough for this task.

One SME described a reluctance to use any scribe, including a medical student, because the risk would be too high of misunderstanding and thus not correctly documenting the historical information, diagnosis, and treatment plan. This was particularly problematic if the physician had information from prior visits, which contributed to these elements, which were not discussed in detail during the visit. NSTIR 7988, p. 28

Coding their diagnoses into progress notes also came in for criticism:

All SMEs described frustration with requirements to enter information into progress notes, …, which were applied to the notes in order to have sufficient justification to receive reimbursement for services. Although all of the SMEs acknowledged the central importance of receiving reimbursement in order to function as a business, this information was often not important for clinical needs. NSTIR 7988, p. 28

Role Based Progress Note

Unlike other areas of the report, the doctors could not agree on what to do, nor does NIST offer any specific cures for documentation problems. Instead, NIST recommends using a new, role based, progress note:

[T]he progress note for a primary care physician would have a different view from a specialist such as a urologist physician, who might not need to see all of the information displayed to the primary care physician. Similarly, the view of the note for primary care providers could differ from the view of a billing and coding specialist. … NSTIR 7988, p. 28

Will ONC Respond?

In this and its prior reports, NIST covers a lot of EHR issues making sensible recommendations that not only improve functionality, but more importantly improve patient safety. However, NIST’s recommendations are just that. It’s not a regulatory agency, nor is supposed to be one. Instead, its role is to work with industry and experts to develop usable, practical approaches to tough technical, often safety related, problems. To its credit, it’s done this in a vast number of fields from airplane cockpits, nuclear reactors, and atomic clocks to bullet proof vests.

However, its EHR actions have not gained any noticeable traction. If any EHR vendor has implemented NIST’s usability protocols, they haven’t said so. They are not alone.

Notably ONC, one of NIST’s major EHR partners, refuses to incorporate any of NIST’s usability recommendations. Instead, ONC requires vendors to implement User Centered Design, but does not define it, letting each vendor do that for themselves.

NIST has many answers to common EHR workflow and usability problems. The question is, who will bring them to bear?

March 26, 2014 I Written By

When Carl Bergman isn't rooting for the Washington Nationals or searching for a Steeler bar, he’s Managing Partner of EHRSelector.com, a free service for matching users and EHRs. For the last dozen years, he’s concentrated on EHR consulting and writing. He spent the 80s and 90s as an itinerant project manger doing his small part for the dot com bubble. Prior to that, Bergman served a ten year stretch in the District of Columbia government as a policy and fiscal analyst.

An Eclectic Gathering of EHR Usability and Project Resources

Here are a few resources that I use to solve a variety of design, project management and other problems. Some, such as NIST’s protocols, are directly EHR related; others aren’t, but easily apply to EHR problems.

So here, as was once said, are a few of my favorite people and things:

  • Dan Bricklin. Hopper and Jobs are gone. Woz is a sage. Gates, Kapor and Norton long ago stopped being systems innovators in favor of being philanthropists. Bricklin, however, the electronic spreadsheet’s inventor, soldiers on. His personal site has much to offer, especially his video on interface development for different types of devices and users.
  • Donald Norman. If you only read one of Norman’s many works on usability, make it The Design of Everyday Things. When you do, you’ll find one of the most cogent, funny and thoughtful studies of user centered design. From his account of slide projectors from hell, to post office doors that trap the unwary, to the best ways to arrange light switches, Norman has good advice for all of us. I first read this twenty years ago, but the advice still resonates. He’s recently revised it and added a free on-line course. After Norman, you won’t look at doors, appliances, much less screens the same way again.
  • Jakob Nielsen. There are people who think if you know Nielsen’s usability approach, you need little else. Then, there are those who think if you know Nielsen’s approach, all hope is lost. No one has a monopoly on good interface design, but Nielsen’s site is a place to stop for tons of notable examples.
  • NIST Protocols. NIST works with the private sector to solve major, operational problems. After Three Mile Island close call NIST redesigned all US nuclear power plants’ control rooms. Recently, they’ve developed EHR usability standards. These are the best, most comprehensive treatment of what not to do. You’ll find them in an appendix in their publication, NISTIR 7804.
  • ONC Repository. Most those in the EHR field know ONC, for better or worse due to its Meaningful Use standards. There’s a lot more. Buried on ONC’s site is its Implementation Resources. The repository has dozens of videos, guides, white papers, toolkits and templates all centered on improving EHR selection, implementation and use.
  • Ross Koppel. Koppel is a grouch. He grouches about the dozens different ways EHRs record simple things, such as, blood pressure. Writing often in JAMA, he notes how health IT systems spawn workarounds, confusion and give users choices that are false, misleading or illogical. In short, he’s produced a treasure trove of frightening observations, embarrassing questions and pointed observations, but his bête noire findings also include correctives. All of this is written in a careful, thoughtful style that makes the subject compelling and chilling.
  • Tom Demarco. Demarco of the Atlantic System Guild has produced a wealth of insightful books, lectures and articles on project management. In the 60s, DeMarco asked himself, if a civil engineer can build a bridge from requirements to operation, why can’t we do the same thing with software. His first take was Structured Analysis and System Specification. It’s still in print and full of practical advice and approaches for project managers. Other works include Peopleware: Productive Projects and Teams, where he takes the odd position that you should treat your staff like people and help them be productive. I especially love his The Deadline: A Novel About Project Management in which the main character is kidnapped and forced to manage a project under threat of death. It’s a comedy.

Interested in EHR usability? Join my LinkedIn group: EHRUsability.

March 10, 2014 I Written By

When Carl Bergman isn't rooting for the Washington Nationals or searching for a Steeler bar, he’s Managing Partner of EHRSelector.com, a free service for matching users and EHRs. For the last dozen years, he’s concentrated on EHR consulting and writing. He spent the 80s and 90s as an itinerant project manger doing his small part for the dot com bubble. Prior to that, Bergman served a ten year stretch in the District of Columbia government as a policy and fiscal analyst.

EHR Usability: Is There a Right Path?

The following is a guest post by Carl Bergman from EHR Selector.

Earlier this fall, the AMA sponsored a Rand Corporation study on physician’s professional satisfaction. Based on interviews with physicians in 30 practices, the study covers a variety of topics from workplace setting to quality of care, EHRs and health reform, etc. At the time, the report generated discussion about dissatisfaction in general with EHRs and MU in particular.

Usability, Part of MU?
Overlooked in the discussion was a new and important recommendation on usability. Here’s what is says:

Physicians look forward to future EHRs that will solve current problems of data entry, difficult user interfaces, and information overload. Specific steps to hasten these technological advances are beyond the scope of this report. However, as a general principle, our findings suggest including improved EHR usability as a precondition for federal EHR certification. (Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy, p.142) Emphasis added.

It would be overkill to say that this represents adopted AMA policy, however, it’s not overkill to say that the recommendation is part of a project that the AMA initiated and supports. As such, it is most significant that it recognizes the need to bring some coherence to EHR usability and that the MU system is the logical place to put it.

Changing the Vendor – User Relationship
One commentator who did notice the recommendation was EHR Intelligence’s Robert Green. In his review, Green took a different tack. While agreeing that usability needs improvement, he saw a different way to get change:

Usability remains an enigma in many clinic-EHR vendor relationships because it hasn’t been nearly as important in the recent years’ dialogue as “meaningful use.” But among the competing priorities, usability among physicians and their EHR vendor is a real opportunity to develop shared expectations for a new user experience.

As a patient, I would rather not see the delegation of the “usability” dialogue of EHR to those in the roles of meaningful use certification. Instead, physicians who have spent many years of their lives learning how to “take care of patients” could seize the moment to define their own expectations with their EHR vendor of choice within and beyond their practice. (How connected is EHR user satisfaction to vendor choice?) Emphasis added.

I think these two different paths put the question squarely. They agree that usability needs increased action. Users have gotten their message across with alacrity: all systems fail users in some aspect. Some fail catastrophically. Though some vendors take usability to heart, the industry’s response has been uneven and sporadic.

Where these two approaches differ is tactics. Rand looks at usability, and sees an analog to MU functions. It opts for adding usability to MU’s tests. Green sees it as part of the dialogue between user and vendor.

As a project manager and analyst, my heart is with Green. Indeed, helping users find a system that’s a best fit is why we started the Selector.

Marketplace Practicalities
Nevertheless, relying on a physician – vendor dialogue is, at best, limited and at worst unworkable. It won’t work for several reasons:

  • Nature of the Market. There’s not just one EHR market place where vendors contend for user dollars, there are several. The basic divide is between ambulatory and in patient types. In each of these there are many subdivisions depending on practice size and specialty. Though a vendor may place the same product name on its offerings in these areas, their structure, features and target groups differ greatly. What this means is that practices find themselves in small sellers’ markets and that they have little leverage for requesting mods.
  • Resources. Neither vendors nor practices have the resources needed to tailor each installation’s interface and workflow. Asking a vendor, under the best of circumstances, to change their product to suit a particular practice’s interface approach not only would be expensive, but also would create a support nightmare.
  • Cloud Computing. For vendors, putting their product in the cloud has the major advantage of supporting only one, live application. Supporting a variety of versions is something vendors want to avoid. Similarly, users don’t want to hear that a feature is available, but not to them.
  • More Chaos. Having each practice define usability could lead to no agreement on any basics leaving users even worse off. It’s bad enough now. For example as Ross Koppel points out, EHRs record blood pressure in dozens of different ways. Letting a thousand EHRs blossom, as it were, would make matters worse.

ONC as Facilitator Not Developer
If the vendor – buyer relationship won’t work, here’s a way the MU process could work. ONC would use an existing usability protocol and report on compliance.

Reluctance to put ONC in charge of usability standards is understandable. It’s no secret that the MU standards aren’t a hands down hit. All three MU stages have spawned much criticism. The criticism, however, is not that there are standards so much as individual ONC’s standards are too arcane, vague or difficult to meet. ONC doesn’t need to develop what already exists. The National Institutes of Standards and Technology usability protocols were openly developed, drawing from many sources. They are respected and are not seen as captured by any one faction. (See NISTIR 7804. And see EMRandEHR.com, June 14, 2012.)

As I’ve written elsewhere, NIST’s protocols aren’t perfect, but they give vendors and users a solid standard for measuring EHR usability. Using them, ONC could require that each vendor run a series of tests and compare the results to the NIST protocols. The tool to do this, TURF, already exists.

Rather than rate each product’s on a pass – fail basis, ONC would publish each product’s test results. Buyers could rate product against their needs. Vendors whose products tested poorly would have a strong incentive to change.

EHRs make sense in theory. They also need to work in practice, but don’t. The AMA –Rand study is a call for ONC to step up and takes a usability leadership role. Practice needs to match promise.

December 9, 2013 I Written By

When Carl Bergman isn't rooting for the Washington Nationals or searching for a Steeler bar, he’s Managing Partner of EHRSelector.com, a free service for matching users and EHRs. For the last dozen years, he’s concentrated on EHR consulting and writing. He spent the 80s and 90s as an itinerant project manger doing his small part for the dot com bubble. Prior to that, Bergman served a ten year stretch in the District of Columbia government as a policy and fiscal analyst.

Where Are Usability Standards For EMRs?

The other day, I was talking with a physician about ambulatory EMRs.  “None of them are any good,” said the doctor, who’s studied EMRs for several years but never invested in one. “I can’t find a single one that I can use.”

Are any of you surprised to hear him say that? I’m certainly not.  Perhaps he’s exaggerating a bit when he says that absolutely none are usable at all, but it’s hard to argue that doctors cope with a counter intuitive mess far too often.  And of course, enterprise EMRs get if anything lower usability ratings from practicing doctors.

All of which brings me around to the notion of EMR usability standards, or rather, the lack of such same. While those in the industry talk often about usability, there’s no real consensus standard for measuring how usable a particular EMR is, despite noble efforts by NIST and impassioned advocacy by usability gurus in the field.

Certainly, private research organizations take usability into account when they survey clinicians on which EMRs they prefer. So clunky EMRs with lousy UIs do pay some kind of price when they’re rated by the clinical user. But that’s a far cry from having a standard in place by which medical practices and hospitals can objectively consider how usable their preferred EMR is going to be.

So, why don’t we have usability standards already in place?  The market still hasn’t punished vendors whose EMRs are a pain to use, so vendors keep on turning our products built around IT rather than clinical needs. The doctor I spoke with may have opted out of the EMR market, but most providers aren’t going to do that, Meaningful Use incentives being just one reason why. (It’s a “handwriting is on the wall” thing.)

It’s a shame CMS isn’t pushing vendors to produce Meaningfully Use-ABLE EMRs. That might do the trick.

December 7, 2012 I Written By

Anne Zieger is veteran healthcare consultant and analyst with 20 years of industry experience. Zieger formerly served as editor-in-chief of FierceHealthcare.com and her commentaries have appeared in dozens of international business publications, including Forbes, Business Week and Information Week. She has also contributed content to hundreds of healthcare and health IT organizations, including several Fortune 500 companies. Contact her at @annezieger on Twitter.

NIST’s EHR Usability Conference Breaks Both Old Ground and New Focuses on EMR Patient Safety Protocol

Regular reader, Carl Bergman from EHR Selector, attended the recent NIST EMR Usability Conference and sent over the following guest post on what was said. Thanks Carl for sharing your experience with us.

A year ago last June I attended NIST’s (National Institute of Standards and Technology) conference on EMR/EHRs usability. [See Carl's post on the NIST EHR Usability Conference from 2011.] It was a mixed bag. There were several excellent presentations on the fundamentals of usability, how to analyze an EMR and where the field was headed. Unfortunately, NIST’s staff took a narrow view confining their work to EMR error conditions and assiduously avoiding interface, workflow and clinical setting issues. It was odd that an agency that prided itself on redesigning nuclear control rooms after Three Mile Island or the design of airplane cockpits would ignore EMR user interfaces.

New Approach: New Protocol

At this year’s conference at NIST headquarters in Gaithersburg, MD, the past was not prolog. Last week’s conference focus was on a comprehensive EMR usability protocol, NISTIR 7804, that NIST produced last February. (For a good synopsis, see Katherine Rourke’s Design Errors That Cause Patient Harm per NIST.) NIST’s staff pulled together a notable group of speakers on patient safety in general and implementing the protocol in particular. (NIST is posting the presentations here.)

The protocol, designed to review an EMR, is not a trivial undertaking since it has about 180 line item questions. It asks, for example, if the EMR:

  • Keeps patient identities distinct from each other? That is, does the system prevent one record from writing over another or erroneously sharing data elements?
  • Lays out pages in a consistent manner using color, icons and links identically?
  • Uses measurements consistently? That is, if weight is entered in pounds and ounces in one place, do they show that way in other places?
  • Displays fields fully rather than being truncated?
  • Sorts logically based on the subject?
  • Show dosages, etc., with all needed information on the page?
  • Displays multipage entries or lookups with proper navigation choices?
  • Has error messages that state what is wrong and how to cure the problem?
  • Accommodates different levels of user knowledge? That is, does it have extended help for novice users, refresher information for occasional users and short cuts for experienced users?

Developers Present in Force

If NIST’s major intent was to get developer attention, they succeeded. Of the hundred or so attendees, about 20 percent were from major systems. 3m, Allscripts, Athenahealth, Centricity, McKesson, NextGen, etc., each had one or more representatives present. Others present included Kaiser, HIMSS, Medstar, First Choice, ACP, Columbia, etc.

Unfortunately, there is no way to know developer reaction to the protocol. The conference had no comment session. I don’t know if this was by design or if time just ran out. NIST staff did indicate that next year the conference would be two days rather than one. However, a year is a long time to wait for reactions. This is especially pertinent since NIST is not a regulatory agency. Its protocols are strictly voluntary and depend on vendor acceptance.

What NIST did do is offer several presentations that emphasized how fragile patient safety can be in an HIT world. One breakout session used an actual, unnamed product’s screen that had dozens of misleading or ambiguous fields. For example, the screen’s fields cut off drug names, used red to indicate several different findings and used a pop up that blocked a view of a pertinent entry.

In another more broadly based patient safety presentation, University of Pennsylvania’s voluble Ross Koppel drove home how common elements in EMRs such as blood pressure – he’s found 40 different ways to show it so far – are subject to many formats for capture and display. Moreover, if you think EMRs have problems, Koppel shows how bar codes and work arounds can play havoc with workflow and patient safety.

Wanted: One Good Policy Compass

For those of us possessed of an EMR design demon, it was both a good chance to wonder out loud just what it all meant and where, if anywhere, things were headed. Sadly, the most common answer was who knows? There were some common points:

  • It’s better to have NIST’s protocol than not.
  • You can forget the FDA playing a bigger role. It’s under funded and over worked.
  • HIMSS will wait for the industry and the industry has shown no hurry.
  • EMR adverse incident reporting would be great, but who would do it and how open would it be?

In short, if you’re shopping for an EMR, regardless of your size, don’t count on anyone handing you a usability report on an EMR anytime soon. Moreover, don’t try to run NIST’s protocol on your own unless you have full access to the proposed EMR, lots of time on your hands and a good grasp of the protocols details.

There are some things you can do. You can ask potential vendors questions such as these:

  • Have they run the NIST protocol and what did they do as a result?
  • If not NIST, do they have a written usability protocol and, if so, can you see it? How have they implemented it?
  • Have they tested their EMR’s usability with outside, independent users? What were the results?
  • Have they used any interface designers?
  • What usability changes do they plan?

There is no guarantee that you’ll get a great product, but it could mean that you get one that doesn’t bite your patients or you.

June 14, 2012 I Written By

When Carl Bergman isn't rooting for the Washington Nationals or searching for a Steeler bar, he’s Managing Partner of EHRSelector.com, a free service for matching users and EHRs. For the last dozen years, he’s concentrated on EHR consulting and writing. He spent the 80s and 90s as an itinerant project manger doing his small part for the dot com bubble. Prior to that, Bergman served a ten year stretch in the District of Columbia government as a policy and fiscal analyst.

NIST May Standardize The Cloud, Even If It’s Too Late For EMRs

Over at the august halls of the National Institute of Standards and Technology, researchers have been compiling data on what makes EMRs usable. A year ago, in April 2011, NIST presented a draft set of usability standards. At the same hearing, a wide range of academics and scholars got up to talk about what they saw as they key issues — including whether EMR workflow should be changed to make it cost-efficient.

Since then, from what I can tell, there’s been a lot of noise but little light shed on the design specs a truly usable EMR should adhere to.  There’s been some progress in the development of HIE connections between EMRs, some worthwhile work EMR return on investment and even some improvements that might leverage EMRs to help doctors collect more from patients.

Talk to many doctors, and they’ll tell you their EMR stinks. Why? Largely because workflow is still inefficient and the “click burden,” which can drive doctors through a dozen steps to get tasks handled, hasn’t been reduced any too much. Some older docs I’ve spoken with even pine for the rough-hewn EMRs of 20 to 30 years ago more, which were at least built by their colleagues.

Honestly, I don’t expect the “awkward interface” problem to go away anytime soon. But while we stew on this issue, you might be interested to learn that NIST is taking over a few related problems in which it could conceivably make a real difference.

A few months ago, NIST released the 16th and final draft of its recommendations on definition of cloud computing. (Talk about insisting on getting it right!)  Not everyone in the health IT industry is even aware that NIST has kicked out a cloud standards document, which our friend Shahid Shah, “The Healthcare Guy,” is urging people to get onto their radar.  Maybe this time, NIST has a chance to actually standardize before an industry runs while with its own implementations of key technology.

I think I’ll finish with Shahid’s comments on the subject, as I think he’s pretty clearly got it right:

My strong recommendation to all senior healthcare executives is that we not come up with our own definitions for cloud components – instead, when communicating anything about the cloud we should instruct our customers about NIST’s definition and then tie our product offerings to those definitions. The essential characteristics, deployment models, and service models have already been established and we should use them. When we do that, customers know that we’re not trying to confuse them and that they have an independent way of verifying our cloud offerings as real or vapor.

March 5, 2012 I Written By

Anne Zieger is veteran healthcare consultant and analyst with 20 years of industry experience. Zieger formerly served as editor-in-chief of FierceHealthcare.com and her commentaries have appeared in dozens of international business publications, including Forbes, Business Week and Information Week. She has also contributed content to hundreds of healthcare and health IT organizations, including several Fortune 500 companies. Contact her at @annezieger on Twitter.

FDA, EHREvent, NIST: Who’s up for an EMR Supercop gig?

Last week I wrote wondering who will police EMRs and EHRs. With the release of IOM’s report recommending the creation of a different federal agency to serve as EMR watchdog, this topic has been generating buzz in healthcare circles. I’m by no means an expert in healthcare IT or policy matters but the discussion surrounding this topic has helped me think things through better than last week. Commenter Don Fluckinger answered the blog post with the first comment on the post – saying “these guys” and pointing to EHREvent.org. Commenter Carl Bergman said the FDA, which is already tasked with gathering adverse events for medical devices, might be the ideal go-to-agency for software adverse events as well. It is my understanding that medical software would receive Category 3 classification, if FDA were to provide the oversight.

IOM’s approach has been to suggest the creation of a non-regulatory, NTSB-like body. IOM’s rationale for undercutting FDA’s role has been that FDA classification system might stifle health IT innovation. (I’ve only had the time to read the very first few pages summarizing the rest of the IOM report, so I’m not sure if/how they address these concerns later.)

Here’s what I don’t get: What’s the point of creating yet another powerless body to issue guidelines? If there’s already a body with regulatory and oversight powers that covers your domain, has a large database of medical device related adverse events, why can its capabilities not be extended further to medical software as well? Further, why are health IT vendors exempt from any slaps on the wrist?

No offense to anyone, but from what I’m reading about EHRevent.org, I don’t see much to recommend them: John says they “are not going to have high enough profile to be able to really collect the reports… a reporting system is great, but if no one knows to report something there, then it’s not worth much. Plus, if someone reports something but the organization doesn’t do anything with that information, it’s not very meaningful”. Valid question but I think there could be some easy workarounds for the problem of not knowing how/where to report shouldn’t be a major issue. Healthcare IT just needs the software equivalents of those “How’s my driving?” flaps adorning the backs of 18-wheelers. The bigger question is what happens when the EMR system fails? Who pays? How much? How does the vendor ensure the failure doesn’t happen again? Do we learn from the cumulative mistakes of the industry? Time will tell.

November 15, 2011 I Written By

Priya Ramachandran is a Maryland based freelance writer. In a former life, she wrote software code and managed Sarbanes Oxley related audits for IT departments. She now enjoys writing about healthcare, science and technology.

It’s About Time: Government Workshop Will Address EMR Usability

As you may know, not long ago I wrote up a rant slamming awkward-to-use EHR interfaces, an article which has drawn plenty of reader debate and discussion.

Today, I learned that the government is paying attention to this issue as well (and not a moment too soon).

The National Institute of Standards and Technology (NIST, to its friends and relations) announced that it would be holding a free workshop on EHR usability intended help all sectors of the industry collaborate on the problem. NIST hopes to attract industry players, academics, government officials and healthcare providers to the same table.

The workshop, which will take place on Tuesday, June 7 on NIST’s Gaithersburg, MD headquarters, will focus on four key questions:

  • What facets of usability should be measured?
  • What measurement methods and protocols should be used to do this?
  • What are some of the challenges to rigorous measurement and how can they be addressed?
  • How can measurement results stimulate a market and support improved usability?

Call me a cynic, but I don’t see participants making a lot of progress on these questions in a single day. Heck, you could spend weeks or months on any of these issues and still end up spinning your wheels.

That being said, it’s always good to see the bureaucrats pay attention to an issue like this. Why? Because if bureaucrats have any virtues, one is that when they grab an issue, they tend to stick to it. (I’d rather see CMS dig into this topic, but hey, NIST’s a start.)

With hundreds of EHR vendors competing for mindshare out there, it’s not likely they’ll come together to set usability standards on their own. But if pesky government types — from both the policy and tech sides — decide to dig their teeth into the usability problem, it’s probably a good thing.

I don’t know about you, but I think attending is a great idea. If any of you make it, please feel free to let us know what you learned. It should be an interesting session.

May 25, 2011 I Written By

Katherine Rourke is a healthcare journalist who has written about the industry for 30 years. Her work has appeared in all of the leading healthcare industry publications, and she's served as editor in chief of several healthcare B2B sites.

ONC ATCB EHR Certification Process

Ok, was that enough abbreviations in the title of a post? Well, if you care about this post, you’ll probably recognize all of the abbreviations.

In a post I did on EMR and HIPAA about SureScripts as an ePrescribing ATCB, there was a comment made that possibly some of the ATCB were “in bed” with ONC in order to get their EHR certification body status. In response to the comment, Mark Joyce from SLI Global Solutions (an ATCB) provided some good insight into the process and costs associated with becoming an ATCB that can certify EHR software.

As the team lead for SLI’s application to the ONC I can assure you that our company has no political connections, traded favors or made contributions that won us our certification by the ONC. It was 10 weeks of grueling research by two independent companies (one company focusing on testing and the other certification) that resulted in a 1200 page application.

The application was in two parts: part one required both companies to expand and/or create a Quality Management System for the new process. It’s no easy task to develop both a 17025 and a Guide 65 conformant QMS. Part two required the applicant to have a thorough understanding of EHR architectures as well the NIST testing procedures and tools.

It was evident by the followup questions from the ONC that the application had been very carefully reviewed.

Obtaining certification from the ONC was no easy task. I am proud to be a part of our companies significant investment in the ATCB testing and certification process.

Yes, becoming an ONC-ATCB is definitely not a walk in the park to achieve. Anyone that says otherwise, likely hasn’t ever been through the process.

January 3, 2011 I Written By

John Lynn is the Founder of the HealthcareScene.com blog network which currently consists of 15 blogs containing almost 6000 articles with John having written over 3000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 14 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John launched two new companies: InfluentialNetworks.com and Physia.com, and is an advisor to docBeat. John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and Google Plus. Healthcare Scene can be found on Google+ as well.