John’s Note: I got the following message from an EMR vendor a while back. I’ve removed the specific names of the EMR vendors to protect the innocent (and guilty in some cases). Plus, the names of the specific companies are tangential to the issue of ratings services like KLAS.

I’m sure this post will drive some interesting discussion in the comments.

I have been talking with [someone] at KLAS since HIMSS about WHY the KLAS data is either 1) Losing Relevance, or 2) Actually Misleading Providers.

[KLAS representative] told me in April that the average annual “accesses” to the KLAS data was about 17,000, while in 2010, they had over 14,000 through mid-April. Big trend change. ONC-driven, no doubt.

I had a discussion with [KLAS representative] at HIMSS and since, outlining how the KLAS data can and probably IS misleading clients. How? For instance….[EMR Vendor A] gets high marks…almost as high as [EMR Vendor B]
.
They have two “konfidence” checks on their data for the 6-25 provider practice size category (KLAS has 1-5, 6-25, and 25+, I believe as their categories, and vendors must submit their reference accounts in those partitions).

If you are a provider who sees high marks for [EMR Vendor A] in the 6-25 provider space…with all the good supporting comments…..well above the other vendors….you would probably rely on that data to decide they have to be on your short list. But everyone knows that [EMR Vendor A] serves mostly primary care and ob/gyn practices. They have adapted templates and have references for those accounts. What if you are an orthopedist or ophthalmologist…or other specialist. They have almost NO references for that. The KLAS data does not break it out. You may be badly mislead….and find out shortly after spending tens of thousands of dollars…that it wont work at all for you.

We have asked them to break out the data by specialty… they know who they are talking to. They know what each practice is. Not hard to add the question…so the data can be sliced that way. Tremendous value-add for ALL practices to know how the vendors break out. Guess what they are going to find out when they do that? And by the way…they ONLY gather data from accounts that have been installed and live for a certain period of time….no data on failed attempts or those that gave up using products.

KLAS agreed in principle it was a good idea to add specialty to their data. They even agreed to start doing it with the last reporting cycle, but didn’t have much time. All the data they have….not normalized by specialty.

LIKE THE GOVERNMENT, THEY ARE ASSUMING ALL SPECIALTIES CAN BE TREATED THE SAME AS FAR AS EMR ADOPTION GOES. Why do they think there is a 50-83% failure rate in the industry? Why has “meaningful use” not been defined for specialties before “certification” of a product can be decided? Does an orthopedist who does hip replacements, rotator cuff surgery and meniscus revisions…..report the same thing as doctors who treat diabetes, liver disease or other costly chronic-care conditions? Heck No.

I don’t think the market fully realizes how “homogenized” the KLAS data is….after all, if the average depth of the Mississippi River is only three feet…..why are there so many different size boats—-barges to sailboats—sailing in it each day?