Free EMR Newsletter Want to receive the latest news on EMR, Meaningful Use, ARRA and Healthcare IT sent straight to your email? Join thousands of healthcare pros who subscribe to EMR and EHR for FREE!

A Tool For Evaluating E-Health Applications

Posted on April 11, 2017 I Written By

Anne Zieger is veteran healthcare consultant and analyst with 20 years of industry experience. Zieger formerly served as editor-in-chief of and her commentaries have appeared in dozens of international business publications, including Forbes, Business Week and Information Week. She has also contributed content to hundreds of healthcare and health IT organizations, including several Fortune 500 companies. Contact her at @ziegerhealth on Twitter or visit her site at Zieger Healthcare.

In recent years, developers have released a staggering number of mobile health applications, with nearly 150,000 available as of 2015. And the demand for such apps is rising, with the mHealth services market projected to reach $26 billion globally this year, according to analyst firm Research 2 Guidance.

Unfortunately, given the sheer volume of apps available, it’s tricky to separate the good from the bad. We haven’t even agreed on common standards by which to evaluate such apps, and neither regulatory agencies nor professional associations have taken a firm position on the subject.

For example, while we have seen groups like the American Medical Association endorse the use of mobile health applications, their acceptance came with several caveats. While the organization conceded that such apps might be OK, it noted that such approval applies only if the industry develops an evidence base demonstrating that the apps are accurate, effective, safe and secure. And other than broad practice guidelines, the trade group didn’t get into the details of how its members could evaluate app quality.

However, at least one researcher has made an attempt at developing standards which identify the best e-Health software apps and computer programs. Assistant professor Amit Baumel, PhD, of the Feinstein Institute for Medical Research, has recently led a team that created a tool to evaluate the quality and therapeutic potential of such applications.

To do his research, a write-up of which was published in the Journal of Medical Internet Research, Baumel developed an app-rating tool named Enlight. Rather than using automated analytics, Enlight was designed as a manual scale to be filled out by trained raters.

To create the foundation for Enlight, researchers reviewed existing literature to decide which criteria were relevant to determine app quality. The team identified a total of 476 criteria from 99 sources to build the tool. Later, the researchers tested Enlight on 42 mobile apps and 42 web-based programs targeting modifiable behaviors related to medical illness or mental health.

Once tested, researchers rolled out the tool. Enlight asked participants to score 11 different aspects of app quality, including usability, visual design, therapeutic persuasiveness and privacy. When they evaluated the responses, they found that Enlighten raters reached substantially similar results when rating a given app. They also found that all of the eHealth apps rated “fair” or above received the same range of scores for user engagement and content – which suggests that consumer app users have more consistent expectations than we might have expected.

That being said, Baumel’s team noted that even if raters like the content and found the design to be engaging, that didn’t necessarily mean that the app would change people’s behaviors. The researchers concluded that patients need not only a persuasive app design, but also qualities that support a therapeutic alliance.

In the future, the research team plans to research which aspects of app quality do a better job at predicting user behaviors. They’re also testing the feasibility of rolling out an Enlight-based recommendation system for clinicians and end users. If they do succeed, they’ll be addressing a real need. We can’t continue to integrate patient-generated app data until we can sort great apps from useless, inaccurate products.

Will AI (Artificial Intelligence) Provide Your Own Personal Health Coach?

Posted on November 20, 2015 I Written By

Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in open source, software engineering, and health IT, but his editorial output has ranged from a legal guide covering intellectual property to a graphic novel about teenage hackers. His articles have appeared often on EMR & EHR and other blogs in the health IT space. Andy also writes often for O'Reilly's Radar site ( and other publications on policy issues related to the Internet and on trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business. Conferences where he has presented talks include O'Reilly's Open Source Convention, FISL (Brazil), FOSDEM, and DebConf.

Adhering to the principle that health improvement is based on sustained behavior change, and that behavior change is based on a profound intervention by health care providers in a patient’s daily activities, a certain fantasy has made the rounds of the health care industry. In this article I’ll describe this fantasy and a product by Lark Technologies that starts to realize the fantasy in real life.

First, a bit of comparison. Some 15 years ago I shed a lot of weight (and kept it off) through firm but supportive monitoring by the health care profession. I went to my primary care physician every three months, and visited a nutritionist twice. I also exploited my personal network, conveying my goal to all my friends and lining them up to support eating habits that would lead there.

This model doesn’t scale well. Furthermore, a visit every 90 days is no match for the temptations that scream at me from the billboards and restaurant windows. (Believe me, as a chronic dieter I am very aware of the food industry’s marketing techniques.) So numerous technologists have imagined virtual assistants that follow you around and act like an intrusive Mom, asking you why you’re buying that donut or whether you’ve signed up for the health club yet.

These assistants would have to be subtle and very well tailored to your personal style to be affective. At the recent Connected Health Conference, MC Joseph Kvedar laid out a requirements list for such assistants. They must be:


To serve billions of people, these systems can’t depend on constant communication with health professionals. Somehow, software must be observer of your habits, come to know you from your demographics and health conditions, and intervene at appropriate moments with messages that have a chance of getting through the armor of your established routines.


This fancy word just reflects the kind of empathetic adaptions each of us does all the time to reflect the situation we’re in. Just as we would shout “Stop” to someone about to step in front of a trolley but “Excuse me, did you want this trolley?” to someone absorbed in her cell phone, contextual software understands that you like donuts (but would enjoy a good fruit salad if offered one), that you like to exercise before work instead of at lunch time, and so on. The interventions it makes for each person would be unique.


Positives work better than negatives in getting people to go along with suggestions. “Did you know that another round on the track will put you ahead of your walking record for yesterday?” works better than “Hey, you’ve been sitting for two hours–get up!”


If a user doesn’t like an app, he always has the option of turning it off. Therefore, a health app must reflect the user’s goals, not the goals of hie doctor, his daughter, or the Centers for Medicare & Medicaid Services. Empowering software will ask you what matters to you–for instance, being able to play with your grandchildren or stay in your third-story apartment–and remind you of these goals as a way to persuade you to stay on track.


I find this trait a bit redundant, if software is empowering. Dr. Kvedar suggested that people using this kind of personal agent get a discount on their health care premiums. I’m a fan of intrinsic rewards, myself. But the distinction can be hard to make. If an app sends you a message from your wife saying, “So proud that you lost five pounds this week!” is it an intrinsic or extrinsic reward?

At the conference I had the privilege of meeting with Julia Hu, cofounder and CEO of Lark Technologies, who showed off their personal weight loss coach, Lark Chat (available for download for Apple and Android). It was amazing how closely this software–available since this past April–matched the simulation that Dr. Kvedar showed off in his opening talk.

Lark Chat uses Siri software to accept voice input or a text message, which is then submitted to artificial intelligence software to respond appropriately to the user. When I told it what I (pretended I) had for lunch, the software readily understood french fries and salad, and made a comment on each. It did not understand what to do with breaded, fried fish, which ought to have triggered a warning. But it has been trained to understand a number of different foods enjoyed by different ethnic groups. Users can also opt into sharing the data collected by Lark so that it can run analytics and improve its interventions.

The interface is enjoyable and popular. According to Hu, “Over the last four months, the Lark coach and its users have text messaged each other 350,000,000 times. Based on a typical chronic disease case manager’s load, that’s equivalent to 25,402 full time nurses and coaches.” This adds up to the longest user engagement record of any interactive apps in the weight loss space.

The industry has been taking notice. Business Insider recently named Lark one of the 10 most innovative apps in the world, and Apple once featured it as the “Best New App” in their App Store. Forrester Research named Lark the “Most Innovative Digital Health Product of the Year” in 2015 and published an exclusive report on it.

People are getting accustomed to apps such as Foursquare and interfaces such as Siri that in previous ages might have been seen as annoyingly intrusive. As our relationships to devices and software evolved, we may find apps such as Lark Chat the perfect support for behavior change. And we may all become better people as a result. If only Mom could have created an app for me.

We Know What’s Right, but It’s Hard

Posted on August 24, 2012 I Written By

John Lynn is the Founder of the blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of and John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

In perusing various blogs, I came across Matthew Gibson, MD’s blog and this really compelling article titled, “It’s So Easy, and Yet…” Here’s one especially poignant section:

What I see day in and day out is complications of simple, easy to manage problems like diabetes, high blood pressure, asthma, etc. These are things that we KNOW how to treat. We know how to prevent complications. And yet, I just had a man last week who required half of his foot to be amputated as a complication of untreated diabetes. I had a woman this week who came in seeing snakes on people’s clothing, because her blood pressure was so high it was affecting her mind. Last month, I saw a man who had large amounts of yeast growing in his mouth and groin because his blood sugar (and thus urinary sugar) was so high.

This morning, I’m caring for a truly pleasant gentleman with COPD (bad chronic lung disease usually caused by smoking). He hasn’t smoked in the last 15 years, but he smoked quite heavily before that. Even though he’s been doing things all right as far as his lungs are concerned for the last 15 years, he has to live with the consequences of his actions prior to that. For the last several days, I’ve seen him decompensate and gasp for air, feeling like he’s drowning, because he can’t get the air to move through his lungs like he should. How did this kind old man get to this point?

At the core of his comments is the idea of how do we motivate ourselves to do something we know we should be doing. This is a really hard question to answer and something we probably will never solve completely. However, I think there’s plenty of room to improve even if we never become perfect at it.

Over on Smart Phone Healthcare we’ve spent a lot of time reviewing various mobile health applications. I’d say that the large majority of mHealth applications are about trying to help solve this problem. Plus, I think the mobile device connected with good data about ourselves is one method that will help us be healthier.

Related to this idea, is what I’ve called treating healthy patients. This is a concept that won’t leave me since I think it will be a fundamental part of the future of healthcare. I believe we’re on the brink of a series of devices and technology that will help us monitor our bodies in such a way that we can identify sickness within us before we feel sick. This information won’t make everyone change their behaviors, but it will help many.

We’re in the very early stages of monitoring our bodies and connecting all that data with action. However, it’s exciting to see that now many of these things are possible thanks to powerful computing and a new generation of devices.

Prescribing Health Apps

Posted on July 6, 2012 I Written By

John Lynn is the Founder of the blog network which currently consists of 10 blogs containing over 8000 articles with John having written over 4000 of the articles himself. These EMR and Healthcare IT related articles have been viewed over 16 million times. John also manages Healthcare IT Central and Healthcare IT Today, the leading career Health IT job board and blog. John is co-founder of and John is highly involved in social media, and in addition to his blogs can also be found on Twitter: @techguy and @ehrandhit and LinkedIn.

I was really intrigued by an article in Forbes by David Shaywitz that takes an interesting look at Digital Health and why “pills still matter.” This paragraph sums up the article quite well and provides some interesting points of caution for those in Digital Health.

I read a tweet recently asserting that physicians may soon prescribe health apps as an alternative to medications; my initial reaction: good luck with that one. It’s certainly easy enough to envision how magical thinking about the power of health apps will soon be replaced by disappointment as app developers realize something drug makers have known for years: it’s hard to improve health, and it can be very difficult to get patients to stick with a treatment long enough to make a difference.

I’ve been watching the idea of prescribing an app for quite a while. In fact, we’ve written about the “virtual pharmacy” on Smart Phone Healthcare a few times. Plus, I’ve talked a number of times with John Moore about his new company Prescribable Apps. Isn’t that the perfect name for an mHealth company?

Everyone has to agree that the concept of being able to prescribe something as simple as a mobile app that can improve your medical condition is very cool. Although David Shaywitz’s warning is a good one: It’s hard to improve health, and it can be very difficult to get patients to stick with a treatment long enough to make a difference.

He’s absolutely right that those are the challenges for mobile health apps. Although, while that is the challenge of mobile health apps, as an entrepreneur it is also the huge opportunity. It will be hard for a mobile health apps to really improve health. Although those that do achieve that noble and important goal will receive a HUGE reward. Just like pharmaceutical companies that discover a new drug that improves health end up with a huge financial windfall, the mobile health apps will see the same result.

Of course there are a lot of important questions about how to know which mobile health apps really do improve health. What’s going to be the filter? Who and how will we measure the effectiveness of all these apps? Then, of course will there be someone monitoring the mobile health apps the way we monitor drugs? Many of these questions will need answers, but I’m quite certain that some mobile health apps will improve health and those that do will be very successful.

I wonder if any EHR companies have thought about how to integrate mobile health app prescription into their EHR. It’s a bit premature to really do so, but I’ll be interested to see how they make it happen.

How to Track Your Health – Jump on the Self-Logging Bandwagon

Posted on February 6, 2012 I Written By

Priya Ramachandran is a Maryland based freelance writer. In a former life, she wrote software code and managed Sarbanes Oxley related audits for IT departments. She now enjoys writing about healthcare, science and technology.

Last week, I wrote about Margalit Gur-Arie’s idea for a national health data repository. Commenters, including Gur-Arie, had some great comments. And one of the things that came out of that post was that if there is ever a community run national database, I would happily log my health data into it, and maybe do it for my parents and kids as well. But I don’t really have a great track record of health-logging. I’m not a Quantified Self afficionado, I don’t even check my weight on a regular enough basis. Having my husband photograph me and gauging my weight by the relative puffiness of my cheeks is what constitutes a weight check in my universe. So what are the odds of me going through piles of paper records and typing it all up for a UHR? Slim to none, but a girl can still dream.

Wired published an article late last month on How to Use Tech to Track Your Health, just for people like me. Don’t worry, assures the article, self-logging ain’t that hard, it’s probably so easy even a cavewoman can do it.

Now, just because the article is on Wired doesn’t mean it comes with awesome pedigree and stellar writing. People, it’s 2012, not 1992. If I hear about one more app that sends you text reminders about getting your annuals done, I’m going to barf. Ditto for those tired ovulation trackers. I don’t know who I blame more. the tech companies that come up with these novel solutions after every iPhone developer and their mother has already created a dozen apps and websites around the idea, or the Wired writer who thought it was newsworthy enough to include in her round-up of the most happening ideas. Either way, I don’t care. I don’t want to know about any more reminder and calendar apps, not unless this magical app connects me directly to someone like Simon Cowell or Idris Elba (ok, I have a thing or two for Brit accents). See, there’s a business idea for you – have celebrities become our health coaches and cheerleaders. Some of us might be ready to pay for the privilege.

There are the usual sleep-pattern analyzer apps and personal sleep coaches that help you get a good night’s worth.

Kvetching aside, not everything in the Wired article sounded like it was floated circa the 1900s. I quite liked reading about the BodyMedia biometric patch that apparently works like one of those ciggie patches, and logs your sleep, heart rate etc for upto a week. It is intended to be used a week prior to your appointments, presumably so you can bombard your doctor with the aforesaid minutiae. I also liked the Moodscope idea – you can go to the Moodscope website and play a game, and your reponses to the game are used to gauge your mood. I can imagine how useful that could be to people who might be prone to depression or who want to chart what external stresses produce what reactions in them.

I haven’t used any of the apps or gadgets mentioned in the article, so I’m just reacting at a very surface level to the idea behind the product/service. And your mileage may, of course, vary.

Check it out here.