Dr. Val raises hell over FreeMD

The medical blog-o-sphere is buzzing after Dr. Val's post about FreeMD. FreeMD is advertised as a free web based triage tool (originally designed to be used for phone triage with nursing staff now offered to the general public) to help people decide whether to go to the doctor. It's owner, Stephen J. Schueler MD is defending its' use but the medical blog-o-sphere is ripping it apart.



See the posts (and comments) at the Happy Hospitalist and Mexico Medical Student.



In defence of the product, Dr. Schueler stated "DSHI has validated this process through millions of interactions over a period of 8 years as well a published studies." The first link is to an interview with Dr. Schueler at a blog called MarketIntel. The second is to a single study which compared a clinic diagnosis to the web diagnosis in 59 students and found good agreement between the two (kappa value of .89). The web tool screened a total of 1290 people and advised 22.7% (293) of them that they did not need to see an MD.



From a purely academic point of view and based on the abstract only, the article is of low quality. The gold standard (the clinic diagnosis) was provided for only 59/1290 subjects. Long term follow-up was not used as a gold standard just the initial clinic diagnosis. Finally, the study reports kappa values. I have designed, researched and published on diagnostic test. Normally you provide sensitivity (how often the test diagnosis disease when it is truly present), specificity (how often it diagnosed lack of disease when it was truly absent) and predictive values. Since the report did not follow up with the 293 people it advised not to see an MD, specificity could not be calculated. Since it applied the gold standard of clinic diagnosis to only 4.6% of people that used the service the reported sensitivity is highly suspect.

The real question is; does FreeMD correctly predict whether someone does or does not need to go to the doctor? If the program is accurate Dr. Schueler, has not provided the evidence in my mind. It is not enough to claim experience as evidence of accuracy in a diagnostic test. It requires more rigorous research. The question of the study would not be diagnosis but the decision to seek medical assistance. Because it is a simple question the study would be straight forward. My opinion is that offering a diagnosis (regardless of the disclaimer) and advice on whether to seek help has greater potential for harm than someone looking symptoms up on-line and deciding themselves, however, I cannot prove the point.



**BTW check out Mexico Medical Student's blog post on Canada Day - the video is NSFW but funny as hell. Thanks for celebrating with us Enrico.