Better evaluation studies in eHealth and telemedicine

Nina Eminovic
Dept. of Medical Informatics, Academic Medical Centre, Amsterdam

Jeremy Wyatt
Health Informatics Centre, University of Dundee, Dundee, UK

Topic: Evaluation and methodological issues in ehealth
Track: Research
Type: Oral presentation

     Full text: Not available
     Last modified: July 26, 2006
     Presentation date: 10/14/2006 1:00 PM in CGEI Telehealth Room
     (View Schedule)

Abstract
Tutorial proposal
Better evaluation studies in eHealth and telemedicine
J.C. Wyatt, Professor of Health Informatics,
Health Informatics Centre, University of Dundee, Dundee, UK
[email protected]

N. Eminović, research fellow
Dept. of Medical Informatics, Academic Medical Centre, Amsterdam, the Netherlands
[email protected]

1 Abstract
Many studies have been performed on eHealth and telemedicine systems using a variety of clinical and non-clinical outcome measures and different research designs, to gain scientific evidence for the benefits of these tools. Although there are strong indications that telemedicine and eHealth are useful, and in some countries telemedicine is already implemented in routine care, there is still little valid scientific evidence for its benefits. Systematic reviews have shown that the main reason for this is the low quality of the evaluation studies. Evaluation studies in telemedicine - and in health informatics in general - are often difficult to design and perform. Nevertheless, health policy makers, clinicians and patients, let alone lawyers and tax payers, will require evidence from well designed and performed evaluation studies prior to its integration into routine care.
Depending on the specialty and goals of the eHealth or telemedicine system, different outcome measures are important. In visually oriented specialties such as dermatology and pathology, the diagnostic accuracy of the digital images is usually first examined, but even here, various approaches and types of analysis have been used. In the first part of this tutorial, existing literature on evaluation in telemedicine will be discussed with an emphasis on teledermatology. The pros and cons of various outcome measures will be explained with examples from the literature.
In the second part of the tutorial, the full range of evaluative approaches that can be applied in telemedicine and a wide range of questions that carefully-designed evaluations can address will be presented. This part of the tutorial will be based on a textbook by Wyatt and Friedman (Evaluation Methods in Biomedical Informatics, second edition, 2005) and illustrated with our experiences from designing and performing pilot studies for the English NHS Direct service and a multi-centre cluster randomized controlled trial in teledermatology.
2 Educational goals
By the end of the tutorial, participants will be able to:
1. Describe the most useful outcome measures in telemedicine and how to measure them
2. Define the aims, process and role of evaluation within the field of telemedicine and eHealth.
3. Identify and develop specific evaluation questions appropriate to a telemedicine or eHealth project.
4. Critique quantitative and qualitative evaluation studies with attention to measurement issues and demonstration study design.

3 Who should attend
The tutorial is appropriate for anyone interested in eHealth and telemedicine, and is essential for anyone designing, carrying out or critically appraising evaluation studies.

4 Level of content (basic, intermediate, and advanced)
80% basic, 20% intermediate

5 Prerequisites

There are no formal prerequisites. However, participants should have a general familiarity with the field of biomedical/health informatics, eHealth or telemedicine. Participants might benefit more if they have some familiarity with basic statistical concepts such as standard deviation and correlation.
6 The name of a reference from an organization that has previously sponsored the same or another tutorial of similar duration by the instructor?
The first author has presented a generic version of this tutorial at AMIA Fall Symposia; AIME and MedInfo conferences
7 list of audio-visual equipment (incl. Web access needed?)
- PC
- Projector