The Health Applications Evaluator was created to provide patients and healthcare professionals with an informed overview of the mobile health and health behavior applications used in Estonia. Each of these applications has a greater or lesser impact on health and, in most cases, also processes health data. Health Applications Evaluator provides comprehensive information on the health applications created and used in Estonia in all categories that may potentially affect users.
It is more convenient to use a web browser on your computer to view the Health Applications Evaluator.
There are three main target groups of the health applications evaluator:
- PEOPLE, who want more information about the health applications they have started to use to maintain health on their own or on the advice of a healthcare professional.
- HEALTHCARE PROFESSIONALS, who want to recommend health applications to their patients and want to get a systematic review and evaluation of them beforehand.
- SOLUTION OWNER, who wants to develop and improve its product through a solid framework that will help one expand a potential medical device.
The evaluator of the health application maps the following information categories for each application:
- general information and purpose, including who the application was created for, who can use it, what the purpose of the application is, and what functions the application has.
- security and privacy, including whether the application complies with data protection requirements and whether the application developer has put in place appropriate security measures.
- the level of user and clinical research, including whether the application developer has measured the impact of the application, whether similar applications have been evaluated elsewhere in the world, user impact or user satisfaction has been studied and whether the application has been adequately assessed for the purpose and standard of the application.
- interoperability and interface, including whether the application developer has assessed the need to transfer data to other health information systems, whether common health standards are used to handle the data.
Assessed health applications
Triumfland Saga
A playful app designed for children that provides evidence-based educational content related to health behaviors. The application has been developed in cooperation with different stakeholders (children, parents, and health experts).
Summary of assessments
In the case of the Triumfland Saga application, the level of clinical evidence corresponding to the purpose set for the application by the manufacturer is high. Several clinical studies have been conducted during the development of the application, which have been published or are in the process of being published. Feedback about the application is constantly collected from users. The level of security and privacy of Triumfland Saga is high - the application covers all GDPR and security requirements. The purpose of the application does not provide for interfacing with other information systems; therefore, the interface category does not apply.
Migrevention
A digital migraine and headache diary that helps the user record headache attacks and track headache triggers.
Summary of assessments
In the case of the Migrevention application, the level of clinical evidence corresponding to the purpose set for the application by the manufacturer is high. A clinical study was conducted during the development of the application, which is being published. Feedback about the application is constantly collected from users. The security and privacy level of Migrevention is high - the application covers all GDPR and security requirements. The purpose of the application does not provide for interfacing with other information systems; therefore, the interface category does not apply.
Kõneravi.ee
Kõneravi.ee is an online environment with more than 150 interactive exercises created in cooperation with experienced speech therapists for everyone who wants to develop their speech. Exercises can be done both in cooperation with a specialist (speech therapist, special pedagogue, or teacher) and independently. Kõneravi.ee makes therapy diverse and interesting for children and patients. For specialists, Kõneravi.ee is a supportive work tool that helps prepare personalized exercise plans, monitor progress, and create new exercises.
Summary of assessments
The level of security and privacy of the Kõneravi.ee application is high - the application covers all GDPR and security requirements. The level of user and clinical studies of Kõneravi.ee is average. The Kõneravi.ee application has an educational purpose, supporting children's speech development, but it is also used as a supporting tool by specialists who support speech development (speech therapists, special pedagogues, pedagogues). The application has been developed in cooperation with industry organizations and specialists and regularly collects user feedback. Currently, the app is not designed for any specific medical condition or speech disorder and does not rely on research conducted with the app itself. However, similar applications have been evaluated around the world and their results show that digital therapy solutions can have a clinical effect in various speech disorders. If the creators want to attribute a clinical purpose to the application, it is important to first conduct a clinical study that would assess the app's suitability for dealing with specific speech development disorders. The purpose of the application does not envisage interfacing with other healthcare information systems; therefore, the interface category does not apply.
TRIUMFLAND SAGA- General information
|
iOS version and evaluation time |
Version 1.2.8.8/February-March 2021 |
|
Android version and evaluation time |
Version 1.2.8.8/February-March 2021 |
|
Platform |
|
|
Minimum requirements for an iOS device or software |
Apple App Store minimum requirements apply. |
|
Minimum requirements for an Android device or software |
Minimum Google Play requirements apply. The application is optimized for Samsung S7 Edge. |
|
Does the application need an external medical or non-medical device to work? |
No |
|
Location of data storage - country, server provider |
The server is located in the European Union (Google Firebase). |
|
What is the purpose of the application? |
Improving children's health education and playful mental health promotion. |
|
The target audience of the application |
Children from 7 to 14 years old. |
|
Is the application intended for use with a specific health problem? |
No |
|
Health issues for which the application is suitable |
Not applicable |
|
Is the application intended for use with a specific health risk? |
No |
|
How many specific unique functionalities are there that fulfill the main purpose of the application? |
11 |
|
List of the main functions of the application |
|
|
How many additional functionalities does the application have that are not necessary to fulfill the main objectives of the application? |
10 |
|
Instructions for use |
Yes, the user manual can be downloaded HERE. |
|
Provider's website reference |
|
|
Price |
Regular price € 29.90 (one-time fee). Access to the app is temporarily sponsored and therefore currently free. |
|
Languages |
|
|
Which specialty professionals prescribe the application? |
Not applicable |
|
Are there any restrictions on using the application (age, specific health problem, etc.)? |
Children from 7 to 14 years old. |
|
Which official organizations (e.g. national organizations or medical professional associations) recommend the application? |
Not applicable. |
|
Medical device risk class (if applicable) |
Not applicable. |
|
Is the application registered as a medical device? |
No |
|
Does the European Union Medical Devices Regulation apply to the application? |
No. Manufacturer's Rationale: The application is only intended to store and, if necessary, download user data. |
TRIUMFLAND SAGA - Privacy and security
|
Does the app have a privacy policy (privacy policy notice)? |
Yes |
|
Does the privacy policy contain the required information (based on the GDPR)? |
Yes |
|
Is the privacy policy accessible during registration and use of the app? |
Yes |
|
Does the app collect personal information? |
Yes |
|
Does the app collect special types of personal information (e.g., health information)? |
Yes |
|
Has the company appointed a data protection specialist? |
No |
|
Is the user asked for consent to process the data? |
Yes |
|
Is the consent given voluntary, specific, informed and unambiguous? |
Yes |
|
Can the user withdraw their consent? |
Yes |
|
Does the user have access to personal information? |
Yes, but one needs to send an email to get access. |
|
Can the user change personal information? |
Yes, but one needs to send an email to make changes. |
|
Can the user request (or download) personal information? |
Yes, but one needs to send an email. |
|
Can the user delete personal information? |
Yes, but one needs to send an email. |
|
Does the app share personal information? |
No |
|
Does personal information move abroad? |
Yes, to store data on a Google Firebase server within the European Union. |
|
Does the application use cookies or similar technologies? |
Yes |
|
Does the application use cookies or similar technologies correctly? |
Yes |
|
Are minors using the app? |
Yes |
|
Does the app require adult (guardian) consent? |
No |
|
Does the app check the user's age? |
No |
|
Does the application use opt-in marketing opportunities? |
No |
|
Does the application provide information on the implementation of correct technical and organizational measures to protect personal data? |
Yes |
|
Is user authentication secure (password policy applied, 2-step authentication, PIN)? |
Yes |
|
Does the application use Mobile-ID, Smart-ID, or a similar eID authentication solution? |
No |
|
Does the app's privacy policy specify where the data will be stored? |
Yes |
|
Does the app's privacy policy state how long data will be stored? |
Yes |
|
Will the user be deleted after a certain period of inactivity? |
Yes |
|
Have there been any reports of data breaches? |
No |
TRIUMFLAND SAGA- Level of user and clinical trials
|
Are medical institutions and/or governmental organizations involved in setting up the application? |
Yes, Tartu University Hospital, Tallinn Children's Hospital Foundation, Helsinki University Central Hospital, The North Estonia Medical Centre, Uppsala University, Tampere University, National University of Singapore. |
|
Are healthcare professionals involved in the content creation of the application? |
Yes |
|
Are people/patients in the application content involved? |
Yes |
|
What are the clinical goals of the application? |
The application has an educational purpose, i.e. it shares health-related information and thus supports health behavior. |
|
Is the demonstration that the application can meet such clinical goals based on previous (international) research on similar technology / solutions? |
The evidence is mainly based on studies performed and / or ongoing in different patient groups based on the application of Triumfland Hero with the same technology but for a different target group: children with cancer, children with type 1 diabetes and children in need of surgery. |
|
What research has been done and what do the results show? |
The results of studies published at the time of the evaluation of the application show that the application provides useful and reliable information to pediatric patients and may have a positive impact on the psychological well-being of children with a diagnosis of cancer.
|
|
Are measurements made and, if so, what measurements are made in the application? |
Due to the educational purpose of the Triumfland Saga game, no diagnostic or therapeutic and interventional measurements are performed in the application. However, during the game, suitable gaming functionalities are offered based on the user's profile. |
|
Has the accuracy of application measurements been evaluated in the past for solutions with similar technology? |
Yes |
|
What specific studies have been carried out on the accuracy of the measurements and what do the results show? |
A study using the same technology, Triumfland Hero, has evaluated the measurement of a child's emotional state against an internationally recognized Strengths and Difficulties Questionnaire (SDQ). The measurement performed in the application was found to be accurate enough to assess the child's emotional state. In the Triumph Saga application, this measurement is not used to assess the child's emotional state, but there is functionality to perform the measurement. |
|
Is the evidence that the application can meet its clinical objectives based on research with that particular application? |
Yes |
|
What kind of research has been done with a specific application and what do the results show? |
The application is based on a survey of general population PROMs based on questionnaires. Results are yet to be published. |
|
Have any research articles been published on the clinical relevance/evidence of a specific application? |
No, but it will be released soon. |
|
In which journals have articles about a specific application been published? |
Not applicable |
|
Does the application manufacturer periodically ask users for feedback based on a methodology (specify frequency and methodology)? |
Yes, the app will ask users for feedback 30 days after using the app, using a tool developed based on various research for that particular app. |
|
To what level of evidence does the application belong to the NICE Digital Health Technology Assessment Framework? |
Level 2, the application is classified as information provision functionality. |
|
Does the application have a minimum or best practice standard of proof of effectiveness? |
Standard for demonstrating the effectiveness of best practice. |
|
Does the application meet this applicable level and standard of proof of effectiveness? |
Yes |
TRIUMFLAND SAGA - Interoperability and data quality
|
Does the application exchange data with other applications? |
No |
|
Does the application use standard terminology (ICD-10, SNOMED, HPO, ATC, etc.) when collecting data? |
No |
|
Does the application use standard clinical data models (e.g. OpenEHR, OpenClinica)? |
No |
|
Does the application provide an API for exchanging data with other applications? |
No |
|
Is the application compatible with other wireless devices (Bluetooth, NFC, GSM) to exchange data? |
No |
|
Is the application able to exchange messages in HL7 V2, 3 and FHIR format? |
No |
|
Does the application support automatic data entry via direct import (csv, CDISC, Json, xml), API, SFTP, IHE, HL7? |
No |
|
Has the application implemented any standard or IHE workflows? |
No |
|
Does the application work with CDA, C-CDA documents (XML based HL7)? |
No |
|
Can the data be exported from the application in a standard format (XML, CSV, CDISC, etc.)? |
Yes (CVS) |
|
Does the application use a GDPR compliant cloud service to store data? |
Yes |
|
Does the application have an interoperability and interface strategy for interfacing with other health information systems (including national systems)? |
No |
Migrevention - General information
|
iOS version and evaluation time |
2.3/October-November 2021 |
|
Android version and evaluation time |
2.3/October-November 2021 |
|
Platform |
|
|
Minimum requirements for an iOS device or software |
iOS 10.0 or later version |
|
Minimum requirements for an Android device or software |
Android 6.0 or higher (API level 23) |
|
Does the application need an external medical or non-medical device to work? |
No |
|
Location of data storage - country, server provider |
Germany, Frankfurt. Server provider Digital Ocean (https://www.digitalocean.com) |
|
What is the purpose of the application? |
The basic version of the app helps the user to record headache attacks in a digital diary, monitor headache factors and use of medication. Its implicit purpose is to support the diagnosis and treatment of primary headaches if the patient chooses to download their headache diary data and share it independently from the app with the headache specialist. The premium version of the application also allows you to use the advice of a headache nurse via online chat. |
|
The target audience of the application |
People with migraine and other primary headache diagnoses. |
|
Is the application intended for use with a specific health problem? |
Yes |
|
Health issues for which the application is suitable |
|
|
Is the application intended for use with a specific health risk? |
Yes |
|
How many specific unique functionalities are there that fulfill the main purpose of the application? |
5 |
|
List of the main functions of the application |
|
|
How many additional functionalities does the application have that are not necessary to fulfill the main objectives of the application? |
0 |
|
Instructions for use |
None |
|
Provider's website reference |
|
|
Price |
The diary is free for the user. As an in-app purchase, you can purchase the Premium version (headache nurse consultation function), which costs €19.99/month. |
|
Languages |
|
|
Which specialty professionals prescribe the application? |
Not applicable |
|
Are there any restrictions on using the application (age, specific health problem, etc.)? |
The user must be at least 18 years old. |
|
Which official organizations (e.g. national organizations or medical professional associations) recommend the application? |
Estonian Headache Society, Estonian Migraine and Headache Patients Association. |
|
Medical device risk class (if applicable) |
Not applicable |
|
Is the application registered as a medical device? |
No |
|
Does the European Union Medical Devices Regulation apply to the application? |
No. Manufacturer's justification: The regulation does not apply to the digital headache diary, because the application is intended only for storing user data and downloading it if necessary. |
Migrevention - Privacy and Security
|
Does the app have a privacy policy (privacy policy notice)? |
Yes |
|
Does the privacy policy contain the required information (based on the GDPR)? |
Yes |
|
Is the privacy policy accessible during registration and use of the app? |
Yes |
|
Does the app collect personal information? |
Yes |
|
Does the app collect special types of personal information (e.g., health information)? |
Yes, health information. |
|
Has the company appointed a data protection specialist? |
Yes |
|
Is the user asked for consent to process the data? |
Yes |
|
Is the consent given voluntary, specific, informed and unambiguous? |
Yes |
|
Can the user withdraw their consent? |
Yes |
|
Does the user have access to personal information? |
Yes |
|
Can the user change personal information? |
Yes |
|
Can the user request (or download) personal information? |
Yes, the user can send one’s headache diary to e-mail. |
|
Can the user delete personal information? |
Yes |
|
Does the app share personal information? |
Yes |
|
Does personal information move abroad? |
Yes, to store data on the Digital Ocean server within the borders of the European Union. |
|
Does the application use cookies or similar technologies? |
Yes |
|
Does personal information move abroad? |
Yes, to store data on the Digital Ocean server within the borders of the European Union. |
|
Are minors using the app? |
No |
|
Does the app require adult (guardian) consent? |
No |
|
Does the app check the user's age? |
No |
|
Does the application use opt-in marketing opportunities? |
No |
|
Does the application provide information on the implementation of correct technical and organizational measures to protect personal data? |
Yes |
|
Is user authentication secure (password policy applied, 2-step authentication, PIN)? |
Yes |
|
Does the application use Mobile-ID, Smart-ID, or a similar eID authentication solution? |
No |
|
Does the app's privacy policy specify where the data will be stored? |
Yes |
|
Does the app's privacy policy state how long data will be stored? |
Yes |
|
Will the user be deleted after a certain period of inactivity? |
Yes |
|
Have there been any reports of data breaches? |
No |
Migrevention - Level of user and clinical trials
|
Are medical institutions and / or governmental organizations involved in setting up the application? |
Yes, Tartu University Clinic. |
|
Are healthcare professionals involved in the content creation of the application? |
Yes |
|
Are people/patients in the application content involved? |
Yes |
|
What are the clinical goals of the application? |
|
|
Is the demonstration that the application can meet such clinical goals based on previous (international) research on similar technology/solutions? |
Yes |
|
What research has been done and what do the results show? |
Previous studies have shown that smartphone applications in the form of electronic headache diaries are reliable methods of data collection, patients prefer electronic diaries over paper diaries, and electronic diaries are more discreet in the workplace. Filling in a digital headache diary allows for more accurate documentation of data and reduction of recall bias, and digital interventions allow for prompting of use through automatic reminders. Data collection in digital diaries can increase patient compliance, minimize secondary data collection errors, reduce administrative burden, and save costs. For many chronic diseases, including headaches, patient counseling and education is important, which helps improve patients' quality of life, increases treatment efficiency, and reduces costs to the health care system. Headache nurse counseling is one part of non-drug treatment according to today's treatment standards. In her master's thesis, Kristi Tamela has highlighted the impact of headache nurse counseling on the quality of life of people with headaches in the Estonian context and emphasizes the significant improvement in the quality of life of patients receiving counseling within six months. There is little research on the use of virtual nurses or virtual agents in medical interventions. One example is the use of a virtual avatar in the treatment of depression (Help4Mood program). There has been some research into the possibilities that a virtual nurse could perform better or worse than a conventional nurse's reception. It is pointed out that collecting patient data, giving instructions to patients and motivating the patient for treatment, improving treatment compliance could be the possibilities of a virtual nurse. The limitations of the virtual nurse are the inability to make decisions, for example, regarding the diagnosis or to cooperate with other team members. Although a virtual nurse is not able to fulfill all the roles of a conventional nurse, it is likely to be able to fill the gap between the possibilities and demands of healthcare (long treatment queues, multiple times the demand compared to the number of specialist headache nurses that are available, cost-effectiveness).
|
|
Are measurements made and, if so, what measurements are made in the application? |
The app does not take measurements. |
|
Has the accuracy of application measurements been evaluated in the past for solutions with similar technology? |
Not applicable |
|
What specific studies have been carried out on the accuracy of the measurements and what do the results show? |
Not applicable |
|
Is the evidence that the application can meet its clinical objectives based on research with that particular application? |
Yes. The Migrevention headache diary and nurse consultation service was included in a 2020 study where patients could use the headache diary or the nurse consultation service as one of the functionalities. |
|
What kind of research has been done with a specific application and what do the results show? |
A study was conducted where, among other things, headache frequency, quality of life and satisfaction were evaluated (the study received the approval of the ethics committee of the University of Tartu - no. 315T-17). A headache diary and nurse advice service were among the functionalities. Digitally delivered care was found to be as good or slightly better than traditional care. It was possible to help 10 times more patients at the same time and with the same number of specialists by providing digital treatment. |
|
Have any research articles been published on the clinical relevance/evidence of a specific application? |
No, the study is still being published. |
|
In which journals have articles about a specific application been published? |
Not applicable |
|
Does the application manufacturer periodically ask users for feedback based on a methodology (specify frequency and methodology)? |
Yes (one-time survey, feedback questionnaire created by the manufacturer). |
|
To what level of evidence does the application belong to the NICE Digital Health Technology Assessment Framework? |
Level 2, the application classifies under simple monitoring functionality. |
|
Does the application have a minimum or best practice standard of proof of effectiveness? |
Minimum standard of proof of effectiveness. |
|
Does the application meet this applicable level and standard of proof of effectiveness? |
Yes |
Migrevention - Interoperability and data quality
|
Does the application exchange data with other applications? |
No |
|
Does the application use standard terminology (ICD-10, SNOMED, HPO, ATC, etc.) when collecting data? |
No |
|
Does the application use standard clinical data models (e.g. OpenEHR, OpenClinica)? |
No |
|
Does the application provide an API for exchanging data with other applications? |
No |
|
Is the application compatible with other wireless devices (Bluetooth, NFC, GSM) to exchange data? |
No |
|
Is the application able to exchange messages in HL7 V2, 3 and FHIR format? |
No |
|
Does the application support automatic data entry via direct import (CSV, CDISC, Json, xml), API, SFTP, IHE, HL7? |
No |
|
Has the application implemented any standard or IHE workflows? |
No |
|
Does the application work with CDA, C-CDA documents (XML based HL7)? |
No |
|
Can the data be exported from the application in a standard format (XML, CSV, CDISC, etc.)? |
Yes. You can export from Qure Server (CSV, XLSX). |
|
Does the application use a GDPR compliant cloud service to store data? |
Yes |
|
Does the application have an interoperability and interface strategy for interfacing with other health information systems (including national systems)? |
No |
Kõneravi.ee - General Info
|
Evaluation time |
October 2021 |
|
What operating systems or platforms does the app support? |
The application is accessible through web browsers. |
|
Does the application need an external medical or non-medical device to work? |
No |
|
Location of data storage - country, server provider |
Estonia, server provider IT Box OÜ |
|
What is the purpose of the application? |
The purpose of the application is to help develop and guide the child's speech through interactive speech therapy exercises. |
|
The target audience of the application |
The application has two target groups:
|
|
Is the application intended for use with a specific health problem? |
No |
|
List of health conditions for which it suits for |
|
|
Is the application intended for use with a specific health risk? |
No |
|
How many specific unique functionalities are there that fulfill the main purpose of the application? |
4 |
|
List of the main functions of the application |
Features aimed at children:
|
|
How many additional functionalities does the application have that are not necessary to fulfill the main objectives of the application? |
1, freely available instructional material for evaluating a child's speech. |
|
Instructions for use |
|
|
Provider's website reference |
|
|
Price |
|
|
Language |
Estonian |
|
Which specialty professionals prescribe the application? |
Not applicable |
|
Are there any restrictions on using the application (age, specific health problem, etc.)? |
The application is not recommended for children under 3 years of age. |
|
Which official organizations (e.g. national organizations or medical professional associations) recommend the application? |
Eesti Kogelejate Ühing, Tartu Ülikooli Kliinikum, Valga Haigla, logopeedid, Afaasialiit, Insult.ee, EdTech |
|
Medical device risk class (if applicable) |
Not applicable |
|
Is the application registered as a medical device? |
- |
|
Does the European Union Medical Devices Regulation apply to the application? |
Ei. Tootja põhjendus: rakendus ei mõjuta kasutaja diagnoosi ega paku ravisoovitusi. |
Kõneravi.ee - privacy and security
|
Does the app have a privacy policy (privacy policy notice)? |
Yes |
|
Does the privacy policy contain the required information (based on the GDPR)? |
Yes |
|
Is the privacy policy accessible during registration and use of the app? |
Yes |
|
Does the app collect personal information? |
Yes |
|
Does the app collect special types of personal information (e.g., health information)? |
No |
|
Has the company appointed a data protection specialist? |
No |
|
Is the user asked for consent to process the data? |
Yes |
|
Is the consent given voluntary, specific, informed and unambiguous? |
Yes |
|
Can the user withdraw their consent? |
Yes |
|
Does the user have access to personal information? |
Yes |
|
Can the user change personal information? |
Yes |
|
Can the user request (or download) personal information? |
Yes |
|
Can the user delete personal information? |
Yes, contacting the service provider by e-mail. |
|
Does the app share personal information? |
No |
|
Does personal information move abroad? |
No |
|
Does the application use cookies or similar technologies? |
Yes |
|
Does the application use cookies or similar technologies correctly? |
Yes |
|
Are minors using the app? |
Yes |
|
Does the app require adult (guardian) consent? |
No |
|
Does the app check the user's age? |
No |
|
Does the application use opt-in marketing opportunities? |
No |
|
Does the application provide information on the implementation of correct technical and organizational measures to protect personal data? |
Yes |
|
Is user authentication secure (password policy applied, 2-step authentication, PIN)? |
Yes |
|
Does the application use Mobile-ID, Smart-ID, or a similar eID authentication solution? |
No |
|
Does the app's privacy policy specify where the data will be stored? |
Yes |
|
Does the app's privacy policy state how long data will be stored? |
Yes |
|
Will the user be deleted after a certain period of inactivity? |
No. The user can activate the same account in the future and continue the exercises from the same place. |
|
Have there been any reports of data breaches? |
No |
Kõneravi.ee - level of user and clinical trials
|
Are medical institutions and / or governmental organizations involved in setting up the application? |
Yes, Estonian Stutterers' Association, Children's Hospital, Speech Therapy Center |
|
Are healthcare professionals involved in the content creation of the application? |
Yes, clinical speech therapists. |
|
Are people/patients in the application content involved? |
The people/patients of the target group have been involved through the Estonian Stutterers' Association, and speech therapists have received ongoing feedback about the content of the application during therapies. |
|
What are the clinical goals of the application? |
The application has an educational purpose - to support the development of a child's speech and to be a suitable support and work tool for a specialist dealing with the development of a child's speech. |
|
Is the demonstration that the application can meet such clinical goals based on previous (international) research on similar technology/solutions? |
Yes |
|
What research has been done and what do the results show? |
Previous studies have shown that providing teletherapy can offer a unique advantage to speech therapists who must provide short, intensive sessions within a day at school. With the support of teletherapy, children achieve comparable results to conventional therapy, and with the remote service model, students have demonstrated better fulfillment of the goals of their individual exercise plan than with classical contact therapy. With the support of training, telepractice methods have been enthusiastically and positively received by stakeholders. |
|
Are measurements made and, if so, what measurements are made in the application? |
- |
|
Has the accuracy of application measurements been evaluated in the past for solutions with similar technology? |
- |
|
What specific studies have been carried out on the accuracy of the measurements and what do the results show? |
- |
|
Is the evidence that the application can meet its clinical objectives based on research with that particular application? |
No |
|
What kind of research has been done with a specific application and what do the results show? |
- |
|
Have any research articles been published on the clinical relevance/evidence of a specific application? |
No |
|
In which journals have articles about a specific application been published? |
- |
|
Does the application manufacturer periodically ask users for feedback based on a methodology (specify frequency and methodology)? |
Anonymous user satisfaction online surveys (via Google Forms) 1 time per year. |
|
To what level of evidence does the application belong to the NICE Digital Health Technology Assessment Framework? |
Not applicable |
|
Does the application have a minimum or best practice standard of proof of effectiveness? |
- |
|
Does the application meet this applicable level and standard of proof of effectiveness? |
- |
Kõneravi.ee - interoperability and data quality
|
Does the application exchange data with other applications? |
No |
|
Does the application use standard terminology (ICD-10, SNOMED, HPO, ATC, etc.) when collecting data? |
No |
|
Does the application use standard clinical data models (e.g. OpenEHR, OpenClinica)? |
No |
|
Does the application provide an API for exchanging data with other applications? |
Yes |
|
Is the application compatible with other wireless devices (Bluetooth, NFC, GSM) to exchange data? |
No |
|
Is the application able to exchange messages in HL7 V2, 3 and FHIR format? |
No |
|
Does the application support automatic data entry via direct import (csv, CDISC, Json, xml), API, SFTP, IHE, HL7? |
Yes |
|
Has the application implemented any standard or IHE workflows? |
No |
|
Does the application work with CDA, C-CDA documents (XML based HL7)? |
No |
|
Can the data be exported from the application in a standard format (XML, CSV, CDISC, etc.)? |
Yes |
|
Does the application use a GDPR compliant cloud service to store data? |
Yes |
|
Does the application have an interoperability and interface strategy for interfacing with other health information systems (including national systems)? |
No |
Evaluation process and evaluators
Applications that come through the following process will be added to the health application evaluator website:
- The application manufacturer responds to a standardised questionnaire.
- The assessment team answers the assessment questions based on the public information provided by the manufacturer.
- The assessment team goes through different usage cases in the application.
- The security and data protection expert of the assessment team assesses compliance with data protection laws. The assessment of GDPR and security is limited to external control of compliance with legal requirements and does not include internal audit.
- The clinical expert of the assessment team assesses the level of clinical proof of the application.
- The application manufacturer can comment on the evaluation results.
- The manufacturer of the application confirms that it will send updated information from time to time or when changes have been made to the application.
- The evaluation team decides whether to publish the application and the evaluation results.
The time of the evaluation shall be indicated for each application.
If the manufacturer has not confirmed that the information is true within a certain period, or if he has not updated certain information, a note will be made with the corresponding application.
NB! The solutions presented on this page are largely based on publicly available information, supplemented by information provided by the manufacturer / developer. Thus, the website reflects a description of the situation during a specific period. Therefore, TalTech cannot guarantee the 100% accuracy of this information, as changes and additions may be made to the applications that cannot be foreseen and reflected promptly.
Evaluators of the health applications
Level of user and clinical trials
Kadi Lubi (PhD) is the head of the e-health curriculum at TalTech. She has an academic background in pharmacy and social sciences. Her doctoral dissertation has also been defended in the social sciences, which dealt with the information-seeking activity of people with a chronic illness, its development and change during the illness. Kadi has 15 years of experience in the pharmaceutical industry in both original and generic medicines and various functions.
Kadi's research interests are related to health communication and the sociological aspects of health and disease, and much of the research has focused on why people do not make new and / or necessary health-related changes. Her work as the Vice-Chairman of the Ethics Committee for Human Research at The National Institute for Health Development (NIHD) has provided additional knowledge and competencies in aspects related to research and research ethics, including the protection of personal data. Kadi has published several research articles and collection chapters.
Riina Hallik (MSc, MMS, RN) started her education with a bachelor's degree in economics, specializing in management, and continued cum laude with a master's degree in health technology (current e-health curriculum) at TalTech. In addition, Riina graduated from the Karolinska Institute in Sweden with a master's degree in health economics, politics and management and a nursing degree from Tallinn Health Care College. Riina has ten years of experience working as an analyst and researcher in the healthcare sector, focusing mainly on various EU-funded projects in the fields of telemedicine, e-health, remote care and home monitoring.
Riina is a lecturer and developer of the subject “Assessment in Healthcare” at TalTech and a methodological advisor for the final theses of the e-health curriculum. Riina has supervised eight master's students and co-authored three 1.1. classification publication and a number of conference presentations. In addition to the university, work experience in the field of health care can be mentioned Pealinna Perearstid OÜ, Tallinn Health Care College, Tallinn Children's Hospital Foundation, PRAXIS Policy Research Center and co-founded start-ups: Dermtest, Likemed and ePrescription.
Depending on the specific application, additional external experts will be involved.
Privacy and security
Olga Vovk (MSc) is a doctoral student at Taltech and an expert in privacy and security, specializing in healthcare. Olga is a lecturer in the field of privacy and security at Taltech, Tallinn Health Care College, EIT Digital Summer School, EIT Health Alumni and Women events. She holds a professional degree in law and a master's degree in e-health. Olga has more than five years of experience as a project manager in leading international companies and more than two years as a co-founder of a start-up company. She specializes in legal and technical issues related to the General Data Protection Regulation (GDPR) and assists healthcare organizations and application developers in verifying compliance. In addition, Olga conducts training on privacy and cyber security and provides legal and technical support to companies.
Janek Metsallik (MSc) is a certified IT architecture specialist specializing in the design and integration of enterprise information systems. Janek Metsallik is completing his doctoral studies at Taltech and is a lecturer for health technology students and IT specialists focusing on information systems analysis, architecture and design methods. Mr. Metsallik has extensive experience in designing architectures for management, banking, healthcare, taxation, customs, administrative information, sports betting, software development, customer relations and industrial automation. He has been the chief architect of the Estonian National E-Health Information System (EHIS) - the first fully operational health information system for healthcare professionals and citizens nationwide. In addition, he has contributed to the development of EHIS interface specifications and central system change and problem management processes. Janek Metsallik has extensive knowledge of digital healthcare services, their standardization, implementation and integration.
Depending on the specific application, additional external experts will be involved.
Interoperability, interface and standards
Peeter Ross (MD, PhD) is the head of the e-medicine center at Taltech and co-founder of the E-health (formerly Healthcare Technology) master's program. He is one of the founders of Estonian e-health and works as a radiologist at the East Tallinn Central Hospital. Peeter Ross is the founder and CEO of the radiology and e-health company SMIS International Ltd and an e-health consultant at the World Bank and the Asian Development Bank. Dr. Ross has previously worked as the Director of Research and Development at East Tallinn Central Hospital, was a member of the Supervisory Board of the Estonian E-Health Foundation and Estonian Health Insurance Fund. Peeter is responsible for researching and teaching healthcare innovation and the introduction of new digital solutions. He has participated in the development and implementation of a nationwide health information system and has been involved in several e-health projects funded by the European Union.
Hany Mina (PharmaB, MSc.) is a certified Project Manager and qualified Pharmacist in Ireland. He has a master's degree in e-health and health technology assessment. Hany specializes in the technical side of healthcare. Mainly how different technologies enable the exchange of healthcare data within organizations, between different organizations and across borders, so that the data reaches the patient. He has contributed to numerous projects focusing specifically on data transfer between health information systems using APIs, IHE workflows, DICOM and HL7 standards, and SFTP.
Depending on the specific application, additional external experts will be involved.
The original creators of the health application evaluator
The evaluator of the health application was launched by the lecturers, experts, analysts and doctoral students of the TalTech e-health curriculum and the e-medicine center: Priit Kruus, Helen Staak, Maarja-Liis Elland, Olga Vovk, Hany Mina, Janek Metsallik, Maarja Kuslapuu, Peeter Ross.
Contact us
If you want to cooperate, help with evaluation, get a consultation about your application or add your application to the evaluator, contact carmen.mae@taltech.ee!
Disclaimer
Tallinn University of Technology (hereinafter TalTech) does not take responsibility for the content of the evaluated applications. Application developers are responsible for the compliance and suitability of the listed applications and content. TalTech is not the creator, owner, editor, administrator, or provider of the applications listed in the Application List (and any information and content contained therein).
TalTech has established rules for reviewing health applications, which are described in detail on this page. This does not mean that TalTech has identified all aspects that are considered or reviewed and checked all aspects of all versions of the applications (including all updates) or that they are suitable for use by all types of users.
The developer of the listed applications is solely responsible for the content, consulting, production and maintenance (including any necessary updates) of the applications and digital tools.
Users must use their skills and independent judgment when using health applications. TalTech shall not be liable for any damages caused by (but not limited to) any information, advice, instructions, content or scripts caused by errors, loss of data, inaccuracies or omissions. TalTech shall not be liable for any personal data entered into the applications. TalTech is not responsible for any advice, services or products you receive through the applications and digital tools listed on the TalTech website.
The list of health applications on the TalTech website is for reference purposes only. It should be considered as a guide or supplement to the advice provided to users by qualified healthcare professionals. Users should contact the developer of the application immediately if there are any issues or deficiencies in the content of the applications.
This project is funded by EU European Regional Development fund through Connected Health Cluster.