A Review of Data Quality Assessment in Emergency Medical Services

Introduction: Data quality is an important issue in emergency medicine. The unique characteristics of emergency care services, such as high turn-over and the speed of work may increase the possibility of making errors in the related settings. Therefore, regular data quality assessment is necessary to avoid the consequences of low quality data. This study aimed to identify the main dimensions of data quality which had been assessed, the assessment approaches, and generally, the status of data quality in the emergency medical services. Methods: The review was conducted in 2016. Related articles were identified by searching databases, including Scopus, Science Direct, PubMed and Web of Science. All of the review and research papers related to data quality assessment in the emergency care services and published between 2000 and 2015 (n=34) were included in the study. Results: The findings showed that the five dimensions of data quality; namely, data completeness, accuracy, consistency, accessibility, and timeliness had been investigated in the field of emergency medical services. Regarding the assessment methods, quantitative research methods were used more than the qualitative or the mixed methods. Overall, the results of these studies showed that data completeness and data accuracy requires more attention to be improved. Conclusion: In the future studies, choosing a clear and a consistent definition of data quality is required. Moreover, the use of qualitative research methods or the mixed methods is suggested, as data users’ perspectives can provide a broader picture of the reasons for poor quality data.


INTRODUCTION
The use of data is the basis for conducting a variety of activities and making decisions on operational and strategic levels in different organizations [1]. While high quality data is essential for the success of organizations, the drawbacks The practice of physicians in the emergency departments is accompanied with multiple pauses that are related to conducting various duties and simultaneous communication between healthcare providers [11]. From the emergency medicine perspective, a pause in the process is an indicator for the potential error in the cycle of data documentation. Therefore, the regular assessment of data quality is crucial to avoid the adverse effects of using low quality data in the process of decision making [12,13]. Moreover, according to Obermeyer et al., compared to high-income countries, in which clinical and emergency care have dramatically improved during the last decades, in developing countries there is still a lack of data from the field of emergency medicine that has made it difficult to make new investments in emergency care. Hence, any improvement to emergency care in these countries will require advances in data collection methods and data quality. Tanzania and South Africa are among the countries which have started working on emergency data quality [14].
To assess data quality, it is pivotal to have a common grasp of its concept. Several investigators have pointed out a lack of a common definition for the data quality in healthcare. However, data quality is defined frequently as fitness for use, i.e. the extent the data can be used for its user purposes [15]. It is also suggested that data is suitable for application when it can provide he required information for its users [16,17].
Data quality assessment has a great impact on data quality development. During the assessment process, the main reasons for data deficiency and errors and the necessary control processes will be identified which eventually may lead to improve the quality of data [18]. Since data quality is a multidimensional concept, it is important to pay more attention to the methods of data quality assessment [19,20]. According to the literature data quality assessment can be performed using quantitative and qualitative methods [4,21]. Objective and subjective assessment methods are other approaches to assess the quality of data. The objective approach focuses on the evaluation of the stored data by calculating specific quality criteria [21,22], the subjective approach, reflects the requirements of the beneficiaries [data collectors, administrators, and data users] and their understanding and expectations of using the data. This approach is a supportive method for the objective assessment of data quality and investigates data quality and its dimensions in a real world setting [23].
In 2014, Chen et al. conducted a systematic review of data quality assessment methods in public health information systems. The researchers noted that public health information systems whether paper or computer-based, are the repositories of public health data and their data are frequently used to monitor public health outcomes [21]. However, as noted before, the context of emergency medical services is different and the quality of data is pivotal at the point of care. Even a small delay in retrieving health care information makes that information essentially useless for emergency care [24]. While there are a wide variety of electronic systems with different functionalities available for use in emergency medical services, these variations may affect physicians' decision-making, clinicians' workflow, communication, and the overall quality of care and patient safety [8]. Moreover, the study conducted by Chen et al. was general and the concept of data quality in the context of emergency medical services was not discovered in particular [21]. In the current study, the researchers aimed to identify the assessment approaches, the main dimensions of data quality and the status of data quality in the emergency medical services.

METHODS
This review was conducted in 2016. Related articles were identified by searching databases, such as Scopus, Science Direct, PubMed and Web of Science using Boolean operators (AND/OR) with the following entry terms: quality, data accuracy, patient records, medical records, assessment, evaluation, health information, health information management, emergency medical services, emergency care, emergency medicine, and pre-hospital emergency care. An example of the search strategy in PubMed was as follows: All of the review and research papers related to data quality assessment in the emergency medical services published between 2000 and 2015 were included in the study. The non-English papers, books and editorial letters were excluded. Initially, 1038 papers were retrieved of which 420 duplicated papers were removed by using EndNote X7 software. The remaining 618 papers were screened based on titles, abstracts, and the relevance to the study subject. At this stage, 56 relevant full text papers were identified. Having read the papers, finally, 34 articles were selected to be reviewed (Fig. 1). Two researchers (MM) and (HA) involved in the process of paper selection and the results were discussed with the third researcher (DKZ). For each paper, information about the author, publication year, study objectives, data quality dimensions, methodology, and a summary of findings were extracted.

RESULTS
Among 34 selected papers, 32 articles were published in the journals and two articles were presented in conferences. Nineteen articles were published between 2010 and 2015. Published papers were from the following countries: USA (21 papers), Australia (eight papers), England (two papers), Norway (one paper), Saudi Arabia (one paper), Canada (one paper). In the following sections, more details about the selected papers are presented.

Research Objectives
The primary objective of all articles was to assess the quality of data; however, some of them set some secondary objectives, too. For instance, some papers investigated the quality of data and its impact on preparing guidelines to support clinical [25] and managerial judgements [26], the concordance of trauma registry and hospital records [27], functional parameters in the emergency care services [28], and designing a dashboard for the emergency department [29] ( Table 1). In four studies, data quality assessment was conducted based on the work processes and the flow of information [29 -31] and some others examined the effect of interventions on the quality of data. Three studies were performed to introduce new techniques for assessing the quality of data. These techniques included simulation [phenomenology] [28], and narrative texts analysis [32]. A number of papers assess the quality of data to determine information gaps. In these papers, completeness, accessibility of information, and documentation errors were the main areas to be assessed [33 -39].

Considine et al. (2006) Australia
To examine the effect of written ED nursing practice standards augmented by an inservice education programme on the documentation of the initial nursing assessment Written ED nursing practice standards improved emergency nurses' documentation of the initial nursing assessment except oxygen saturation, heart rate or blood pressure.

Travers et al. (2006) America
To measure the time of availability of participating EDs' diagnosis data in a statebased syndromic surveillance system.

Availability, Timeliness
Quantitative/ Prospective study A majority of the ED visits transmitted to the state surveillance system did not have a diagnosis until more than a week after the visit. Reasons for the lack of timely transmission of diagnoses included coding problems, logistical issues and the lack of IT personnel at smaller hospitals.
Porter et al. The incomplete concordance of diagnoses of the selected chronic diseases generated via different modules of the same information system raises doubts about the reliability of data and information quality collected, stored and used by the EMR.

Gao et al. (2012) Australia
To adopt a process-oriented approach to understand how data quality issues emerged through the ED data collection and reporting processes.

Completeness, Consistency Timeliness Accuracy Integrity Conformity
Qualitative/ Literature review The development of the ED process maps is central to a comprehensive data quality assessment. These process maps will not only serve as a roadmap of where to look for data quality problems, but would also allow for possible optimization of information resources

Ward et al. (2013) America
To assess operational data quality in an emergency department (ED) immediately before and after an EHR implementation Accuracy Timeliness Quantitative/ Crosssectional study Using electronic timestamps for operational assessment and decision making following implementation should recognize the magnitude and compounding of errors when computing service times.
Hu et al. In two articles, a data quality assessment model was presented and discussed. According to this model, data quality assessment can be conducted with respect to three dimensions which are technical, organizational and individual [29,31]. Although most papers had assessed the quality of data in the emergency departments, a few papers had focused on the quality of data in pre-hospital emergency care services [33, 40 -43].

Data Quality Dimensions
Having reviewed the literature it was revealed that among different dimensions of data quality, five dimensions have received more attention in healthcare. These dimensions were accuracy, timeliness, completeness, relevancy, and consistency [44]. The definitions of these dimensions are presented in Table 2. Table 2. -Definition of dimensions of data quality.

Accuracy
The extent to which data is correct and reliable. (1,2) Completeness The extent to which data is not missing and in of sufficient breadth and depth for the task at hand. (1,2) Timeliness The extent to which the data is sufficiently up to data for the task at hand. (1,2) Accessibility The extent to which data is available, or easily and quickly retrievable. (1, 2)

Consistency
Representation of data values remains the same in multiple data items in multiple locations. (1,2) Errors Omission Occasionally, correctness included completeness, due to the fact that some researchers consider missing data to be incorrect (ie, errors of omission)/missing data. (3) Commission Discrepancy or inaccurate (3) Inaccurate/error Error terms that were commonly used to describe accuracy and quality. (3) In the current study, the findings showed that in the emergency care services, the main dimensions of data quality were completeness, accuracy, consistency, accessibility, and timeliness. According to Table 3, all papers had investigated one or more dimensions of data quality. Interestingly, the results showed that the definitions of data quality dimensions were occasionally different in various studies. For instance, terms like missing, incompleteness, information gap, and omission were used to address the completeness dimension in some studies and invalid values were regarded as missing data (incompleteness) in one study [45] Similarly, the lack of common understanding among researchers and differences in definitions could be seen in other data quality dimensions, such as accuracy, consistency and accessibility.

Methods of Data Quality Assessment
The objective or quantitative methods were the most common method used to assess the quality of data. Among quantitative methods (e.g., retrospective, cross-sectional survey), retrospective and cross-sectional studies with statistical data were the most common ones. The subjective or qualitative methods (including review of publications and documentations, interviews with key informants, and field observations) were used only in four papers [12,29,31,58], and the mixed methods [quantitative and qualitative] were applied in three studies to assess the quality of emergency medical services data [37,46,47]. In these articles, the quantitative methods had been used to measure the dimensions, and the qualitative methods had been utilized to investigate users' perspectives about the quality of data or the main reasons for the information gap.

Reported Findings After Data Quality Assessment
The results of the studies could be categorized in three categories. The first category was related to assessing different dimensions of data quality and the following findings were reported by the researchers: completeness (between 30% and 100%), accuracy (between 57% and 99%), consistency (between 54% and 98%), accessibility (between 50% and 98.7%) and data errors were found in timestamps [28,48,49].
The second category was related to the studies in which an intervention was performed to improve data quality. In this category, different approaches, such as simulation (phenomenology) [28] and data linkage [40] were used to improve data quality. Some researchers examined the effect of interventions on the quality of data. The interventions included the use of a structured encounter form for documentation [50], the use of a standardized patient transfer form [35,51], a patient-centred health information technology (a mobile kiosk) [7], the use of Electronic Health Records (EHR) [48], the use of vital signs data recorder [VSDR] technology in pre-hospital care [52], electronic documentation [53], and narrative texts analysis [32]. One study applied educational interventions to develop documentations [34] and one study examined the role of pharmacists in identifying discrepancies in medication histories [54].
The third category was related to the papers which focused on the data quality issues in the emergency medical services. These issues included information gaps and related causes, data quality and dashboard development projects, data quality issues emerged during data collection and reporting processes, data quality issues from the emergency department staff perspectives.

DISCUSSION
Emergency medicine is an information intensive speciality in which timely access to accurate patient information is of high importance [55]. Therefore, data quality assessment is necessary in this field. In the current study, papers related to data quality assessment in the field of emergency medical services were reviewed and the results showed that there were different and sometimes ambiguous definitions for data quality dimensions and related characteristics. The results are in line with the findings of other studies conducted by Chen et al. [21] and Michnik and Lo [56]. The lack of common understanding among researchers and differences in definitions of data quality dimensions have led to conducting various studies and obtaining different results. For example, in some papers, data availability was regarded as equivalent to completeness [25,45,49], accessibility [57], and missing data [58]. In one study, conflict was considered equivalent to inconsistency [58], while in another study it was regarded as data omission [54]. It seems that the definitions and characteristics of the data were different based on the intuition, past experiences, and assessment purposes [21]. Therefore, the use of ontology-based or standard definitions in the future studies can help to be able to compare the research methods and the results [21].
According to the findings, the most common data quality dimensions assessed in the emergency medical services were completeness, accuracy, and consistency followed by accessibility and timeliness. In EMS, data completeness and data accuracy are important for improving the quality of care and for making right decisions. Similarly, data consistency is important to show the agreement between two or more sources of information and the possibility of data linkage between the primary and secondary care systems [59]. The linkage between data sources, in turn, can help to promote the quality of communication and information flow across the healthcare organizations [41]. The characteristics of emergency medical services also necessitate information being accessible for treating patients in a timely manner [55]. It seems that these data quality dimensions can be considered as a data quality framework for emergency medical services (Fig. 2). According to Almutiry et al., the existing data quality frameworks are based on literature review, industrial experiences or intuitive understanding and the definition of a dimension may vary from one framework to another [6]. Moreover, the frameworks are too generic to adopt, may not reflect the nature of a domain, or may have some irrelative attributes. Therefore, such a data quality framework for emergency medical services can help to cover the main quality dimensions which are more relevant to the nature of emergency medical services. The proposed data quality framework is more comprehensive than the framework suggested by Chen et al. in which the main data quality dimensions were completeness, accuracy and timeliness [21]. Fig. (2). Data quality framework for emergency medical services.
Regarding the assessment methods, the findings demonstrated that there was no unique method to assess the quality of emergency medical services data and a variety of research methods was used by the researchers. This is consistent with the findings reported by Chen et al. [21]. However, the findings of the current study showed that the use of Data quality framework for Emergency Medical services Completeness Accuracy Timeliness Accessibility Consistency quantitative research methods had the highest frequency. In these studies, quantitative methods were mainly used to assess data accuracy, data consistency, data completeness, and data availability which are in line with the reported findings in other studies [53]. According to the literature, in order to assess different aspects of data quality, objective and subjective methods can be used. However, quality is a very subjective concept and depends on many other factors. As Pipino et al. noted assessing data quality requires the awareness of the fundamental principles and should be assessed by using both subjective and objective data quality metrics [18]. The use of mixed methods [quantitative and qualitative] and different information resources, such as files, organizational documents, and users' perspectives has also been suggested in other studies [19,21,60].
In fact, performing subjective and objective data quality assessments helps to compare the results of the assessments, to identify discrepancies, and to determine root causes of errors for determining and taking necessary actions for improvement [61]. In fact, the subjective assessment is a supplement to the objective assessment and is useful for designing effective strategies for improving data quality [15,21,22,62].
The results also revealed that the quality of data could be affected by a number of technical, organizational, and individual factors [31,48,53]. Since, these factors may also influence data accessibility, they should be taken into account when designing information systems [55]. Overall, different studies showed that technical factors, such as ED information system user-interface and data extracting methods may influence the timiliness, accuracy, and completeness of data, while data consistency and data integrity are mainly affected by the organizational factors (e.g., waiting time) and individual factors (e.g., capturing all required data from patients] [29,31]. The results showed to improve the quality of data in emergency medical services, different interventions can be made; however, most studies have used different types of health information technologies and structured forms. This shows that technical factors and information systems can be replaced with the traditional paper-based records; however, they need to be designed and implemented carefully as the quality of data can be easily affected by the quality of these system [55].
Overall, it can be concluded that data quality in emergency medical services deserves more attention, scientific research and investment. The data quality framework propsed in the current study can be used in different settings of emergnecy medical services to compare the quality of data and to understand how to improve the current situations. Moreover, it is a simple and clear framework which can be easily used by the health care practitioners in the field to examine the quality of data in their own settings. The results of this type of research can also help to develop better quality assurance strategies, to support health care delivery, and to improve patient care.

CONCLUSION
Emergency medical services require high quality data to support healthcare delivery and decision making and data quality assessment is of great importance in this field. The results of the current study showed that due to the diversity of definitions and terminology used to assess the dimensions, characteristics, and attributes of data quality, comparing findings reported in different studies was difficult. Therefore, in the future studies, more attention should be paid to choose a clear and a consistent definition of data quality. In this study, a data quality framework was proposed to be used in the context of emergency medical services. The main data quality dimensions were completeness, accuracy, consistency, accessibility and timeliness. This framework shows that data consistency and data accessibility are two important quality dimensions for emergency medical services in addition to data completeness, accuracy and timeliness.
Regarding the data quality assessment methods, the use of qualitative research methods or the mixed methods is suggested, because data users' perspectives can provide a broader picture of data quality and related issues. Finally, emergency data quality needs to be assessed at different organizational levels using different resources. This approach can help to identify quality issues and the most appropriate interventions to improve data quality.

CONSENT FOR PUBLICATION
Not applicable.

CONFLICT OF INTEREST
The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGMENTS
This study was funded and supported by Iran University of Medical Sciences, Tehran, Iran (IUMS/SHMIS_93/10).