RESEARCH ARTICLE
Standards for Scalable Clinical Decision Support: Need, Current and Emerging Standards, Gaps, and Proposal for Progress
Kensaku Kawamoto*, 1, Guilherme Del Fiol1, David F. Lobach1, Robert A Jenders2
Article Information
Identifiers and Pagination:
Year: 2010Volume: 4
First Page: 235
Last Page: 244
Publisher Id: TOMINFOJ-4-235
DOI: 10.2174/1874431101004010235
Article History:
Received Date: 1/7/2009Revision Received Date: 17/6/2010
Acceptance Date: 6/8/2010
Electronic publication date: 14/12/2010
Collection year: 2010
open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.
Abstract
Despite their potential to significantly improve health care, advanced clinical decision support (CDS) capabilities are not widely available in the clinical setting. An important reason for this limited availability of CDS capabilities is the application-specific and institution-specific nature of most current CDS implementations. Thus, a critical need for enabling CDS capabilities on a much larger scale is the development and adoption of standards that enable current and emerging CDS resources to be more effectively leveraged across multiple applications and care settings. Standards required for such effective scaling of CDS include (i) standard terminologies and information models to represent and communicate about health care data; (ii) standard approaches to representing clinical knowledge in both human-readable and machine-executable formats; and (iii) standard approaches for leveraging these knowledge resources to provide CDS capabilities across various applications and care settings. A number of standards do exist or are under development to meet these needs. However, many gaps and challenges remain, including the excessive complexity of many standards; the limited availability of easily accessible knowledge resources implemented using standard approaches; and the lack of tooling and other practical resources to enable the efficient adoption of existing standards. Thus, the future development and widespread adoption of current CDS standards will depend critically on the availability of tooling, knowledge bases, and other resources that make the adoption of CDS standards not only the right approach to take, but the cost-effective path to follow given the alternative of using a traditional, ad hoc approach to implementing CDS.
1. INTRODUCTION
An important rationale for the adoption of electronic health records and other health information systems is their capability to enable clinical decision support (CDS), which entails the act of providing clinicians, patients and other health care stakeholders with pertinent knowledge and/or person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care [1]. Indeed, a systematic review of computer-based, clinician-directed CDS systems found that over 90% of the systems significantly improved clinical care in randomized controlled trials, provided that the CDS was made available automatically as a part of clinician workflow, offered an actionable recommendation, and was delivered at the time and location of clinical decision making [2]. As an example of the potential magnitude of CDS effectiveness, a time series study conducted at Brigham and Women’s Hospital in Boston found that a CDS-enabled computerized provider order entry system reduced the incidence of serious medication errors by 86%, with increasing benefits resulting from the provision of more advanced CDS capabilities [3].
Despite the potential for CDS to significantly improve health and health care, the reality remains that most clinical care continues to be provided with only minimal CDS support, if at all [1,4]. While there are many reasons for this limited adoption of CDS capabilities, one important reason is the predominant use of non-standard approaches to implementing CDS that are oftentimes specific to a given CDS application and implementation setting [1,5,6]. As a result, CDS capabilities developed at one institution oftentimes cannot be easily transferred to other health care institutions, or even to other types of CDS applications within the same institution [1,7].
One promising approach to enabling the widespread deployment of CDS capabilities is the centralized management of machine-executable knowledge resources, which are then leveraged across multiple care settings by CDS engines interfaced with various health information systems [1,8-11]. However, this approach to CDS implementation is oftentimes hampered by the significant heterogeneity that often exists across institutions with regard to patient data representation, knowledge representation, and approaches to leveraging knowledge resources to provide CDS [1,5]. Thus, the widespread availability of robust CDS capabilities is critically dependent on the development and adoption of standards that encompass these facets of CDS delivery [1,12,13]. Accordingly, we provide here an overview of the types of standards required for implementing CDS in a scalable manner; current and emerging standards that address these needs; gaps and challenges that remain; and a proposal for moving forward. The assessment provided in this manuscript is based on the authors’ experience co-chairing the Health Level 7 (HL7) Clinical Decision Support and Arden Syntax Work Groups (RAJ, KK, and GDF); leading the development of HL7 standards related to CDS (KK, GDF, and RAJ); co-developing an implementers’ guide to CDS published by the Healthcare Information and Management Systems Society [14] (RAJ); and leading and/or participating in the implementation of various operational CDS applications, including for Duke University (KK, GDF, and DFL); Columbia University (RAJ); the Cedars-Sinai Medical Center (RAJ); and Intermountain Healthcare (GDF).
2. STANDARDS NEEDED FOR SCALABLE CLINICAL DECISION SUPPORT
In considering the types of standards needed to scale the deployment of advanced CDS capabilities, it is useful to first consider what is needed to develop and deploy CDS. At a broad functional level, CDS implementation needs can be classified as consisting of (i) the need to communicate with various systems regarding relevant health care concepts; (ii) the need to create and represent clinical knowledge that can be used to enable automated CDS; and (iii) the need to utilize these clinical knowledge resources to deliver CDS interventions within health information systems such as electronic health record (EHR) systems and computerized provider order entry systems. In turn, various types of standards can greatly facilitate meeting these implementation needs. Depending on the needs of clinicians, developers, and knowledge engineers, the same clinical knowledge may be represented at different levels of specificity and structure. For example, the Guideline Interchange Format (GLIF) represents knowledge on three levels: a conceptual flowchart, a computable specification that unambiguously identifies concepts and logic, and an implementable specification that can be executed at a particular health care organization using its specific data repositories and CDS system [15]. Standards may come into play at all these levels. In this section and in the following section on the current standards landscape, we describe the types of standards needed to facilitate CDS implementation, the currently available standards, and the gaps and challenges that remain. Table 1 provides a summary of the key points from this analysis.
CDS Implementation Needs, Standards that Can Facilitate Implementation, and Gaps and Challenges
CDS Implementation Need and Related Standard Needs | Specific Standard Need | Current and Emerging Standards | Gaps and Challenges |
---|---|---|---|
CDS implementation need: need to communicate with other systems about relevant health care concepts Related standard needs: standards for data representation and mapping |
Standard terminologies | Unified Medical Language System [36] and component terminologies (e.g., SNOMED SNOMED [20], LOINC [37], RxNorm [38]) | Overlapping and semantically non-compatible terminologies are in concurrent use, including non-standard terminologies |
Standard information models | HL7 version 2 and version 3 information models [39] openEHR archetypes [40] ASTM International Continuity of Care Record [41] HL7-ASTM International Continuity of Care Document [42] HL7 virtual medical record (vMR) standard (under development) [43] |
Insufficiently tight binding to terminologies Too much flexibility and complexity in models designed for expressive documentation rather than for CDS Hard to implement/understand Lack of tooling Lack of standard information models on inputs and outputs for CDS Low adoption |
|
Standards for patient data expected to be available for CDS | HL7 virtual medical record (vMR) standard (under development) [43] | Different granularity and scope of data being collected | |
Standard approaches for terminology and ontology inferencing | HL7 Common Terminology Services standard [46] | Many terminologies are semantically incompatible | |
CDS implementation need: need to create and represent clinical knowledge that can be used for CDS Related standard needs: standards for knowledge representation |
Standardized representation of clinical knowledge in non-executable format suitable for translation into executable format | ASTM International Guideline Elements Model (GEM) standard [27] | Significant medical knowledge exists outside of individual clinical practice guidelines and across multiple guidelines Most knowledge continues to be produced in non-standardized formats No clear path from non-executable to executable knowledge |
Standardized representation of clinical knowledge in an executable format | Standards for representing clinical rules (HL7 Arden Syntax standard [8], HL7 GELLO standard [9]) Standards for representing knowledge documents (HL7 Structured Product Label [51] standard, HL7 Order Set draft standard [52], HL7 Health Quality Measures Format draft standard [53]) |
No widely agreed upon standard for representing clinical practice guidelines Limited tooling and support for implementation Minimal availability of compliant knowledge in most cases |
|
CDS implementation need: need to utilize clinical knowledge to deliver CDS interventions within health information systems Related standard needs: standards for leveraging knowledge resources to deliver CDS |
Standardized approaches to utilizing machine-executable clinical knowledge to generate CDS | Standards for accessing CDS capabilities through a service call (HL7 Decision Support Service draft standard [33], OMG Decision Support Service standard [34], HL7 Context-Aware Knowledge Retrieval (“Infobutton”) standard [56]) | Semantics of service payloads still undergoing standardization (e.g., through vMR project) No standard for identifying and retrieving machine-executable medical knowledge resources themselves No commonly accepted meta-data model for knowledge resources |
Standardized approaches to interacting with health information systems to deliver CDS | Standards for retrieving patient data from health information systems (HL7 Retrieve, Locate, and Update Service draft standard [58]; various information model and terminology standards) Standards for EHR functionality (HL7 EHR Functional Model [59], Certification Commission for Health Information Technology certification criteria [60]) |
Standards for EHR functionality are not defined in a semantically interoperable manner Lack of standards on EHR services (e.g., for order placement, alert delivery) Current HL7 vMR project is still in development and does not encompass EHR services Lack of use of standard business process modeling approaches in health care Need to accommodate care settings with varying degrees of health information system infrastructure (or none at all) |
A. Need for Standards for Data Representation and Mapping
Because a CDS module typically must communicate with other systems regarding health care concepts, it can be very difficult to create CDS resources that can be re-used across different conceptual representations of health care. In this light, knowledge sharing has two main forms: knowledge reuse and knowledge transfer. In knowledge reuse, CDS knowledge is employed for a different purpose or application than its original form within its original organization. For example, CDS knowledge that establishes that a patient has kidney failure may be reused within the same organization in a pharmacy system to establish a drug dose, in a radiology system with regard to iodinated contrast administration, and in a nursing documentation system to provide guidance about fluid administration. Knowledge also may be transferred across organizational boundaries for the same purpose. In both reuse and transfer, common information models are needed due to the significant challenges associated with mapping CDS knowledge to different, non-standard information models [16-19]. For example, if a CDS knowledge resource expects to have access to a problem list and laboratory data but a deployment context has neither, the CDS resource cannot be deployed in this context. Similarly, if a CDS knowledge resource expects to have access to a granular problem list encoded in the Systematized Nomenclature of Medicine (SNOMED) [20] but only has access to a more generic problem list encoded using the 9th edition of the International Classification of Diseases (ICD9) [21], the CDS resource may not be usable because the level of granularity required and supported by SNOMED is not supported by ICD9.
Several types of standards can facilitate the implementation of CDS by providing a common framework for conceptualizing health care. Such enabling standards include standard terminologies and information models for representing health care data, as well as standards for the types of patient data that are expected to be available for CDS within specified contexts such as EHR systems [22]. Moreover, CDS implementations can be aided by the availability of standard approaches for obtaining terminology and ontology inferences, such as the mapping of concepts across terminologies and the identification of concept properties and semantic relationships [23,24].
B. Need for Standards for Knowledge Representation
At the core of a CDS implementation is the representation of clinical decision logic in a detailed, machine-executable format that can be used to drive automated CDS. As an important step in this knowledge creation process, CDS knowledge engineers must first create detailed, human-readable specifications of the knowledge based on clinical practice guidelines and other sources of clinical knowledge. This step is necessary because narrative clinical guidelines often lack the detail and algorithmic specificity required for easy translation into a machine-executable format [25].
The widespread use of a standard format for representing human-readable medical knowledge intended for translation into machine-executable knowledge would facilitate the development of CDS capabilities, as CDS developers would be able to develop processes and tooling to enable large-scale, replicable, collaborative, and decentralized knowledge development processes to support the several steps that precede the conversion of knowledge into a machine-executable format. For example, significant efficiencies could be gained if national guideline task forces and other creators of the original knowledge agreed to use the same format for specifying the underlying clinical decision logic at the level of detail required for converting the knowledge into a machine-executable format. Standardization in this area could leverage relevant work conducted both within and outside of the context of standards development organizations to ease the conversion of clinical guidelines from textual descriptions to computer interpretable specifications [10,26-29].
Following the specification of medical knowledge in a detailed, human-readable form, the clinical knowledge specification must be converted into a machine-executable format that can be appropriately interpreted and utilized by a CDS system. While various approaches are available for such machine-executable knowledge representation [30,31], these different formalisms generally have formalism-specific infrastructure needs that can require substantial resources and expertise to implement. Thus, the widespread use of one or more common knowledge representation approaches could substantially facilitate the use of the encoded knowledge to provide CDS.
C. Need for Standards for Leveraging Knowledge Resources to Deliver CDS
In providing CDS, one of the most difficult challenges is leveraging machine-executable medical knowledge resources and electronic clinical data to deliver useful CDS interventions within the context of various health information systems. While this task can be manageable when interacting with a single health information system using a single type of knowledge resource, the heterogeneity of knowledge resources and health information systems makes it difficult to re-use a CDS implementation infrastructure developed for one clinical context within other applications and care settings. For example, a knowledge execution environment designed to leverage Arden Syntax Medical Logic Modules [32] cannot readily use executable medical knowledge encoded using the Guideline Interchange Format (GLIF) [10]. Similarly, a knowledge execution environment designed to work with one commercial EHR system cannot be easily interfaced with another commercial EHR system, or potentially even with the same EHR system deployed in a different health care setting, to conduct such necessary system-to-system interactions as obtaining relevant patient data from the clinical database or delivering alerts to relevant care providers. Thus, in order for knowledge resources to be usable in a scalable manner across various applications and institutions, standardized approaches are needed for how machine-executable, implementable knowledge resources are leveraged within health information systems to generate and deliver CDS interventions. For example, regardless of the underlying knowledge representation approach used, various knowledge resources could potentially be accessed through a standard system-to-system interface that is agnostic of the underlying knowledge representation formalisms in use [33-35]. Moreover, because CDS modules that leverage knowledge resources typically interact with other components of the health information system to generate and deliver the decision support intervention, there is a need for standards on how such CDS modules interact with health information systems to obtain required data and deliver needed interventions. For example, if various EHR systems provided common software service interfaces for retrieving the patient data needed for CDS and for delivering patient care recommendations to clinicians, these standard interfaces would significantly facilitate the scalable implementation of CDS using a common set of machine-executable knowledge resources.
3. CURRENT STANDARDS LANDSCAPE
For many of the areas requiring standardization, relevant standards do exist. This section provides a high-level overview of current and emerging standards in each of these areas, as well as gaps and challenges that remain. This assessment of the CDS standards landscape is provided below and is also summarized in Table 1.
A. Standard Terminologies
Various standard terminologies are already well-developed [30]. Most of these terminologies are compiled in the National Library of Medicine’s Unified Medical Language System [36], which maps concepts from each of over 100 source terminologies to unique master concepts. Notable standard terminologies included in the Unified Medical Language System include the Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) [20], the Logical Observation Identifiers Names and Codes (LOINC) [37], RxNorm [38], and the International Classification of Diseases (ICD) [21].
Currently, standard terminologies such as SNOMED CT [20], LOINC [37], and RxNorm [38] are available for CDS with relatively adequate breadth, depth, and granularity [24]. Thus, the challenge lies less with the lack of relevant standards, but more with the fact that multiple terminologies are in concurrent use, including many non-standard terminologies (e.g., vendor-specific laboratory terminologies). The concurrent use of these various terminologies is a significant challenge, as differences in the granularity of terminologies oftentimes prevent a concept from one terminology from being mapped one-to-one to an equivalent concept in another terminology. As a result, a CDS resource designed for use in one setting may not be readily usable in another setting that uses a different set of terminologies, even when similar concepts are being captured.
B. Standard Information Models
Standard information models are also available for supporting CDS in a scalable manner. Perhaps most prominent among these information models are the models used for HL7 version 2 messages, which are ASCII-encoded messages that communicate various information relevant to the care of a patient [39]. While easy to understand and widely adopted, these models have been the target of substantial criticism. In particular, HL7 version 2 information models have been criticized for their adoption of loose semantics that ultimately compromise interoperability, including the optional use of non-standard coded values and the use of “Z segments” that allows non-standard data elements to be represented in message instances. In response to a desire to improve the semantic interoperability of such messages, HL7 embarked on the creation of version 3 messaging standards in the mid-1990s that would be based on a unified, rigorously modeled set of information models all derived from a generic representation of health care known as the Reference Information Model [39]. Much of the ongoing work at HL7 focuses on the continued development of version 3 information models for various health care domains. Of note, however, HL7 version 2 information models are still much more widely adopted than version 3 information models in operational clinical systems, especially in the United States. Thus, for the foreseeable future, HL7 version 2 standards will likely need to be supported for the purposes of enabling standards-based and scalable CDS.
Beyond HL7, other standard information models include openEHR archetypes [40] and the ASTM International Continuity of Care Record [41]. Moreover, the clinical content of the Continuity of Care Record is now available in the Continuity of Care Document information model developed by HL7 and ASTM International [42].
Furthermore, work has recently begun within the HL7 CDS Work Group to establish a standard for a concept known as the virtual medical record (vMR) [43]. As originally proposed in the literature, a vMR consisted of a standardized definition of both (i) the semantics of information communicated between a CDS engine and a health information system and (ii) the functional capabilities of a health information system available to the CDS engine (e.g., to order a prescription or to deliver an alert) [44]. Currently, the HL7 vMR project is seeking to standardize the first of these two aspects of a CDS engine’s interaction with a health information system (i.e., the information models used to represent the patient data used by a CDS engine and the information models used to represent the inference results returned by a CDS engine) [43]. This project holds significant promise for standardizing how information is communicated between health information systems and CDS engines to generate patient-specific care assessments and recommendations.
Despite these promising standards, important gaps and challenges remain. First, current information models oftentimes lack sufficiently tight binding to terminologies. For example, a clinical information model may recommend the use of SNOMED CT to represent medical diagnoses but also allow the use of ICD9 or ICD10. Given the previously noted challenge with mapping concepts from different terminologies, this is a significant problem for implementing scalable CDS. Furthermore, as a second challenge, many of the current generation of information model standards – including many of the HL7 version 3 information model standards – entail much more flexibility and complexity than is required or desirable for CDS. For example, the HL7 version 3 information model for laboratory observations contains over fifty component classes, significant optionality, and several recursive relationships [45]. While this type of an information model does enable very sophisticated and complete clinical documentation, such extensive expressivity and flexibility make these information models much harder to understand and to implement in operational systems. This difficulty is exacerbated by a lack of tooling and other practical resources to support the widespread use of these information models by health information technology (IT) professionals. Moreover, much of the information captured in these models is largely irrelevant for CDS applications (e.g., who performed the laboratory test). At the same time, simplified patient information models appropriate for use as inputs by CDS systems have not yet been standardized, and standard information models for CDS outputs are largely non-existent. Finally, there is little alignment between these standard models and the logical structure or schemata of clinical data repositories in actual use. As a consequence of these various factors, there is currently very limited usage of standard information models within most operational CDS systems.
C. Standards for Patient Data Expected to be Available for CDS
From the perspective of a CDS implementer, the standardization of the type of patient data expected to be available for use within a CDS application would significantly reduce the complexity of developing and implementing CDS capabilities. While it may be unrealistic to have a single standard which is universally adopted, it would be possible to have a small number of standards on expected data availability (e.g., one standard for a full EHR environment, another standard for an environment only with access to claims data, and a third standard for an environment with access to claims data and laboratory data). While the HL7 vMR project is working to address this issue, a significant challenge with this regard is that this project is still under development and that different health IT systems continue to collect data with different granularity and scope.
D. Standard Approaches for Terminology Inferencing
A critical CDS implementation need is the mapping of clinically relevant concepts to terms used within various health information systems. These terminology needs for CDS typically manifest in two forms: (i) the need to identify which concepts are subsumed by a parent concept, and (ii) the need to translate concepts across vocabularies. For example, consider a simple CDS scenario in which there is a need to identify whether a patient has diabetes, so that appropriate disease management guidance can be provided. If the underlying health information system uses SNOMED CT to capture problem list data, there would be a need to identify all of the SNOMED CT codes that could have been used to note that a patient has diabetes (i.e., all of the SNOMED CT codes that are subsumed by the parent code of 73211009 [diabetes mellitus], such as 44054006 [diabetes mellitus type 2] and 190392008 [diabetes mellitus – poor control]). Moreover, if the CDS resource has been encoded using SNOMED CT codes for problem data, whereas a health IT system used ICD9 for capturing problem data, there would then be a need to identify matching concepts for diabetes in ICD9 (e.g., by identifying all ICD9 codes that map to the SNOMED CT codes for diabetes previously identified, and then by further identifying all ICD9 codes that are subsumed by the mapped ICD9 codes for diabetes).
A terminology service can provide these types of terminology inferencing capabilities through a service interface. Standardization of such service interfaces can enable CDS implementations to leverage different terminology services based on their capabilities, rather than being locked into a single solution because of investments in proprietary system-to-system interfaces. Fortunately, the HL7 Common Terminology Services version 1 [46] and 2 [47] standards provide standard specifications for how various terminology inferencing capabilities can be obtained from a software service. Perhaps more importantly, significant tooling and resources are being developed to support the use of this standard. Most notably, the National Cancer Institute has funded the development of an open-source terminology server known as LexBIG that is compatible with the Common Terminology Services standard [48]. This initiative provides an excellent model of how a CDS-related standard, when adequately resourced, can be taken from specification to robust, implementable software.
Despite the promise of this domain of standardization, it is still the case that many terminologies remain semantically incompatible, regardless of the tooling available for inferencing between such terminologies. Consequently, the use of standard information models with a limited set of bound terminologies is still essential to developing CDS in a highly scalable manner.
E. Standard Representation of Non-Executable Clinical Knowledge
Since 2002, the Guideline Elements Model (GEM) has been available as an ASTM International standard for the representation of the contents of clinical practice guidelines in a structured, non-executable format that is suitable for translation into an executable format [27]. Now in its second version (GEM II), this approach to knowledge modeling includes an XML Schema defining a structured format for extracting the relevant content of a clinical practice guideline, an object-oriented data model, and a freely downloadable tool for editing GEM guidelines known as the GEM Cutter.
A significant gap that exists in this aspect of the CDS standards continuum is that while GEM provides a relevant standard for the structured representation of individual clinical practice guidelines, significant medical knowledge exists outside of individual clinical practice guidelines (e.g., knowledge on drug-drug interactions). The absence of this knowledge from a published guideline can complicate its translation into an implementable or computable format [25]. Moreover, CDS applications often must integrate the guidance provided by multiple relevant clinical practice guidelines in a given domain, some of which may provide conflicting advice. Another important challenge is that the vast majority of guideline content producers continue to generate clinical practice guidelines in non-structured, non-standardized formats that must then be interpreted by an external group into a structured and potentially standardized format. Moreover, even when captured in a format such as GEM, there is still no clear path for moving medical knowledge from a non-executable format into an executable format that can be used in a scalable manner across multiple applications and care settings, although some groups have demonstrated translations of GEM to more computable formalisms such as the Arden Syntax [49].
F. Standard Representation of Executable Clinical Knowledge
Several standards are available for representing machine-executable clinical guidelines. These standards include two standards that can be used to represent machine-executable clinical rules. First, the HL7 Arden Syntax standard specifies various aspects of Medical Logic Modules, which include a specification of the following: knowledge meta-data; when the module should be triggered (e.g., the storage of a new potassium laboratory value in an EHR); how required data should be retrieved; how the input data should be used to generate a conclusion; and how that conclusion should be communicated to relevant end-users (e.g., as a text page) [8]. The Arden Syntax does have several challenges, however. While proprietary compilers have been developed for executing Arden Syntax Medical Logic Modules, there is a lack of a non-proprietary Arden compiler that works across applications and care settings. Furthermore, the current Arden Syntax specification lacks a standard model for representing input data (known as the “curly braces problem” because the data requirements are specified within curly braces in an implementation-specific manner). Despite the fact that the Arden Syntax is supported by several commercial EHR systems, there has not been much knowledge sharing among health care institutions using Arden. One exception to this limited sharing of Arden Syntax Medical Logic Modules has been knowledge sharing among institutions that are clients of the same CDS software vendor, as various implementations of the same core system can transfer knowledge with relatively less re-engineering [50].
GELLO is another HL7 standard for the representation of clinical rules in a machine-executable format [9]. Specified as an extension to the Object Management Group’s Object Constraint Language, GELLO provides a grammar for expressing the input data for clinical rules as well as how those data are to be manipulated to generate patient-specific inferences. As with the Arden Syntax, however, GELLO does not yet have a non-proprietary compiler that allows GELLO expressions to drive CDS across applications and care settings.
In addition to standards for representing clinical rules, standards are available for representing highly structured knowledge documents that can be used to drive CDS. One of these standards is the HL7 Structured Product Labeling standard [51], which is used by the U.S. Food and Drug Administration to provide medication labeling information in a structured format. Furthermore, the HL7 Order Set specification [52] enables the consistent exchange of information on order sets used within computerized provider order entry systems. This specification underwent successful balloting as a draft standard in September 2008 and is currently undergoing final enhancements based on peer review comments prior to being officially published as an HL7 draft standard. Also, the HL7 Health Quality Measures Format draft standard provides a standard format for representing quality measures that are primarily intended for quality reporting purposes but could also be leveraged for point-of-care CDS applications [53].
Despite these available standards, no standard is widely agreed upon for representing clinical practice guidelines in a machine-executable format. While the HL7 CDS Work Group has attempted in the past to specify and adopt a single, common model for representing clinical practice guidelines in a machine-executable format, attempts to date have been unsuccessful. Moreover, as is often the case with current standards within the health IT space, limited tooling and support is available for implementing CDS using many of these standards. Furthermore, in most cases, there is still limited availability of knowledge resources that are standards-compliant and easily available for use in CDS implementation efforts.
G. Standard Approaches to Utilizing Executable Clinical Knowledge
Distinct from the standardization of the representation of machine-executable medical knowledge is the standardization of how such knowledge is utilized. Two HL7 standards are currently available for accessing and utilizing machine-executable CDS resources through the calling of an external software service. First, the HL7 Decision Support Service draft standard [33] provides a standard approach to providing structured patient data and receiving structured, patient-specific inferences based on the use of Decision Support Service knowledge modules. This approach to providing CDS as a service is based on a services-based approach to CDS known as SEBASTIAN, which has been previously described [54]. Used in combination with other standard services within a service-oriented architecture, the use of Decision Support Services could help fulfill the strategic objectives of the Roadmap for National Action on CDS commissioned by the U.S. Office of the National Coordinator for Health IT [12]. As with other HL7 service standards, a detailed technical specification for the Decision Support Service has been developed and adopted by the Object Management Group (OMG) [34] through a joint HL7-OMG initiative known as the Healthcare Services Specification Project [55].
Furthermore, the HL7 Context-Aware Knowledge Retrieval (“Infobutton”) standard [56] provides a specification of how data on a patient, a clinical question, and the relevant clinical context can be passed to an external software service to retrieve back knowledge resources relevant for that specific situation as an attempt to help clinicians and patients fulfill their knowledge needs. The Infobutton standard has achieved rapid adoption among knowledge resource publishers and EHR vendors and has been selected by the Healthcare Information Technology Standards Panel (HITSP) as the recommended standard for retrieving context-relevant medical knowledge [57]. In addition, the HL7 Infobutton project is now developing a specification for service-oriented implementations of infobuttons as a profile within the HL7 Decision Support Service standard.
Despite these available standards, significant gaps and challenges remain. First, the semantics of service payloads for CDS are still undergoing standardization. The HL7 vMR project is expected to address this gap, but this work is still ongoing. Furthermore, no standard is available for identifying and retrieving machine-executable medical knowledge resources themselves (e.g., Arden Syntax Medical Logic Modules or HL7 Order Sets). Also, there is still work required on defining a commonly accepted meta-data model for knowledge resources.
H. Standard Approaches to CDS Delivery
Several standards are available to facilitate the scalable integration of medical knowledge resources within health information systems to enable CDS. The HL7 Retrieve, Locate, and Update Service draft standard [58] specifies a standard service interface for locating, retrieving, and updating patient health information, and the various terminology and information model standards discussed earlier are available for facilitating the communication of such information in a consistent manner. Furthermore, the HL7 EHR Functional Model [59] specifies the functions that should be available within compliant EHR systems, as does the Certification Commission for Health Information Technology [60].
Despite these important standards, significant gaps and challenges still remain for the standard delivery of CDS within health information systems. First, there is a lack of standards for the EHR services upon which many CDS implementations depend, such as for order placement or alert delivery. While the HL7 EHR Functional Model and the criteria set forth by the Certification Commission for Health Information Technology do encompass the capabilities that should be available within an EHR system, these specifications are functional in nature and not defined at the level of detail needed for semantic interoperability. The HL7-OMG Healthcare Services Specification Project is seeking to address this lack of standard service definitions in health care, and this initiative already has developed several relevant standards including the HL7 Retrieve, Locate, and Update Service draft standard [58] and the HL7 Decision Support Service draft standard [33]. However, much work still remains in this arena. Also, as discussed earlier, while the notion of a vMR has traditionally encompassed the standardization of EHR services, such standardization of EHR services is not within the scope of the current HL7 vMR project.
A further challenge to scalable CDS delivery is the lack of use in health care of standard business process modeling approaches [61]. If widely adopted within health care, standard business process modeling approaches would allow CDS implementers to specify, share, and adapt workflow specifications of human and computer actors relevant in the delivery of CDS.
Finally, a significant challenge to the standardized delivery of CDS is the enormous heterogeneity that currently exists, and is likely to continue to exist, in the health information system infrastructure available across different care settings [62]. Moreover, even if all EHR systems eventually supported standard interfaces and semantics for interaction with a CDS engine, such standardization would still leave significant gaps in the capacity for scalable CDS delivery if many clinicians continue to practice without the aid of EHR systems.
4. DISCUSSION AND RECOMMENDATIONS
Given the current standards landscape and our collective experience developing and implementing health IT standards for CDS, we provide the following recommendations for improving the scalable deployment of robust CDS capabilities through standardization:
- First and foremost, we recommend that standards development organizations focus on supporting the operational use of standards relevant for CDS. Currently, many health IT standards suffer from barriers that limit their widespread adoption, including excessive complexity, limited tooling, and poor documentation on how the standards should be used in operational clinical settings. Measures that can be taken to improve the usability of standards include a focus on implementability and simplification where possible; supporting the development and dissemination of improved tooling; and improved documentation on how to make use of the standards. We view the current lack of use of standards within many CDS systems not as the fault of CDS implementers, but partially as a consequence of CDS standards that are too complex and/or poorly supported to be used. Developers of CDS standards should strive to make the standards as easy to understand and use as XML, HTML, SQL, and other ubiquitous IT standards.
- Second, we recommend that efforts should be made to harmonize and recommend the use of existing and future standards for CDS. Both Integrating the Healthcare Enterprise (IHE) [63] and the successor to HITSP [64], whose initial contract with the U.S. Department of Health and Human Services was completed in April 2010, would be appropriate forums for conducting this harmonization process.
- Third, we recommend that health care stakeholders, and in particular the federal government, provide much greater support for the creation and adoption of health IT standards relevant to CDS. The U.S. federal government’s support for the harmonization of health IT standards through HITSP [64] was an important step in this direction. As this manuscript has outlined, the availability of competing and non-compatible standards is a significant barrier to scalable CDS. However, perhaps more important is the lack of relevant standards altogether, as well as the lack of appropriate resources and support for utilizing current and emerging standards. Federal support could significantly accelerate the rate at which these standards and supportive resources are developed and deployed.
- Fourth, we recommend that authors of clinical practice guidelines and other knowledge resources that could be used for CDS publish that knowledge in at least a computable, and if possible a fully implementable, format that incorporates standards and is unambiguously represented. The widespread availability of such high-quality, structured medical knowledge would greatly facilitate implementation of that knowledge in computer-based CDS systems and hence enable the more widespread dissemination of the knowledge to clinical end-users in an actionable format. Requirements for the establishment of such standards-compliant knowledge resources would include (i) the development of widely-accepted knowledge representation standards; (ii) tooling and best-practice guidance on how to create standards-compliant knowledge; (iii) the establishment of knowledge repositories with standardized approaches to identifying, accessing, and leveraging the deposited knowledge (e.g., through an HL7/OMG Decision Support Service interface [33,34]); (iv) the development of a legal and financial framework that is conducive to the contribution of such CDS content; and (v) the active participation of relevant stakeholders, including the federal government, as discussed next.
- As a final recommendation for enabling scalable and high-quality CDS, we recommend that the federal government put into place a robust process and funding for developing standards-compliant CDS knowledge resources that are easily accessible to the wider health care community. Large repositories of high-quality, standards-compliant CDS knowledge resources would catalyze the adoption of the relevant standards and would enable the scalable, widespread adoption of advanced CDS capabilities. Organizations that could participate in this knowledge development effort include federal agencies, academic institutions, and commercial vendors. Already, the federal government has demonstrated how such activities can be supported through projects such as the National Cancer Institute’s LexBIG project [48] and the National Library of Medicine’s support for a SNOMED CT license for all U.S. users [65]. Extension of such federal support for CDS will likely play a critical role in the widespread, scalable, and standards-based delivery of CDS.
In pursuing these standardization efforts, an important issue for explicit consideration will be whether an aspect of CDS is truly ready for standardization. Ideally, CDS standards will be based on multiple operational implementations and well-accepted best practices, so as to ensure the robustness and utility of the standard. When there is still a lack of consensus on best practices and the need for significant research and development, it may be best to hold off on standardization efforts so as to avoid the development of standards that hinder innovation, inadequately meet business needs, and/or fail to achieve significant adoption. At present, aspects of CDS standardization that may fall into this category of requiring further research and development include (i) the representation of clinical practice guidelines in a machine-executable format and (ii) the process of converting clinical knowledge from textual formats to computer-interpretable formats. For these and other cutting-edge areas of CDS, it will be important for standardization efforts to be coupled with continued research and development efforts to explore and validate best practices for enabling robust and scalable CDS.
Finally, with regard to research and development of a scalable and standards-based CDS infrastructure, it is important to note that the U.S. federal government is sponsoring several large-scale research efforts consistent with the recommendations provided above. These federally funded CDS efforts include the Morningside Initiative [66], which is a multi-institutional, public-private partnership to develop and disseminate CDS knowledge, as well as the GLIDES project [67], which is a project sponsored by the Agency for Healthcare Research and Quality (AHRQ) to demonstrate a systematic and replicable process by which knowledge contained in practice guidelines can be transformed into computer-based CDS and taken to scale to improve the quality of healthcare delivery in the U.S. Furthermore, the CDS Consortium [68] is a multi-institutional effort funded by the AHRQ to assess, define, demonstrate, and evaluate best practices for knowledge management and CDS in healthcare IT at scale. Of note, the CDS Consortium has developed preliminary recommendations for the health IT standards necessary for scalable CDS [69], and these recommendations are consistent with the recommendations provided in this manuscript. Another notable federally funded CDS initiative is a public-private initiative sponsored by the AHRQ known as the Hardened Rules project [70], in which care recommendations from the U.S. Preventive Services Task Force are being translated into shareable CDS modules encoded using the HL7 Arden Syntax standard. Also of note, the U.S. federal government has specified that healthcare providers must make use of CDS in order to obtain federal funding for the adoption and “meaningful use” of EHR systems [71]. These and other federally sponsored, large-scale CDS initiatives will be critical to the achievement of widespread and scalable CDS enabled by the core set of standards outlined in this manuscript.
5. CONCLUSION
CDS holds great promise for improving health and health care, but the limited scalability of many current approaches has limited the actual impact of CDS on health care. CDS standards – and, perhaps more importantly, robust tooling and resources supportive of these standards – will be critical to robust CDS capabilities becoming more widely available. While many relevant standards do exist, many gaps and challenges still remain. Overcoming these challenges will require a concerted effort to develop missing standards, to make the standards as easy and beneficial to use as possible, and to define how these standards should be used in a coordinated and practical fashion. Federal support for these efforts will likely play an important role in determining the degree to which these efforts are successful.
ACKNOWLEDGMENTS
All authors contributed to the conception and design of the manuscript, and KK drafted the manuscript. All authors contributed to the critical revision of the manuscript.
Preparation of this manuscript was funded by grant K01HG004645 from the U.S. National Human Genome Research Institute (KK) and by grant number K01HS018352 from the Agency for Healthcare Research and Quality (GDF). The funding sources played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.
CONFLICT OF INTEREST
KK and DFL are co-founders of Clinica, Inc., which is developing an open-source Decision Support Service compliant with the HL7 and OMG Decision Support Service standards discussed in this manuscript. Clinica will make any of its intellectual property required for implementing to the HL7 and OMG Decision Support Service standards available to others for free.
REFERENCES
[1] | Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support J Am Med Inform Assoc 2007; 14(2): 141-5. |
[2] | Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success BMJ 2005; 330(7494): 765-8. |
[3] | Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention J Am Med Inform Assoc 1999; 6(4): 313-21. |
[4] | Simon SR, Kaushal R, Cleary PD, et al. Physicians and electronic health records: a statewide survey Arch Intern Med 2007; 167(5): 507-12. |
[5] | Kawamoto K. Integration of knowledge resources into applications to enable clinical decision support: architectural considerations In: Green RA, Ed. Clinical Decision Support: the road ahead. Boston: Elsevier Academic Press 2007; pp. 503-38. |
[6] | Sittig DF, Wright A, Osheroff JA, et al. Grand challenges in clinical decision support J Biomed Inform 2008; 41(2): 387-92. |
[7] | Wright A, Sittig DF. A four-phase model of the evolution of clinical decision support architectures Int J Med Inform 2008; 77(10): 641-9. |
[8] | Pryor TA, Hripcsak G. The Arden syntax for medical logic modules Int J Clin Monit Comput 1993; 10(4): 215-4. |
[9] | Sordo M, Boxwala AA, Ogunyemi O, Greenes RA. Description and status update on GELLO: a proposed standardized object-oriented expression language for clinical decision support Stud Health Technol Inform 2004; 107(Pt 1): 164-8. |
[10] | Boxwala AA, Peleg M, Tu S, et al. GLIF3: a representation format for sharable computer-interpretable clinical practice guidelines J Biomed Inform 2004; 37(3): 147-61. |
[11] | Ram P, Berg D, Tu S, et al. Executing clinical practice guidelines using the SAGE execution engine Medinfo 2004; 11(Pt 1): 251-5. |
[12] | Kawamoto K, Lobach DF. Proposal for fulfilling strategic objectives of the U.S. roadmap for national action on decision support through a service-oriented architecture leveraging HL7 services J Am Med Inform Assoc 2007; 14: 146-55. |
[13] | Kawamoto K, Lobach DF, Willard HF, Ginsburg GS. A national clinical decision support infrastructure to enable the widespread and consistent practice of genomic and personalized medicine BMC Med Inform Decis Mak 2009; 9: 17. |
[14] | Osheroff JA, Pifer EA, Teich JM, Sittig DF, Jenders RA. Improving outcomes with clinical decision support: an implementer's guide 2005. |
[15] | Peleg M, Boxwala AA, Ogunyemi O, et al. GLIF3: the evolution of a guideline representation format AMIA Annu Symp Proc 2000; 645-9. |
[16] | Peleg M, Keren S, Denekamp Y. Mapping computerized clinical guidelines to electronic medical records: knowledge-data ontological mapper (KDOM) J Biomed Inform 2008; 41(1): 180-201. |
[17] | German E, Leibowitz A, Shahar Y. An architecture for linking medical decision-support applications to clinical databases and its evaluation J Biomed Inform 2009 Apr; 42(2): 203-18. |
[18] | Peleg M, Wang D, Fodor A, Keren S, Karnieli E. Lessons learned from adapting a generic narrative diabetic-foot guideline to an institutional decision-support system Stud Health Technol Inform 2008; 139: 243-52. |
[19] | Parker CG, Rocha RA, Campbell JR, Tu SW, Huff SM. Detailed clinical models for sharable, executable guidelines Medinfo 2004; 11(Pt 1): 145-8. |
[20] | International Health Terminology Standards Development Organisation. Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) Available from: http://www.ihtsdo.org/snomed-ct [cited 2010 Feb 12]; |
[21] | World Health Organization. International Classification of Diseases (ICD) Available from: http://www.who.int/ classifications/icd/en [cited 2010 Jun 14]; |
[22] | Jenders RA, Corman R, Dasgupta B. Making the standard more standard: a data and query model for knowledge representation in the Arden syntax AMIA Annu Symp Proc 2003; 323-0. |
[23] | Bodenreider O. Biomedical ontologies in action: role in knowledge management, data integration and decision support In: Yearbook of Medical Informatics. 2008; pp. 67-79. |
[24] | Cimino JJ, Zhu X. The practical impact of ontologies on biomedical informatics In: Yearbook of Medical Informatics. 2006; pp. 124-35. |
[25] | Tierney WM, Overhage JM, Takesue BY, et al. Computerizing guidelines to improve care and patient outcomes: the example of heart failure J Am Med Inform Assoc 1995; 2(5): 316-22. |
[26] | Peleg M. Guideline and workflow models In: Greenes RA, Ed. Clinical Decision Support: the Road Ahead. Boston: Elsevier Academic Press 2007; pp. 281-306. |
[27] | Shiffman RN, Michel G, Essaihi A, Thornquist E. Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation J Am Med Inform Assoc 2004; 11(5): 418-26. |
[28] | Kaiser K, Akkaya C, Miksch S. How can information extraction ease formalizing treatment processes in clinical practice guidelines? A method and its evaluation Artif Intell Med 2007 Feb; 39(2): 151-63. |
[29] | Shalom E, Shahar Y. A graphical framework for specification of clinical guidelines at multiple representation levels AMIA Annu Symp Proc 2005; 679-83. |
[30] | Hammond WE. The making and adoption of health data standards Health Aff (Millwood) 2005; 24(5): 1205-3. |
[31] | Peleg M, Tu S, Bury J, et al. Comparing computer-interpretable guideline models: a case-study approach J Am Med Inform Assoc 2003; 10(1): 52-68. |
[32] | Karadimas HC, Chailloleau C, Hemery F, Simonnet J, Lepage E. Arden/J: an architecture for MLM execution on the Java platform J Am Med Inform Assoc 2002; 9(4): 359-68. |
[33] | Health Level 7. HL7 Decision Support Service specification (Draft Standard for Trial Use) Available from: http://www.hl7.org/v3ballot2009jan/html/infrastructure/dss/dss.htm [cited 2010 Jun 14]; |
[34] | Object Management Group. OMG Clinical Decision Support Service standard (adopted beta specification) Available from: http://hssp-dss.wikispaces.com/omg_specification [cited 2010 Jun 14]; |
[35] | Peleg M, Steele R, Thomson R, Patkar V, Rose T, Fox J. Open-source publishing of medical knowledge for creation of computer-interpretable guidelines 3581 In: Tenth Conference on Artificial Intelligence in Medicine: Lect Notes Comput Sci; 2005; pp. : 156-60. |
[36] | National Library of Medicine. Unified Medical Language System Available from: http://www.nlm.nih.gov/resear ch/umls [cited 2010 Jun 14]; |
[37] | Regenstrief Institute. Logical Observation Identifiers Names and Codes (LOINC) Available from: http://loinc.org/ [cited 2010 Jun 14]; |
[38] | National Library of Medicine. RxNorm Available from: http://www.nlm.nih.gov/research/umls/rxnorm/ [cited 2010 Jun 14]; |
[39] | Kabachinski J. What is Health Level 7? Biomed Instrum Technol 2006; 40(5): 375-9. |
[40] | Garde S, Hovenga E, Buck J, Knaup P. Expressing clinical data sets with openEHR archetypes: a solid basis for ubiquitous computing Int J Med Inf 2007; 76(Suppl 3): S334-41. |
[41] | Ferranti JM, Musser RC, Kawamoto K, Hammond WE. The clinical document architecture and the continuity of care record: a critical analysis J Am Med Inform Assoc 2006; 13(3): 245-52. |
[42] | Health Level 7. Continuity of Care Document (CCD) Release 1 Available from: http://www.hl7.org/Library/ General/HL7_CCD_final.zip [cited 2010 Jun 14]; |
[43] | Health Level 7. HL7 Virtual Medical Record (vMR) Project Wiki Available from: http://wiki.hl7.org/index.php? title=Virtual_Medical_Record_(vMR) [cited 2010 Jun 14]; |
[44] | Johnson PD, Tu SW, Musen MA, Purves I. A virtual medical record for guideline-based decision support AMIA Annu Symp Proc 2001; 294-8. |
[45] | Health Level 7. HL7 Version 3 Laboratory Result Event Refined Message Information Model Available from: http://www.hl7.org/v3ballot2010may/html/domains/uvlb/editable/POLB_RM004000UV.htm [cited 2010 Jun 14]; |
[46] | Health Level 7. HL7 Common Terminology Services standard Available from: http://www.hl7.org/v3ballot2010sep/html/infrastructure/cts/cts.htm [cited 2010 Jun 14]; |
[47] | Health Level 7. HL7 Common Terminology Services 2 Service Functional Model (Draft Standard for Trial Use) Available from: http://www.hl7.org/v3ballot2010sep/html/infr astructure/cts_r2/cts_r2.htm [cited 2010 Jun 14]; |
[48] | Pathak J, Solbrig HR, Buntrock JD, Johnson TM, Chute CG. LexGrid: a framework for representing, storing, and querying biomedical terminologies from simple to sublime J Am Med Inform Assoc 2009; 16(3): 305-15. |
[49] | Agrawal A, Shiffman RN. Using GEM-encoded guidelines to generate medical logic modules AMIA Annu Symp Proc 2001; 7-11. |
[50] | Middleton B, Anderson J, Fletcher J, Masarie FE Jr, Leavitt MK. Use of the WWW for distributed knowledge engineering for an EMR: the KnowledgeBank concept AMIA Annu Symp Proc 1998; 126-30. |
[51] | Schadow G. Assessing the impact of HL7/FDA Structured Product Label (SPL) content for medication knowledge management AMIA Annu Symp Proc 2007; 646-50. |
[52] | Health Level 7. HL7 Order Sets specification Available from: http://www.hl7.org/v3ballot2010sep/html/domains/uvds/uvds_OrderSets.htm#REDS_DO010001UV-OrderSets-ic [cited 2010 Jun 14]; |
[53] | Health Level 7. HL7 Health Quality Measures Format: eMeasures (Draft Standard for Trial Use) Available from: http://www.hl7.org/v3ballot2010sep/html/domains/uvqm/uvqm.htm [cited 2010 Jun 14]; |
[54] | Kawamoto K, Lobach DF. Design, implementation, use, and preliminary evaluation of SEBASTIAN, a standards-based Web service for clinical decision support AMIA Annu Symp Proc 2005; 380-4. |
[55] | Kawamoto K, Honey A, Rubin K. The HL7-OMG Healthcare Services Specification Project: motivation, methodology, and deliverables for enabling a semantically interoperable service-oriented architecture for healthcare J Am Med Inform Assoc 2009; 16(6): 874-1. |
[56] | Health Level 7. HL7 Context-aware Information Retrieval (Infobutton) standard Available from: http://www.hl7.org/v3ballot2010may/html/domains/uvds/uvds_Context-awareKnowledgeRetrieval(Infobutton).htm [cited 2010 Jun 14]; |
[57] | Healthcare Information Technology Standards Panel. HITSP transaction T 81 - retrieval of medical knowledge transaction Available from: http://www.hitsp.org/ConstructSet_ Details.aspx?&PrefixAlpha=3&PrefixNumeric=81 [cited 2010 Jun 14]; |
[58] | Health Level 7. HL7 Retrieve, Locate, and Update Service specification (Draft Standard for Trial Use) Available from: http://www.hl7.org/v3ballot2009jan/html/infrastructur e/rlus/rlus.htm [cited 2010 Jun 14]; |
[59] | Health Level 7. HL7 Electronic Health Record Technical Committee home page Available from: http://www.hl7.org/EHR [cited 2009 Jun 11]; |
[60] | Certification Commission for Health Information Technology. CCHIT: Certification Commission for Health Information Technology Available from: http://www. cchit.org [cited 2010 Jun 14]; |
[61] | Organization for the Advancement of Structured Information Standards. Standards for business process modeling, collaboration, and choreography Available from: http://xml.coverpages.org/bpm.html [cited 2010 Jun 14]; |
[62] | DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care--a national survey of physicians N Engl J Med 2008; 359(1): 50-60. |
[63] | Siegel EL, Channin DS. Integrating the Healthcare Enterprise: a primer. Part 1. Introduction Radiographics 2001; 21(5): 1339-41. |
[64] | Foxhall K. HITSP working to harmonize. The national data exchange group is making tough decisions to provide the industry with a sound set of standards Healthc Inform 2006; 23(9): 28. |
[65] | National Library of Medicine. FAQs: Inclusion of SNOMED CT in the UMLS Available from: http://www.nlm. nih.gov/research/umls/Snomed/snomed_faq.html [cited 2010 Jun 14]; |
[66] | American Medical Informatics Association. AMIA strategic initiatives and programs: Morningside Initiative Available from: http://www.amia.org/inside/initiatives/cds [cited 2010 Jun 14]; |
[67] | GLIDES Project. GLIDES Project Home Page Available from: http://gem.med.yale.edu/glides [cited 2010 Jun 14]; |
[68] | Clinical Decision Support Consortium. Clinical Decision Support Consortium Home Page Available from: http://www.partners.org/cird/cdsc/default.asp [cited 2010 Jun 14]; |
[69] | Sittig DF, Wright A, Ash JS, Middleton B. A set of preliminary standards recommended for achieving a national repository of clinical decision support interventions AMIA Annu Symp Proc 2009; 2009: 614-8. |
[70] | Agency for Healthcare Research and Quality. AHRQ's National Resource Center for Health IT: the Next Phase Available from: http://healthit.ahrq.gov/portal/server.pt/gateway/PTARGS_0_11699_910395_0_0_18/NRC%20one%20pager.pdf [cited 2010 Jun 14]; |
[71] | U.S. Department of Health and Human Services. Meaningful Use Available from: http://healthit.hhs.gov/portal/ server.pt?open=512&objID=1325&parentname=CommunityPage&parentid=1&mode=2 [cited 2010 Jun 14]; |