Behavioral Health Order Sets in a Hybrid Information Environment

John Strauss*, Peggy Olbrycht, Vincent Woo
Centre for Addiction and Mental Health, Toronto, Canada

Article Metrics

CrossRef Citations:
Total Statistics:

Full-Text HTML Views: 3010
Abstract HTML Views: 2311
PDF Downloads: 295
Total Views/Downloads: 5616
Unique Statistics:

Full-Text HTML Views: 1407
Abstract HTML Views: 1244
PDF Downloads: 163
Total Views/Downloads: 2814

© John Strauss; Licensee Bentham Open.

open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License ( which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

* Address correspondence to this author at the Centre for Addiction and Mental Health, 1001 Queen St. W., Toronto, ON, M6J 1H4, Canada; Tel: 416 254 3413; Fax: 416 583 1367; E-mail:,



The Centre for Addiction and Mental Health (CAMH) is a 500 bed freestanding psychiatric hospital in Canada. We are in the process of preparing for an integrated commercial clinical information system, which will have computerized physician order entry (CPOE) functionality.


As a preparation for CPOE, we developed inpatient order sets (OSs). Development teams from individual clinical programs created and sent their OSs to an OS Working Group for initial endorsement, and then to Pharmacy & Therapeutics and Medical Advisory committees subsequent approvals.


In twelve months we created and introduced 22 behavioral health OSs across eight clinical programs in our hybrid information system with an excellent adoption rate (>97%) by clinicians.


The development and implementation temporarily contributed to a multifactorial flow problem in the emergency department (ED), which was addressed by substantially simplifying the General Admission via the ED OS. Also, as the OSs were developed and sent for approval the project identified areas where local clinical practice can improve. Our electronic-paper hybrid set of clinical systems was a major factor impacting the effort.

Keywords: : Order sets, behavioral health, EMR.


The Centre for Addiction and Mental Health (CAMH) is a 500-bed freestanding mental health hospital with 3000 employees, including 130 full-time medical staff. There are a total of eight Clinical Programs or service lines, and an active Primary Care service. We currently have a hybrid of electronic and paper documentation systems; physician orders are on paper with the exception of laboratory and dietary orders, which are electronic. Our order set (OS) initiative was launched to aid our preparations for a Computerized Provider Order Entry (CPOE) project that was imminent. To assist with this project a vendor was used to support the development of order sets (OSs) and the corresponding clinical process redesign work that are mandatory prerequisites to CPOE. The goal was to develop and implement OSs in a hybrid clinical information environment across eight inpatient programs over one year. Reasons for OSs are many [1], including 1) OSs are required as clinical content in preparation for Computerized Provider Order Entry (CPOE) [2,3] ; 2) Paper-based OSs will ready our staff for CPOE and standardization of care [4] ; and 3) OSs can improve quality [5-8], safety [9], resource utilization [10], order completeness, workflow, staff and patient satisfaction [11]. As is the case with the use of electronic systems more generally, OSs are widely used in physical medicine, but less so in behavioral health. As inpatient psychiatry gradually adopts electronic medical records, OSs will play a growing role in bringing clinical content to electronic ordering. Our main research questions were: How readily will clinicians adopt paper order sets in a hybrid information system? What obstacles will need to be overcome?


We applied a variant of the methodology recommended by our vendor, In February an OS working group (OSWG) was formed and it consisted of representatives from physicians, nursing informatics, clinical forms administration, medical informatics, health information systems analyst, allied health, nursing (nurse educator and advanced practice nurse), health records, laboratory, and clinical informatics. The OSWG began having weekly meetings in March 2010. Its mandate included:

  1. support the creation and maintenance of standardized modular OSs;
  2. establish a Standard Reference OS with the oversight of for a guideline in the creation of CAMH OSs;
  3. facilitate multidisciplinary communication and collaboration;
  4. ensure compliance with Institute and Safe Medication Practices (ISMP) standards, hospital abbreviation standards, hospital formulary, and any hospital policies that may influence OS content.

Meetings were set with all physicians of each clinical program and with managers and their clinical staff to inform them of the implementation plan, educate them about OSs and convey what was expected of them.

The eight clinical inpatient programs and primary care were informed that members from each program were required to participate on their local program content teams in April. There was an initial meeting with each content team in May to inform them of the time line (6 weeks) to develop their OSs complete with input and local consensus from all physicians in their program [12]. Each team would meet weekly there after to develop content for their respective programs; they were also responsible to seek feedback and agreement from all clinicians, including review by Clinical Program Medical Directors. Throughout the development phase, the OSWG would also meet weekly to review and recommend changes to the originating content team.

After final revisions were completed on the OSs, they were submitted for approval. The first level of approval was from the OSWG; OSs then went on to the Pharmacy and Therapeutic Committee for subsequent approval; then lastly to the Medical Advisory Committee (MAC) for final vetting. Table 1 shows all the OSs that were finally approved by the MAC.

Table 1.

Order Sets with Corresponding Content Teams

Order Set Number Order Set Content Team
1 ED/EAU*** Alcohol Withdrawal with Diazepam CATS*
2 Mood and Anxiety Disorders Direct Admission Mood and Anxiety
3 Bowel Care Primary Care
4 Diabetes Management Primary Care
5 Electroconvulsive Therapy (ECT****) Referral Mood and Anxiety
6 Electroconvulsive Therapy (ECT****) Treatment Mood and Anxiety
7 Electroconvulsive Therapy (ECT****) Direct Admission Geriatric Mental Health
8 ED/EAU Hypertension Crisis Primary Care
9 Dual Diagnosis Direct Admission Dual Diagnosis
10 Dual Diagnosis Privileges and Passes Dual Diagnosis
11 General Admission via the ED CATS*
12 General Psychiatry Unit/Acute Care Unit Direct Admission CATS*
13 Law and Mental Health Direct Admission LAMHP**
14 Medical Withdrawal Service + 21 Day Program Direct Admission Addictions
15 Metabolic Monitoring Schizophrenia
16 Women’s Program Direct Admission Women's Program
17 Geriatric Mental Health Dysphagia Management Geriatric Mental Health
18 Geriatric Mental Health Falls Prevention Geriatric Mental Health
19 Geriatric Mental Health Behavioral Management Geriatric Mental Health
20 Geriatric Mental Health Direct Admission Geriatric Mental Health
21 Schizophrenia Direct Admission Schizophrenia
22 Nicotine Replacement Therapy (NRT) Primary Care/Addictions

Centralized Assessment, Triage and Support.

** Law and Mental Health Program.

*** Emergency Department/ Extended Assessment Unit.

**** Electroconvulsive Therapy.

The paper OSs consisted of physician orders, alerts and clinical information. After final approval at MAC, OSs were implemented. Communications were sent to all clinicians outlining instructions, implementation dates, available education, support and contact information. The medical staff received training in the use of OS at their usually scheduled physician meetings and were supplied all relevant implementation information. Throughout the implementation of the emergency department all residents on their ED rotation were educated in the use of the paper OSs. Clinical staff provided post-implementation comments and suggestions to the lead of the OSWG.

Implementation was over three months starting in late September and finishing mid-December.

During the implementation of the twenty-two OSs, clinical support was provided to each area in the form of a clinical informatics nurse physically present for 12 hours daily, and then being available via phone for the remaining 12 hrs. Initial support lasted one to three weeks depending on the size of the program. Following the initial support period, 24-hour phone support by the clinical informatics nurse went for a period of 6 months (Table 2).

Table 2.

Order Sets Implementation Schedule

Clinical Program Date
Geriatric Mental Health September 27
Addictions October 4
Schizophrenia October 12
Law and Mental Health (LAMHP) October 24
CATS November 8
Women’s Program November 22
Mood and Anxiety December 6
Dual Diagnosis December 13

This project was approved by our institutional research ethics board. In order to help assure support and engagement at all levels of the organization there was regular communication with the executive team which started in January 2010. Communications and updates to all clinicians were ongoing for the project's duration.


The clinical programs employed OSs and the direct admission OSs were added to each unit's routine paper admission packet. Auditing of admission OS use was done six months post-implementation and we observed that 150/152 (98.7%) of all admissions in March and April 2011 used a Direct Admission or General Admission via the ED OS. Sixty percent of audited admissions used the General Admission via the Emergency Department (ED) OS, as the ED is the entry route for most CAMH inpatients. Nursing signatures were complete in 146/150 (97.3%) of admissions. Post-implementation reports from nursing indicated that nurses were very much in favour of the paper-based OSs for their completeness, clarity, and efficiency in providing care. At the same time, reports from the Emergency Department (ED) informed the OSWG that their admission OS was too lengthy.


Throughout the development and implementation stages communication with and engagement of clinicians was essential to the success of the project. Post-implementation reports from nursing indicated that nurses were very much in favour of the paper-based OSs for their completeness, clarity, and efficiency in providing care.

Several issues were encountered during implementation and were responded to with iterative modifications to order sets. Changes requested by clinical areas were made, revised OSs were approved, put into clinical use, and that cycle was repeated as necessary. Nursing highlighted one key area: nurses questioned why they were required to sign off items on an OS that were not doctors' orders. Sign off of such items was required in some OSs because some of the content teams had tried to use the OSs to improve care quality by including nursing tasks that do not require a physician order, or by including certain kinds of documentation (e.g. alerts). The OSs have since been reviewed and unnecessary orders, fields and items have been removed. A second issue was the implementation of paper OSs in a hybrid paper-electronic set of clinical systems. Hybrid OSs were needed, with the majority being on paper, while lab and dietary orders were electronic. Specific groups of lab orders were created, one for each OS, within our electronic ordering system. The splitting of OSs into digital and analog portions made necessary by our hybrid environment reduced the usability of OSs. An accompanying benefit was a rise in the rate of electronic lab orders at admission. Thirdly, most inpatient areas introduced OSs and a separate and new inpatient medication reconciliation procedure at the same time, so staff had to accommodate two process changes.

As noted above, another problem requiring resolution happened in the Emergency Department (ED). Our ED is a high-volume area staffed mainly by a large number of residents who rotate, with any given resident providing periodic coverage. Medical staff and residents in the ED found that the admission OS was too long – combined with the new admission medication reconciliation steps, ED admission ordering became too complex. Both medication reconciliation and OS projects had an opportunity to simplify their processes: the CATS (ED) content team reduced the General Admission via the ED OS from three pages to one. The revised abbreviated version was accepted by medical staff and residents in the ED and is currently in use.

There were some limitations to the OS design and implementation. Usability testing [13, 14] was not part of the project, and would have helped to avoid the flow problem in the ED. For speed and ease of implementation with limited resources, we focused on the pragmatic local consensus method [12], based on our standard of care. Lastly, the foundation of all CAMH OSs was a standard reference OS based on the vendor's standard reference OS derived from acute care settings; as such the standard reference OS was not behavioral health-specific.


Over one year, we established an approvals process, developed OS content, and implemented twenty-two inpatient OSs in eight clinical programs. To our knowledge, this is the first report on order sets from a freestanding psychiatric facility with a hybrid information environment. Reflecting our hybrid clinical systems, the OSs themselves were hybrid, principally paper-based, with corresponding electronic dietary and laboratory portions for each OS. Initially the OSs were designed to be quite comprehensive; experience of the hybrid OSs by ED and nursing staff has since led to review, with scrutiny to augment efficiency. Despite a hybrid information environment, and a separate concurrent medication reconciliation project, clinical staff fully embraced OSs for inpatient behavioral health. In the future, we will increase evidence-based content, convert the OSs to fully electronic versions for CPOE, and add mental health protocols and pathways.


The author confirms that this article content has no conflict of interest.


We would like to acknowledge the assistance of Dr. Chris O'Connor and Kathy DeCaire of, for supporting our development of behavioral health order sets.


[1] Bobb AM, Payne TH, Gross PA. Viewpoint: controversies surrounding use of order sets for clinical decision support in computerized provider order entry J Am Med Inform Assoc 2007; 14: 41-7.
[2] Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE Implementation J Am Med Inform Assoc 2003; 10: 229-34.
[3] Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system J Am Med Inform Assoc 2003; 10: 322-9.
[4] Heffner JE, Brown K, Ellis R, Brown S. Using intranet-based order sets to standardize clinical care and prepare for computerized physician order entry Jt Comm J Qual Saf 2004; 30: 366-76.
[5] Ozdas A, Speroff T, Waitman R, Ozbolt J, Butler J, Miller RA. Integrating “best of care” protocols into clinicians' workflow via care provider order entry: impact on quality-of-care indicators for acute myocardial infarct J Am Med Inform Assoc 2006; 13: 188-96.
[6] Chisolm DJ, McAlearney AS, Veneris S, et al. The role of computerized order sets in pediatric inpatient asthma management Pediatr Allergy Immunol 2006; 17: 199-206.
[7] O'Connor C, Adhikari NK, DeCaire K, Friedrich JO. Medical admission order sets to improve deep vein thrombosis prophylaxis rates and other outcomes J Hosp Med 2009; 4: 81-9.
[8] Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on inpatient management of hyperglycemia: a cluster-randomized trial Endocr Pract 2010; 16: 209-18.
[9] Hermayer KL, Cawley P, Arnold P, et al. Impact of improvement efforts on glycemic control and hypoglycemia at a university medical center J Hosp Med 2009; 4: 331-9.
[10] Guttmann A, Zagorski B, Austin PC, et al. Effectivness of Emergency Department Asthma Management Strategies on Return Visits in Children: A Population-Based Study Pedaitrics 2007; 120: e1402-0.
[11] Institute for Safe Medication Practices. ISMP's Guidelines for Standard Order Sets. PA, USA: ISMP 2010; pp. 1-10.
[12] Novak LL. Making sense of clinical practice: order set design strategies in CPOE AMIA Annu Symp Proc 2007; 2007: 568-72.
[13] Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets? J Am Med Inform Assoc 2011; 18: 276-81.
[14] Chan J, Shojania KG, Easty AC, Etchells EE. Usability evaluation of order sets in a computerized provider order entry system BMJ Qual Saf 2011; 20(11): 932-40.