NPfIT was established in 2002 to deliver “twenty-first century IT support” for the NHS, defined as a set of electronic services to provide lifelong health records, direct booking of outpatient appointments and transmission of prescriptions from primary care to pharmacy, all based upon a robust information architecture and secure broadband network [1].

There has been sustained and widespread criticism, not always well-informed, of the perceived failures of NPfIT and its overly centralist, secretive and monolithic approach [2, 3]. For example, the 2002 Gateway Review by the UK Office of Government Commerce (OGC) into the business justification for what was then called the Integrated Care Records Service (only recently published following requests made under the provisions of the Freedom of Information Act [4]) stated that they found “much talk of what the IT programme will achieve, but little recognition of the potential impact of this on current practices, procedures and systems, both technical and organisational” [5].

Similar themes continue to be echoed [6]. The evaluation report on the NPfIT Summary Care Record pilot implementation in North West England suggested that the project’s failure to adopt a socio-technical approach was one reason why the system was not embraced by busy clinicians [7].

Comparable problems have been reported in other major health informatics (HI) implementations [8, 9], so this issue is by no means unique to the English programme.

The roots of socio-technical thinking can be found in research studies by the Tavistock Institute in the 1950s. For example, investigations in the British coal industry found that autonomous groups of miners had re-invented surprisingly effective innovations in mining methods contrary to “obvious” techniques suggested by increasing mechanization [10]. The socio-technical approach was built upon a philosophy of improving worker satisfaction and quality of life, with the belief that this leads to improved productivity and effectiveness [11].

In this paper, the “socio-technical” perspective is understood to refer primarily to the insight that a technical system implementation inevitably affects and is affected by the interdependent social system within which and upon which it operates [12, 13]. This concept has been used to formulate design principles [14] (see Box 1) and methods [15, 16] for information systems.

This approach requires a more nuanced worldview than the positivist mindset typically associated with reductionist physical science and “hard” technology. It utilizes perspectives such as constructivist or interpretive epistemologies and a social interactionist or negotiated view of reality [17] involving complex adaptive systems [18]. This paper argues that these factors are particularly crucial in HI and introduces a tool that supports adoption of a socio-technical approach in HI.

The organization and practice of healthcare is one of humanity’s most complex endeavours. Clinical work has been described as mostly “unpredictable and non-routine” [19] and “highly contingent, ad-hoc and idiosyncratic” [20]. Dealing with multiple uncertainties is a challenge for clinicians [21, 22], so arguably even more so for planners, designers or implementers of HI solutions [23, 24]. In particular, the NHS is a paradox: nominally a centrally-controlled monolith but in fact a coalition of competing confederacies of autonomous interests.

We argue that this irreducible complexity means that a socio-technical approach is vital to the success of innovations and transformations that are based on HI. The socio-technical approach would help to negotiate these complexities using principles such as minimal process specification, multi-stakeholder participation, user leadership, local adaptability and holistic evaluation [14].

Unfortunately, knowledge of socio-technical principles and their vital importance in HI does not seem to be widespread among practitioners. The socio-technical assessment tool for health informatics (STAT-HI) proposed here aims to help bridge the gap between research and practice by distilling key factors of good practice into a simple, readily accessible and applicable form.

The tool is expressed as a reflective checklist to validate the socio-technical soundness of an HI implementation approach. It does not aim to check every aspect of project fitness for purpose.

This approach seeks to utilize the ‘checklist effect’. Merely having a structured set of tasks or questions can in itself improve outcomes. This effect is so strong that it is a well-known source of bias in studies that compare interventions that use any checklist-based approach (whatever its merits) against those that do not [25]. Checklists have proved very powerful in improving aspects of clinical practice [26].

The checklist was conceived as a categorized summary of principles derived from a synthesis of critical success factors identified in the literature. It emulates the approach and certain principles of the AHRQ [27], GEP-HI [28] and STARE-HI [29] standards for designing and reporting system evaluation in health informatics.

The primary sources used were seminal UK reports on reasons for success or failure in major IT projects [30, 31] and a selection of HI studies [32-39] encountered in an ongoing literature review of health informatics theory by the present authors and colleagues [40]. This selection does not purport to be comprehensive or to form a systematic review of socio-technical principles in HI, but is presented as a starting point for discussion and development. The sources were selected on face validity as offering immediately relevant and directly applicable socio-technical assessment criteria.

The checklist emphasises particular elements of programme governance, cultural factors, the approach to changes in workflow and practice and the need for strong and balanced clinical leadership and governance. The headings echo the critical success factors identified by Whetton [17] (see Box 2) and Georgiou & Westbrook’s ten questions to consider before implementing computerized physician order entry (CPOE) [41] (see Box 3).

The STAT-HI checklist is presented as a tool for project/programme managers, project/programme boards and other professionals who provide external assurance and audit functions.

It is suggested that a required reading list should be promulgated to HI practitioners alongside this checklist, comprising eight books, papers and electronic sources [30-32, 34, 37-39, 45].

The reading list supports the aim of the checklist to be reflective, rather than simply a tick-box exercise. The checklist is an opportunity to learn, to enlarge one’s worldview and to change practice. Schön [46] argued that reflective practice goes beyond problem-solving by considering problem setting: has it been ‘named and framed’ rightly, or is evidence emerging that it needs to be rethought?

Therefore we suggest that a reviewer (whatever their professional role) making an assessment using STAT-HI should give narrative justification for their considered responses to each of the questions, adducing data to substantiate their commentary as necessary to satisfy both their own ethical standard and the independent critical judgement of the relevant project/programme assurance bodies.

By definition, this checklist utilizes existing evidence and prior good practice recommendations. Its only claim to originality is the condensation and annotated checklist presentation of key guidance and the suggestion of a specific minimum knowledge base for HI practitioners.

Arguably there are socio-technical principles embedded in numerous extant methods, even if not explicitly. Examples of this include the OGC programme management guidelines, Soft Systems Methodology and the ETHICS method.

OGC’s Managing Successful Programmes [45] acknowledges that change programmes involve ambiguity and the need to establish both organizational readiness for change and capability to manage and deliver change. Programmes are described as led by vision not specification. They are most effective when issues are debated freely and risks evaluated openly. The programme team needs staff with the relevant skills and experience to manage cultural and people issues involved in the business change.

Checkland’s Soft Systems Methodology is a process of enquiry that accepts and examines the multiple perceived realities of different stakeholders in a problem scenario. It envisages a group learning cycle that seeks to discover feasible improvements that can be accommodated within the continually re-negotiated social interpretations of individuals and organizations. Its philosophy directly rejects the idea that its conceptual models represent descriptions of an absolute reality, but regards “systems” as devices to think about the purposeful activities of various actors and thus formulate questions of the problem situation [47]. So if used for HI, the soft systems approach might in a sense be regarded as an application of socio-technical principles but with a more strongly constructivist interpretation of what we mean by “system”.

Mumford devised the ETHICS method (Effective Technical and Human Implementation of Computer-based Systems) as an explicitly socio-technical approach [15-16]. It emphasizes participatory diagnosis of problems, setting objectives for both efficiency and work satisfaction and ensuring compatibility of the technical and organizational systems. Mumford’s extensive socio-technical work seems to be little known outside academia, particularly since the 1980s [11].

Given the above, we echo Cobb’s paradox: "We know why projects fail, we know how to prevent their failure – so why do they still fail?" [30].

One factor is that health informatics and project management are not controlled professions with mandatory regulatory standards. Hence, the professional competence of some NHS HI project managers is limited to general IT project experience, which tends to bring the implicit but incorrect view that HI is "just another" industry sector. This has been another general criticism of the NPfIT “commodity” approach to HI. If anything, the unregulated nature of project management and health informatics means that their need for guidance is even greater than a formally controlled profession.

A converse aspect of the problem is information overload. There is simply so much guidance available on project and programme management in healthcare. In some respects this parallels medical practice, where for many years it has been recognized that practicing clinicians cannot be expected to keep track of every new piece of research evidence. Instead, important new knowledge is highlighted through summary publications, clinical guidelines and professional standards. In fact, the latency inherent in the medical profession is such that even solid research evidence only changes clinical practice slowly, patchily and when multiple interventions are used for reinforcement [38, 48].

By contrast, HI benefits and project management methodologies do not have a robust evidence base but are largely aspirational [42] or normative [49]. In fact there are legitimate grounds for caution with respect to HI, given the evidence that products are not equal [50], things can be made worse [51] and that the method of implementation can make all the difference to success or failure [34, 52].

At times HI is used instrumentally, beyond its functional purposes, to enforce “improvements” in ways of working as defined by the project sponsors [53]. This adds yet another layer of complexity and potential opposition, especially if the merits of the given changes are disputed.

Of course, the invidious effects of politics and commercial interests cannot be ignored in this context. The NHS is unavoidably part of the political game, given its funding and structure. A taxpayer-funded IT programme has legitimate expectations of optimum progress from both its political masters and its commercial suppliers. But when these interests seem to sacrifice good practice for the sake of apparent progress or simply meeting contractual milestones, then the taxpayers have sound reason for concern.

It must also be stressed that the anticipated benefits of planned change in healthcare systems, particularly when based on HI, are not deterministic. The inherent complexity of the socio-technical system in healthcare means that apparently minor changes can have profound unpredictable consequences [18, 37]. Indeed, it has been argued that any ICT “solution” actually adds complexity, and hence risk, by increasing the number of information and process interrelationships in the world [54].

NHS Connecting for Health (CFH), the English Department of Health agency responsible for NPfIT, publishes an “Organisational Readiness Assurance Guide” for HI planning [55]. This document seems to assume a linear model of change that comprises work process redesign, system deployment and establishing a new status quo. It does not address unintended consequences or complex adaptive change. Its subtitle “Are you ready?” reinforces the concept of the organization or individual as passively receptive to a “deployment”. It does suggest that new working practices and roles be subjected to an impact analysis and requires that both line managers and “stakeholders” agree to the new working practices and roles, but does not specify whether that includes the clinicians themselves. Arguably, its helpful direction on applying lessons learned from other HI implementations itself implies that the socio-technical perspective needs greater emphasis.

The genesis and focus of this paper has been on HI in the UK. However, the essential principles discussed here would intuitively seem rather general across the global HI domain.

At best, this proposed tool can offer a selection of key messages and maxims to practitioners. This is significantly less than a formal theory of HI programme management, but is arguably an improvement on the status quo.

Obviously, this proposed tool will not remove political or contractual pressure to impose systems to meet arbitrary deadlines. However, it could provide a mechanism to create or channel balancing feedback pressure within or upon the governance structures of HI programmes where participants have concerns that good practice may be compromised by inappropriate commercial or governmental timescales.

The suggested approach will be debated and the content and format tested with the UK Faculty for Health Informatics. If the tool seems to have face validity based on this expert consensus, the aim will be to apply it to current or recent HI projects to measure its utility. One extension may be to devise a numerically scored assessment rating. Further thought is needed about how in practice the rating would be conducted for it to be credible, and how any proposed remedial action would be validated. Further essential content for the checklist may be added from analysis of the recently published AMIA collection of case studies of failed HI projects [56].