Health Systems and Policy Research

Keywords

Information; Readmission; Scotland; Mental health; Community mental health team

Introduction

Studies from developed countries UK, USA, Canada suggest that around 13-14% of patients are readmitted soon after discharge from acute psychiatric care [1-4]. Readmissions are disruptive and distressing for patients, their families and are benchmarked as an area for improvement nationally and internationally [5,6-8].

As part of developing more efficient services The Scottish Government developed a target for mental health services to reduce readmission rates [1,2]. The development of SPARRA was one of the commitments of Scottish Governments policy document -Better Health Better Care [2]. SPARRA was developed to support the Health Efficiency Access and Treatment (HEAT) readmissions target [9,10]. The Mental Health Division of the Scottish Government commissioned the Information and Statistics Division (ISD) to develop a statistical method to identify people at high risk of readmission, with the intention of providing this information to clinical services.

SPARRA is an algorithm for estimating risk of readmission to psychiatric units, based on psychiatric inpatient history in the 3 years proceeding the year of interest. This data is used by ISD to calculate individual patient risk scores, which are disseminated to National Health Service Boards. National Health Service Scotland (NHS Scotland) has 14 regional NHS Boards which are responsible for improving and providing frontline healthcare services.

SPARRA data was supposed to identify patients at highest risk of psychiatric readmission, so appropriate community resources in each of the health boards are put in place to provide care in community rather than an inpatient admission. The Table 1 [7,8] below details the factors predictive of readmissions as per SPARRA (MH).

Age

Number of Psychiatric Admissionsin the previous 3 years

Time since most recent admission

Total number of bed daysin the previous 3 years

Urban/Rural

Principal Diagnosis, in the previous 3 years

Table 1: Risk factors found to be predictive of readmission as per SPARRA MH).

There are 6 main predictor variables for readmissions. The most significant effect is that of the number of previous psychiatric admissions, higher the number of previous admissions the higher the likelihood of further psychiatric admissions. The next influential variable is the time since most recent admission – there is a clear decline in effect as the time gets further away from reference category. Diagnosis of Schizophrenia and Bipolar Disorder are strongly predictive of future psychiatric admissions. Bed days, age and rurality have moderate effects. The measure relies on hospital admission data [7,8]. The risk data sent out is for all risks, that is 0-100% risk of readmission. NHS Boards can choose which data to look at for example high risk (>50%) or medium risk (30-50%) of re-admission.

As the measure was intended to supply national data, on advice from ISD it was issued only when all psychiatric hospital admission records had been submitted, meaning that it was based on information that was at least six months old.

There was very limited uptake of the SPARRA (MH) data in reducing readmission rates and this study intended to establish why this was the case.

Method

A method based on the framework approach to qualitative research, was used. In this, ‘the research brief is commissioned, aims and objectives are highly focused and the researchers work with structured topic guides to elicit and manage data’.

Three of the NHS areas using the system were selected at random. Each was invited to identify a Community Mental Health Team that was using the data, and another which was not. A semi-structured interview schedule was developed and piloted on a volunteer service. Preliminary categories were identified, and the interview schedule revised. The revised interview schedule was then used to conduct telephone interviews with the six CMHT team leaders, encompassing those using and not using the measure.

Qualitative analysis, using content analysis, was used to identify themes in the data. Handwritten notes were kept of the meetings, including verbatim quotes, and then transcribed.

Two researchers (SJ and CS) reviewed the data independently and identified themes. These results were then compared, discussed a final set of themes agreed.

In quotations, the teams are identified by area, and as a user (U) or non-user of the data (N-U).

Results

All of teams received SPARRA data. Five of the six teams received data directly, and the sixth received it via its team psychologist. One of the teams that did not use SPARRA MH (Area 2, N-U) had used it in the past, but had ceased to make regular use of it. All six teams reported that the data was read, even if it was not then used within the team.

Purpose

When asked to describe the purpose of SPARRA MH, all six teams identified it as containing information on risk of readmission:

“(It is) linked to reduce hospital re-admission (and) keep people at home… reduce the length of stay in in-patients’ (Area 1, U)‘tracking those at risk of readmission to hospital” (Area 3, N-U).

“Yes… people likely to be re-admitted to hospital” (Area 2, N-U).

None of the six teams reported knowing of national communication on the measure. All the teams using the information in practice reported that information on SPARRA MH and its use was communicated locally, at least to some extent.

“Yes, from the in-patient manager and the bed manager” (Area 1, U).

“It is variable.” (Area 2, U)

Two of the three areas not using the measure reported that this information was not communicated well locally, despite being in the same area as teams that did report local communication. The third team, which had stopped using the measure in practice, did report that there had been clear local communication.

The Measure

Some teams accepted the measure.

“The validity of the SPARRA MH itself is good” (Area 1, U).

“Happy with (the) principle of it” Area 3, N-U).

This was not universal, however, and there was evidence of uncertainty about its construction:

“(I am) unaware of how it is compiled…need to understand what is behind it… (I would) like to have a brief summary of the tool in the first place – how it works…if people knew what went in to SPARRA, and how it was measured, this would help those using it” (Area 3, U).

“SPARRA data is not robust enough at times, and can’t be used on its own” (Area 2, N-U).

Access and Presentation

There was concern about the presentation of the measure, even from one of the teams that reported they were using it.

“…unable to understand the SPARRA output – on one piece of paper would be helpful….(there are) six pieces of paper for each patient” (Area 1, U).

Accessing the measure was not always easy, “(there is) limited access via … shared drive, ‘though this is being changed now.” (Area 3, U). Another team which used the measure had adapted the output to make it easier to use in practice, “to make it practical to read, we have to delete some fields” (Area 2, U).

Other comments indicated that teams experienced problems in understanding the information, or wanted changes to the lay out.

“(Needs) clearer presentation: don’t know who’s who” Area 3, N-U).

“Cumulative totals could (be) broken down easier” (Area 1, U).

“Admin would be helpful in deciphering the information sent” (Area 3, N-U).

Use in Practice

In one team, the data was used very regularly, ‘on Fridays and at business meetings’ (Area 1, U). Teams often felt that the people identified in MH SPARRA were people of whom they were already aware.

“All the names that we would expect are contained within it” (Area 2, U).

“Nothing we did not know before” (Area 3, N-U).

There were some advantages reported.

“(No new information) but good as a reminder for patients who are at risk of re-admission. It is good to identify patients which might at times (be) forgotten after discharge” (Area 2, N-U).

“(We were) probably unaware of bed days and specific stats. It has made clear to us information on these patients, and formalised it well” (Area 1, U).

“..(the) team looks at those with increased input, decides on possible use of assertive outreach and crisis (services)…discusses those discharged from hospital” (Area 1, U).

“(We) sat as a team to look at those with increased risk. (It) focuses you on thinking about patients when we sit down and discuss the ones contained within SPARRA MH” (Area 2, U).

“Helpful to have a measure around” (Area 3, U).

The team which had used the measure in the past reported that,

“SPARRA data is not considered on its own, but is used with other patient information.” (Area 2, N-U).

Suggestions for improvement focused on making the information more current:

“Should be proactive, and used in that regard” (Area 3, N-U).

“More real-time data provided, in up to date fashion” (Area 2, U).

Some comments on areas for improvement suggested that they related to internal team structures,

“Involvement of medical staff” (Area 1, U).

“Should not just be sent to nurses!” (Area 3, N-U).

All three of the teams that had been identified as SPARRA MH users reported that its use affected treatment plans to some extent. In some cases it also seemed to be delivering functions other than those planned.

“Useful to see what patients of ours are using in-patient beds” (Area 1, U).

“It gives us support for our work: we are doing what we should be doing” (Area 2,U).

Discussion

Teams using and not using the measure all received the measure, either directly or via one team member. All of the team contacts were able to explain what the measure was intended to describe, and all teams reported that information on the measure and its use was available locally.

The distinction between users and non-users proved to be less clear cut than expected. All six teams received the data in some way, and in all six at least one team member read the information received. In one team which had used the information in the past, but now identified itself as not using the data, comments suggested that the data was used at times.

There was evidence of uncertainty on how the measure was produced, and how it should be interpreted, in both users and non-users. The presentation was often regarded as unhelpful, and at least one team had simplified the data internally. The time delay in issuing data was a concern for both using and non-using CMHT’s.

For teams which made use of it, the data seemed to fulfill several functions. It helped to identify who had been in hospital, as well as who might be at risk of readmission. It acted as a prompt for review, and in one case it was seen as providing reassurance of good practice [11].

Previous work on the use of measurement in mental health services suggests that introducing measures in to practical use can be challenging [12,13]. Innovation often spreads in an uneven manner [14], and Damanpour points out that change does not happen in a vacuum [15]. The education and training of staff, managerial attitudes, internal and external communication and technical knowledge are all likely to affect the extent to which an innovation is embraced in practice.

Ganju comments that, in the United States, the intention that ‘performance measures would inform and improve services and supports in a quality improvement framework has been largely unfulfilled’ [16]. In this Scottish initiative, the measure was developed by a central group, and piloted in two local areas (7,8). The pilot was concerned with the clinical predictive value of the measure, rather than the factors that might affect its use in practice.

This evaluation demonstrates that organisational and contextual factors were important. Community Mental Health Teams, even when using the measure, often reported uncertainties about how the measure was developed and constructed. While there was local dissemination of the data, there was not a corresponding distribution of information about the measure itself.

The pilot sites which had used the measure had access to the team that developed the process by which it was produced. The teams contacted in this work sometimes found the data difficult to interpret, and expressed a need for support and guidance on its use and interpretation. The support for the pilot sites was not replicated in the wider dissemination of the model. Having measures that are clearly aligned with good practice is important to clinicians [17], and it was apparent that some teams saw little practical value of this measure, perhaps influenced by the consistent view that there had been no useful dissemination of national information on the measure.

Even services that did not use the measure sometimes had views on how it could be improved. The face validity of the measure was important, but there was also an impact of the presentation, and of the time lag in the information. The pilot period had concentrated on technical aspects of the measure in clinical practice, but it is apparent that time could have been spent on developing an improved understanding of how the measure would be used in clinical practice, and of the optimum presentation of the data.

This study was a small qualitative evaluation, and cannot demonstrate representativeness across Scotland. It was a purposive sample intended to obtain qualitative information from teams in the same area using and not using the data in clinical practice. While the small size of the sample limited extrapolation of the findings, it did give the opportunity for team members to explain their views, and to describe how the measure was used, or not used, in practice.

What this study adds

This evaluation suggests that the immediate uptake of a new national data source, intended to support clinical practice, was limited by the systems which supported its implementation. The measure was disseminated to all areas, but in the absence of a clear national framework, and of accessible information on its development and use, its adoption was limited. In order to maximise the return from this type of initiative, greater attention is needed to how a measure will be used in practice, what support and information is required, and in what form information should be presented.

Acknowledgments

The authors would like to thank Dr. Moira Connolly and Dr. Denise Coia for their advice.

17210

References

1. (2003) Commission for Health Improvement. Psychiatric Readmissions(Adults of Working Age).
2. Leslie DL, Rosenheck RA (2000) Comparing quality of mental health care for public-sector and privately insured populations. PsychiatrServ51: 650.
3. (2012) National Association of State Mental Health Program Directors Research Institute.
4. (2011) Health Indicators. Canadian Institute for Health Information and Statistics Canada.
5.  Hermann RC, Mattke S, Somekh D, Silfverhielm H, Goldner E, et al. (2006) Quality indicators for international benchmarking of mental health care. Int J Qual Health Care 18: 31-38.
6. Rumball-Smith J, Hider P (2009) The validity of readmission rate as a marker of the quality of hospital care, and a recommendation for its definition. N Z Med J 122: 63-70.
7. (2009) Information and Statistics Division Mental Health Programme. SPARRA Mental Health: Scottish Patients at Risk of Readmission and Admission (to psychiatric hospitals or units): A report on the development of SPARRA MH and recommendations for its use. Edinburgh.
8. (2008) SPARRA Mental Disorder: Scottish Patients at Risk of Readmission and Admission(to psychiatric hospitals or units) A report on the work to identify patients at greatest risk of readmission and admission to psychiatric hospitals or units in Scotland.
9. Smith J, Firth J (2011) Qualitative data analysis: the framework approach. Nurse Researcher 18:52-62.
10.  Kimberly J, Cook JM (2008) Organizational measurement and the implementation of innovations in mental health services. Adm Policy Ment Health 35:11-20.
11. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C,et al. (2009) Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 36:24-34.
12. Greenhalgh T, Glenn R, Macfarlane F, Bate P, Kyriakidou O (2004) Diffusion of innovations in service organizations: Systematic literature review and recommendations for future research. Milbank Q 82:581-629.
13. Damanpour F (1991) Organizational innovation: a meta-analysis of determinants and moderators. Acad Man J 34:555-590.
14. Ganju V (2006) Mental health quality and accountability: the role of evidence-based practices and performance measurement. Adm Policy Ment Health 33:659-665.
15. Hermann RC, Chan JA, Zazzali JL, Lerner D (2006) Aligning Measurement-based Quality Improvement with Implementation of Evidence-based Practices. Adm Policy Ment Health 33:636-645.