Hospitals star ratings and clinical outcomes in England

The English Department of Health has developed global measures of the performance of all NHS bodies.  Since 2000-1, the trusts get zero, one, two, or three starts to indicate performance. 

This rating may not reflect the effectiveness of clinical care measured in patient outcomes because of the lack of accurate routine data.  One exception is in adult critical care; the authors checked whether a hospital’s rating provided an indication of its clinical outcomes in such care. The distribution of star ratings for the 102 acute hospital trusts was associated with teaching status (university hospitals had more stars than non-university hospitals) but not size of its critical care unit.

Rating and crude mortality for critical care admissions were significantly associated: mortality in trusts with three stars was about 4% lower than in trusts with zero stars.  However, case mix of critical care admissions also differed considerably.  Rating was inversely associated with the mean age of critical care admissions.  The association between rating and hospital mortality was no longer significant when case mix differences were taken into account.

For adult critical care, star ratings do not reflect the quality of clinical care provided by hospitals.  Patients do just as well in a trust with no stars as they do in one with three stars.  The crude mortality data, used in the rating system, are misleading because they ignore the fact that higher rated trusts tend to be teaching institutions with patients who are less severely ill on admission to critical care units.

On might nor expect to find an association between the rating of the whole trust and the effectiveness of critical care.  Hospitals are complex organisations containing many services; performance across a hospital will not be uniform – a poorly rated hospital may contain some excellent services and vice versa.  Secondly, ratings are determined by a small number of process measures; outcome measures play only a small role and are based on scant poor quality data, which do not adequately account for case mix.

However if these findings reflect other areas of hospital care, the government is not yet fulfilling its ‘commitment to provide patients and the general public with comprehensive, easily understandable information on the performance of their local health services.  Outcome ought to be a principal concern alongside process indicators, such as waiting times and cleanliness; to fulfil its aim, the government needs to use specialised clinical databases

Ref: Web

Back to top

Hospital mortality monitoring groups

This BMJ article describes the need for monitoring of hospital mortality.

In September 2000 heart transplantation at St George’s Hospital, London, was suspended because of concern that more patients were dying than previously.  The newspapers reported that 80% mortality in the last 10 cases had been of particular concern because this was ‘more than five times the national average.”  The authors of this article tested these assumptions – that surgical results had been satisfactory but later became unsatisfactory – against numerical criteria.

A retrospective analysis of deaths after heart transplantation was made.  Seven methods were used: mortality above national average, mortality excessively above national average, test of moving average mortality, test of number of consecutive deaths, sequential probability ration test (SPRT), cusum with v-mask, and CRAM chart.  The national average mortality was not available, and a rate of 15% was used instead as the benchmark.

All 371 patients who received a heart transplant in the programme, 1986-2000 were studied. All methods provided evidence that the 30-day mortality had been high at some stage.  At the end of the series the average mortality, sequential probability ratio, and cusum tests indicated a level of deaths higher than the benchmark while the remaining four tests yielded negative results.

If the decision to test for outlying mortality is made retrospectively, in the light of the data, it is not possible to determine the false positive rate. 

Prospective on-site mortality monitoring with the CRAM chart is recommended as this method can quantify the death rate and identify periods when an audit of cases is indicated, even when data from other hospitals are not available.  A hospital mortality monitoring group can routinely monitor all deaths in the hospital, by speciality, using hospital episode statistics (HES) data and appropriate statistical methods.

Ref: Web

Back to top

Hand washing in hospital

Bandolier No. 73 examined the importance of hospital acquired infection to health services. In England it costs £1 billion a year, kills 5000 patients, and consumes the resources of 27 four hundred bed hospitals. Effective hand washing can reduce that substantially, though there are few reports of effects over long periods.

A long-term hospital wide study, demonstrating that an effectively implemented hand washing policy reduced hospital acquired infection by half and saving precious resources

The study was started in the University of Geneva Hospitals. Hand washing behaviour was monitored and the incidence of hospital acquired infection measured. At several times trained staff observed health professionals at pre-specified time periods throughout the hospital. An additional measure of efficacy was the amount of alcohol-based hand rub solution dispensed by the pharmacy.

Individual bottles of alcohol-based chlorhexidine solution were distributed, including specially designed flat containers so that individuals could easily carry their own supply. There was a series of grand rounds in individual medical departments.

The proportion of observed instances where necessary handwashing did indeed take place increased from 48% in 1994 to 66% by the end of 1997. Moreover, there was a change in behaviour, with hands being more often disinfected than just washed. New cases of MRSA fell by half The cost of the programme was estimated at £155,000 including direct and indirect costs. The authors conservatively assumed that 25% of the observed reduction in infections was due to the hand washing programme, thereby preventing 900 infections. At an average cost estimate of about £1400 per infection, their estimate of overall savings of £1,260,000 far outweighed the costs.
One of the main consequences of infection is bed blocking, and massive inefficiency in hospital systems. Reducing hospital acquired infection by half, has an important impact on improving the efficiency of bed usage

 
Ref Bandolier

Back to top