Chapter 35. Data - reading the signals

Evidence obtained

The Inquiry obtained statements from a number of organisations and individuals concerned with the collection and analysis of data about neonates. These include:
1. Professor Richard Feltbower, Professor Elizabeth Draper and Dr Sarah Seaton on behalf of the Paediatric Intensive Care Audit Network (PICANET)¹
2. Professor Modi on behalf of the National Neonatal Research Database²
3. Professor Karen Luyt on behalf of the National Child Mortality Database³
4. Dr Edile Murdoch on behalf of the Maternity and Neonatal Outcomes Group⁴
5. Professor Marian Knight MBE on behalf of MBRRACE-UK.⁵ Professor Knight also gave evidence to the Inquiry.
The Inquiry also obtained written and oral evidence from Sir David Spiegelhalter.⁶ Sir David is Emeritus Professor of Statistics at the University of Cambridge. His work has been principally as a medical statistician, focusing in particular on methods for monitoring and comparing performance in health services. Amongst other important and far-reaching projects, he led the statistical team on the inquiry into deaths of babies with congenital heart disease at the Bristol Royal Infirmary, and worked on the statistical team for the Shipman Inquiry. He was also a frequent presence on radio and television during the Covid-19 pandemic in 2020 and 2021.

Retrospective audit and real-time prospective monitoring

In his first statement, Sir David drew attention to the value of “formal statistical process control” as a means of monitoring “adverse clinical outcomes” and detecting “clusters of failures”.⁷ He described two broad types of system: (a) retrospective audit and (b) real-time prospective monitoring. Retrospective audit involves data collection and aggregation over a period of time, comparison of centres, and results being fed back to centres for further analysis. The advantages of such monitoring include quality control of data and centralised scrutiny. The disadvantages are that retrospective audit involves historical data, and is not designed to detect “clusters of failures”.⁸ Real-time prospective monitoring, meanwhile, is based on the continuous monitoring of accumulated experience. Sir David describes the advantages of real-time prospective monitoring as a “strong feeling of local ownership”,⁹ the use of accumulated data, and rapid response, with formal monitoring systems to trigger investigation. The disadvantages are that it requires buy-in from centres, rapid access to high-quality data, care in setting parameters and thresholds, and explanation and acceptance of more complex monitoring.¹⁰
In practice, real-time prospective monitoring may only report monthly or even quarterly, but still uses methodology that monitors events without regard to annual boundaries. In his second statement, therefore, Sir David said that it might be clearer to label the two types of system as (a) fixed-period monitoring and (b) continuous monitoring.
Sir David spoke highly of the PICANET system, which is used in paediatric intensive care units in various hospitals. Units submit data to PICANET’s online system. In PICANET’s written statement to the Inquiry, Professor Feltbower explained that PICANET “undertakes outlier analysis of risk-adjusted excess mortality compared to the expected mortality based on disease severity at the time of admission using funnel plots”.¹¹ Professor Feltbower said that any provider who falls above the upper line of the funnel plot is considered a potential negative outlier, and this triggers an established management process set out in PICANET’s Outlier Policy.
The PICANET system combines fixed-period and continuous monitoring. The results from retrospective audit are provided to units in an annual report that contains data from the previous three-year reporting period.¹² In real time, the system produces resetting sequential probability ratio test (RSPRT) plots, which show the odds of mortality over variable time periods alongside two thresholds. PICANET reviews the plots quarterly in comparison with the thresholds and categorises each unit as either ‘satisfactory’, ‘cause for close monitoring’ or ‘cause for concern indicating internal review’. Professor Feltbower tells us that “RSPRT plots can be accessed at any time by each paediatric intensive care unit and PICANet”, with units supported by guidance to assist them in identifying and responding to potential issues with the quality of care in a time-sensitive fashion.¹³

The role and limitation of monitoring systems in identifying criminal acts

In oral evidence, Sir David told the Inquiry that the events at the Countess were “exactly the situation where real-time monitoring … is so vital”.¹⁴ It is common sense that, when it comes to the early detection of a member of staff causing deliberate harm, it is real-time monitoring or continuous monitoring that will be of the most value.
The case of Harold Shipman provides a stark example. In 2005, Dame Janet Smith DBE found in the Report of The Shipman Inquiry that the GP Shipman killed approximately 250 patients. A major reason for his popularity was his willingness to visit his elderly patients at home. Mr Justice Forbes remarked when sentencing Shipman that none of his victims had realised that Shipman “brought death; death which was disguised as the caring attention of a good doctor”. Sir David was part of the statistical team for the public inquiry that examined how events might have been monitored at the time, and whether Shipman’s crimes could have been stopped earlier.
Sir David explained in his oral evidence that, if a continuous monitoring system had been in place at the time, it would have sounded “a very strong alarm” after Shipman had killed 40 people.¹⁵ Sir David is clear about the significance of a very strong alarm (or any alarm) and the limitations of statistics. He told the Inquiry that a “statistical monitoring system cannot say why something has happened … a signal can only indicate that someone should look carefully at what is going on”.¹⁶
In 2022, Dr Kirkup’s investigation into maternity services in East Kent led to the publication of his report, Reading the Signals. Dr Kirkup’s first recommendation was the establishment of a taskforce “to drive the introduction of valid maternity and neonatal outcome measures capable of differentiating signals among noise to display significant trends and outliers, for mandatory national use”.¹⁷
This recommendation led to the creation of the Maternity and Neonatal Outcomes Group. In a statement to the Inquiry, Dr Murdoch stated that the group’s task was to develop a tool that could identify signals about potential critical safety issues in maternity care that could lead to adverse outcomes.¹⁸
Dr Murdoch describes a data-driven early warning system in this way:
“Safety signal systems work through monitoring real time changes in the trends of defined critical safety outcomes. A signal prompts an early critical review to understand the causes of the signal change. It is the subsequent assessment and review that will identify if there are safety issues to be acted upon [emphasis added].”¹⁹
As Dr Murdoch says later in her statement, “[a] safety signal system demonstrates unusual changes in signals but cannot explain why the signal has changed [emphasis added]”.²⁰ In the course of insightful oral evidence, as before, Professor Knight similarly pointed out the need for narrative-based medicine alongside statistical analysis. She told the Inquiry: “[W]e have to have both the statistics and the stories [emphasis added].”²¹

MOSS and MBRRACE-UK: what data do they collect?

The safety signal system being developed by the Maternity and Neonatal Outcomes Group is called the Maternity Outcomes Signal System (MOSS).^* As with any data system, MOSS has set parameters in terms of the data it is analysing. The data used by MOSS is taken from that gathered by MBRRACE, together with data from the Neonatal Data Analysis Unit in relation to hypoxic ischaemic encephalopathy at birth. The agreed parameters for MOSS exclude pre-term babies (i.e. those born before 37 weeks’ gestation) and those who died more than 28 days after birth.²² Accordingly, in its current form it would not be capable of capturing or identifying any anomalies in relation to the deaths of Baby A, Baby C, Baby E, Baby I, Baby O or Baby P, who were all born pre-term.
In her statement to the Inquiry, Dr Murdoch said: “While there may be benefit in a neonatal outcome signal system, this was not part of Recommendation 1 and based on our current knowledge, there is a good tool already available (the MBRRACE-UK real time data monitoring tool).”²³
MBRRACE undertakes surveillance of all stillbirths, late foetal losses and neonatal deaths (up to 28 days of age).²⁴ Accordingly, all the babies named on the indictment who died, except Baby I, fell within the parameters of the data that MBRRACE considers.
Reports are made online to MBRRACE, with responsibility for reporting the deaths sitting within the Trust in which the death occurs. The data provided is analysed and then presented as both a ‘crude mortality rate’ and a ‘stabilised mortality rate’ or a ‘stabilised and adjusted mortality rate’.

MBRRACE data signals in respect of neonatal deaths at the Countess from 2015 to 2016

Professor Knight provided written and oral evidence to the Inquiry. Two figures within her written statement (reproduced below as Figures 4 and 5) set out the crude mortality rates and stabilised and adjusted mortality rates for babies born at the Countess between 2013 and 2016.²⁵ Professor Knight said that the crude mortality rate reflects the number of deaths over the total number of births in a year.²⁶ The stabilised rate considers random variation in the rate.²⁷ She explained that, because perinatal deaths are relatively rare, the numbers are small. This means there can be random fluctuation in the data, which results in big changes in the rate.²⁸ The adjustment rate, meanwhile, considers differences in the population of women giving birth and the population of babies at different hospitals based on national statistics.²⁹

Figure 4: MBRRACE-UK chart – crude mortality rates for babies born at the Countess of Chester Hospital NHS Foundation Trust at 24 weeks or later gestational age, 2013 to 2016

Figure 5: MBRRACE-UK chart – stabilised and adjusted mortality rates for babies born at the Countess of Chester Hospital NHS Foundation Trust at 24 weeks or later gestational age, 2013 to 2016

Source: Witness statement of Prof. Marian Knight MBE INQ0006757/8/paras 28-30, 20/12/2023

Professor Knight explained to the Inquiry that in 2015 and 2016 “the rate at the Countess of Chester for neonatal deaths [was] more than 10% higher than the average for hospitals with similar characteristics in terms of the neonatal care and their birth population”.³⁰ She characterised this as a signal that the Countess “should be reviewing in detail the deaths of the babies that occurred in their care”.³¹ Likewise, Sir David said of Figures 4 and 5: “[W]e are particularly interested in red, which means they are estimating it’s 10% higher than the average for their — for their tier for their group. And so for 2015 it concluded more that it was red, more than 10% higher; 2016 up to 10% higher.”³² This placed the Countess as the highest in its tier group, which Sir David stated “would be sufficient to generate a signal and alert warranting investigation”.³³
Before he gave evidence, Sir David was sent a document prepared by the Inquiry legal team titled ‘Countess of Chester Evidence about the Number of Deaths on the Neonatal Unit Pre-2015’.³⁴ Sir David considered the numbers of recorded deaths from 2010 to 2016. He explained: “So the number of deaths between 2010 and 2014, 1, 3, 3, 2, 3, it actually shows surprising consistency. I would have expected more variability … Anyway, it went up to 8 in 2015.”³⁵ Sir David concluded:
“[T]he probability of getting eight or more deaths in 2015 I assessed to be around 0.008 … that would generally be considered sufficient to trigger an alert signal, someone should look at this locally. But not extreme enough to be considered an outlier and I think this is very useful to put this in perspective.”³⁶

There was no obvious explanation for the unusual pattern of deaths in 2015 that the data highlighted, and there was a requirement to understand and investigate each death.
However, the figures provided by neonatal units and hospitals for the year 2015 were analysed and reported by MBRRACE in June 2017.³⁷ The figures provided for 2016 were reported in June 2018.³⁸ In oral evidence, Sir David emphasised this “substantial delay”.³⁹ Leaving aside for a moment what they were told by medical staff about neonatal deaths at the hospital, in 2015 and 2016 it follows that neither the executives nor the Board at the Countess were aware of the need to investigate neonatal deaths as a result of MBRRACE’s monitoring system. However, by June 2017, when the MBRRACE 2015 report was received (and the unusual rise to eight neonatal deaths was identified), they were.

Current reporting and viewing arrangements

In oral evidence, Professor Knight explained why MBRRACE’s monitoring system takes the time that it does. MBRRACE receives data from a Trust about the deaths of babies in a particular year at the end of that year. It then cross-checks the information provided with data available from the Office for National Statistics. MBRRACE then sends the cross-checked data back to the Trust for data confirmation. Once confirmation has been received, MBRRACE produces stabilised and adjusted mortality rates and prepares its own report. In 2015 and 2016, this process took 18 months; it now takes 14 months.⁴⁰
In 2019, MBRRACE provided all NHS Trusts and Health Boards with access to a real-time or continuous data viewer, enabling immediate and ongoing monitoring of the stillbirths and neonatal deaths reported to it. In her written evidence, Professor Knight states that this provides a means for responding immediately to any concerning trends.⁴¹ It permits a user to log on and to look at the most recent figures for their hospital. It enables comparisons with its own previous data to be made. The system is reliant upon timely data being provided by each Trust or Health Board in respect of all baby deaths.
Professor Knight informed the Inquiry that MBRRACE has no power to mandate that organisations use this tool.⁴² She described use of the tool and monitoring of the data as variable. Some organisations use it several times a week; others once every couple of months.
In September 2021, the Maternity Incentive Scheme was introduced. This scheme includes requirements to notify MBRRACE of all neonatal deaths, stillbirths and foetal losses within 7 days, and to provide additional surveillance data within 30 days. Professor Knight explained that each hospital has a lead neonatal reporter and a lead maternal reporter who are tasked with reporting deaths via the online notification system.⁴³ Since the introduction of the Maternity Incentive Scheme, Professor Knight says “we now see nearly 100% of deaths notified within that time”.⁴⁴
Dr Kingdon (former President of the RCPCH) gave evidence to the Inquiry about the difficulties experienced by clinicians when entering data, because the various hospital systems do not all communicate with each other.⁴⁵ When asked about the practicalities, Professor Knight confirmed that, although many hospitals use the electronic patient record system, there is no universal system in the NHS. This makes it “complex” to automatically extract data from hospital systems.⁴⁶ Thus, MBRRACE uses a manual system, which Professor Knight acknowledged may be a burden for those reporting the data.⁴⁷ Professor Knight also confirmed that there had been a move away from the BadgerNet system, saying “[n]ot everybody uses BadgerNet any more”.⁴⁸
Sir David was told that the Inquiry had received evidence that the BadgerNet system was no longer being used in some hospitals because other systems could not communicate with it. He lamented the “lack of [interoperability] of medical systems” and labelled it as “an utter disaster in this country”.⁴⁹

Conclusions and recommendations

I have reflected upon the burden of reporting in respect of two monitoring systems that have an impact upon babies and neonates in hospital. In the end, it is not for me to decide whether two systems are necessary. MBRRACE’s remit involves pre-term and term babies. MOSS’s remit involves term babies and babies with severe brain injuries. Each monitoring system has set its own parameters with specific, clinical purposes in mind. They have each been designed to identify areas of risk to babies and to highlight where there is any need for investigation. Where there is no clear agreement that one system can achieve the work of both, as appears to be the case, the need for both remains.
Professor Knight stated that it was a “worry” that hospitals will have two different signalling systems; that it may be “confusing” and too many signals could mean “we are not going to be able to see … the most important signals amongst the noise”.⁵⁰ Sir David commented that it “remains to be seen how [the systems] might complement each other”.⁵¹
Without data monitoring in respect of babies in hospital, trends and areas of safeguarding concern may be difficult to detect. The collection, analysis and use of statistical monitoring tools is important in the context of highlighting the need to investigate events. Sir David’s evidence in respect of Shipman’s crimes demonstrates that monitoring tools can identify unusual patterns of deaths and prevent ongoing criminal acts. Despite the numbers being very much smaller, the MBRRACE system also raised a signal in respect of the number of neonatal deaths at the Countess in 2015, and the need for further, rigorous investigation of those deaths.
To be its most effective in a healthcare setting, data in respect of babies must be accessible in continuous time and should be analysed objectively and dispassionately, with proper regard for the protection of babies in hospital from risks of harm. Healthcare professionals need to understand the importance of data, and the damage to safeguarding if collection of data is slipshod or not prioritised. If data is difficult to input, and its analysis is not relevant or timely, it can too easily appear tedious or pointless for those clinicians who are required to input it.
It is not possible for me to make findings upon the evidence I have heard about the specific computer systems or technology available in hospitals to ensure the smooth running of the MBRRACE and MOSS data-monitoring systems. It will be obvious to all that the ability to enter data and to retrieve it should be as simple and time-effective as possible. Systems need to be interoperable. I have heard evidence that a software patch can be put in place in the short term (within three months) to enable this.⁵² On behalf of the NHS, it was said there are “complex data questions” and “there’s not a timetable for it because it’s a national issue”.⁵³ It is my view that it is urgent precisely because it is a national issue.
I recommend that NHS England and/or DHSC undertake to provide by 31 March 2027 a clear and timed route to ensuring that computer systems are harmonised across the NHS, so that by December 2028 data relating to babies and neonates in hospital may be entered and reviewed in a timely manner and on a continuous monitoring basis. The timescales are long because of the cumbersome procurement process and the need to get this right first time. There have been too many wasteful NHS technology experiences.
Following commencement of the Data (Use and Access) Act 2025, both DHSC and NHS England now point to a future mandatory information standard for neonatal care as the solution to the lack of interoperability between data systems, but I have received no timeline for the development nor for the implementation of the standard. In May 2026, I was informed that the mandatory information standard for neonatal care remained at an “early stage” (with discovery work completing at the end of June 2026). In this correspondence, it was also reiterated that the Single Patient Record timeline set out in the 10 Year Health Plan still stands. The first stage of this started in April 2026, but again there was no confirmed timeline for the inclusion of neonatal records within the Single Patient Record.
This is not good enough. The position is urgent. Assuming the evidence I received about the software patch was accurate, that work must be completed to achieve interoperability. It is important that this work on the development of a mandatory information standard with requirements for interoperability between systems is taken forward and implemented as soon as possible.
Professor Knight suggested that training in the interpretation of the data produced by MBRRACE is necessary. MBRRACE has created training materials and has worked in partnership with MOSS to develop a governance process around the use of the real-time data viewer.⁵⁴ Professor Knight also suggested that there should be a pre-determined route to senior management in the event of any concern about trends or patterns, with a view to an action plan being developed.⁵⁵ I agree.
I therefore recommend that hospitals should name a lead reporter with responsibility for inputting data regularly (at least weekly) and logging into real-time data viewers on neonatal and maternity units. Whilst the lead reporter does not necessarily need to be a clinician, they should be someone trained to understand and interpret the available data.
Furthermore, I recommend that, save where data requires immediate action, in which case a report should be made to the Medical Director and the Board immediately, the lead reporter should report to Trust Boards at least every six months and advise (a) whether any alerts or a need for investigation have arisen and (b) the investigation or action plan in place. This should enable Trust Boards to address whether any safety issues identified through monitoring systems are being adequately interrogated.

Footnotes

* Its standard operating procedures were updated on 25 November 2025; see NHS England, ‘Maternity Outcomes Signal System (MOSS) standard operating procedures’, 25 November 2025.

Endnotes

Cookies on The Thirlwall Inquiry