Detail Data posed a series of written questions to the Northern Ireland Courts and Tribunals Service (NICTS) and received the following responses.
NICTS: Children Order data in relation to orders made is recorded on ICOS (Integrated Court Operations System) at the application level. The statisticians then extract this data at the application level and apply a weight to the data to report on the orders at the participant level i.e. to indicate all relevant parties involved. The orders historically have been reported at the participant level since 2007, however in light of these issues we are proposing to introduce a new methodology to report on orders made at the application level. This will allow the user to compare data consistently across the time series 2007 – 2015, and 2016 data going forward.
DD: You stated that during training in early 2014 it was highlighted to staff that when resulting proceedings in family courts that they must select all the relevant parties when entering a result. When exactly, following the training in early 2014, did it come to light that not all relevant parties were being entered when resulting proceedings?
NICTS: Prior to the training it came to light that there was inconsistency in the way in which different court offices resulted these cases and it was decided that all the relevant parties should be selected and this was built into the training.
DD: Can you explain in more detail what it means to have ‘all relevant parties selected’? Is this, for example, all siblings affected by the same order, own motion orders?
NICTS: As a result of the training all the parties to the proceedings should now be selected including the trust, all the respondents and other parties.
DD: What relevant information had not been entered?
NICTS: Prior to the training all the parties to whom the order applied had not all always been selected.
DD: Were final and interim orders being under or over reported as a result?
NICTS: Prior to the training both final and interim orders were being under reported at the participant level only.
DD: How long had this being going on?
NICTS: It is difficult to put an exact date on this, but the data (at participant level) has potentially been recorded inconsistently since April 2007, when the data was first recorded on ICOS.
DD: Is the 2014/15 data (from September 2014 to August 2015) provided to me correct?
NICTS: It would be possible to report on the August 2014 – September 2015 orders data, as recording practices would have been consistent across the period. However, if it was helpful we could provide data at the application level for the three years, which disregards which participants, have been selected, and looks purely at the orders being made. This is the proposed methodology we are looking at to report orders being made going forward.
DD: I had submitted two FOIs in relation to this data why was the change in practice not disclosed at any point?
NICTS: This change in practice following training and the affect it had on the recording of orders was not identified until the follow up questions to the second FOI were submitted.
DD: You stated the percentages below are all an accurate reflection of the data recorded on ICOS, however caution does need to be taken when interpreting the increases in interim and final orders made prior and post to 2014 as recording practices did change. What was the change in recording practice – what was previously being recorded and what is now being recorded?
NICTS: There has been no change in the content of the orders, however the change was to the selection of participants associated with the order and this was introduced to bring consistency in practice to all court venues. Previously participants to whom the orders applied were selected inconsistently, and now post the training all participants to whom the orders applied are being selected.
DD: You said following training in early 2014 it was highlighted to staff they must select all the relevant parties when entering a result but when in practice did this change in recording start taking effect?
NICTS: Training ran from the 21 January to 4 March 2014.
DD: What timeframe covers the previous recording practice – from what date to what date?
NICTS: April 2007 to 4 March 2014.
DD: In light of this discovery do you accept that some of the data recorded on ICOS is wrong?
NICTS: The information recorded on ICOS is correct as the orders are recorded at the application level; however there were inconsistencies in the practice of the selection of participants to whom the order applied. The data is recorded on ICOS at the application level, and it is the statisticians who then, after the data is extracted from ICOS, apply a weight regarding the participants involved to report on the orders at the participant level. Going forward, it is this methodology of applying the weight to the orders to report them at the participant level that will be removed.
DD: You stated the impact of this change in practice on the recording of orders within the Children Order business area has only now come to light. Did our enquiries bring the situation to light?
NICTS: Yes.
DD: You also stated as a consequence "we will be reviewing the methodology we use to report data on orders made, with a view to amending our data collection process and we would intend to publish a back series of data derived using this new methodology". When did/does that review get underway?
NICTS: The review has started with immediate effect.
DD: When will the amended back series of data be published?
NICTS: A period of consultation with our customers will commence in early April and we are endeavouring to have an amended back series of data using our proposed new methodology of counting the numbers of orders made regardless of the parties involved, available at the end of June 2016.
DD: What period will the back series of data cover?
NICTS: April 2007 to the end of 2015, with the new methodology being used in publications relating to 2016 onwards.