So, this week Acas published yet another ‘independent’ research report purporting to show how wonderful Acas is. It’s a dense and lengthy tome – 106 pages, not including the copy of its survey questionnaire – and, to convince you of its authoritativeness, it’s packed full of phrases like “multivariate analysis”, “post-stratification weight”, “rebasing exercise”, “potential non-response bias”, and “cognitive testing”.

But the main thing Acas wants you, the general public, and government ministers to take away is that “seven out of ten (71%) of [potential employment tribunal] claimants avoided going to [tribunal] after receiving help from Acas”.

If we are of a strong constitution, we can imagine the junior injustice minister, Shailesh Vara MP, doing a little jig around his office when he was informed of this finding. For, why worry about the justice-denying impact of employment tribunal fees, if seven out of ten workplace disputes are happily resolved by Acas? For free! It can only be a matter of time before the 71% figure finds its way into a speech by Mr Vara or his less intellectually-challenged but equally amoral boss, Michael Gove.

Except that, for all the post-stratification weighting, logistic regression and multivariate analysis, that 71% figure is simply rubbish.

To see why that is so, let’s take this infographic, as tweeted by Acas (as well as the more detailed chart on page 72 of the report – see further below).

That’s right: 31 + 17 + 22 + 1 = 71. In other words, if you took any 100 cases as they entered the early conciliation process introduced in April 2014, 31 cases would be settled via that early conciliation process – let’s call this Stage A. According to the more detailed chart on page 72 of the full report (see below), among those 31 cases would be 29 that result in a COT3 (Acas) settlement, and two that result in a private settlement, thanks to the involvement of Acas. And, of the remaining 69 cases, 17 (25%) would not be settled but would nonetheless not proceed to an employment tribunal claim, thanks to the involvement of Acas.

Then, of the 35 cases (50% of 69) that proceed to an employment tribunal claim, 22 (a stonking 64%) would be settled without reaching a tribunal hearing, either through post-ET1 conciliation by Acas (57%), or by private settlement (7%) – let’s call this Stage B. And one employment tribunal claim would otherwise be withdrawn before reaching a tribunal hearing, thanks to the involvement of Acas. And 31 + 17 + 22 + 1 = 71.

The first thing to say here is that, even if we accept these figures at face value, Acas did not prevent 71% of cases “reaching tribunal”, as the above tweet disingenuously suggests. It only prevented 48% of cases ‘reaching tribunal’. The other 52% either went to tribunal, or Acas cannot claim credit for them not having done so. The 71% figure relates to those cases that didn’t go to a full tribunal hearing. But it was always the case that only a minority of all ET cases went to a full hearing.

However, we cannot accept the research figures at face value. For they bear little if any relation to what happens in the real world – that being the place where workers who have been mistreated or exploited by their employer have to try and find justice (if they can afford to do so). It would be lovely if such workers could simply ask Acas’ independent researchers to resolve their dispute for them, through a process of logistic regression, but sadly the real world doesn’t work like that.

In the real world, we know the actual Acas settlement figures for Stages A and B, and they are not 29% and 57%. According to the early conciliation updates published by Acas, the actual figure for Stage A in 2015/16 was just 17%, not 29%. And, according to the most recent set of HMCTS quarterly tribunal statistics, the actual proportion of tribunal claims resolved by means of an Acas settlement (at Stage B) was 32% in Q1 of 2015/16, 35% in Q2, and 40% in Q3 – all somewhat short of 57%. Let’s run with 35%.

If we plug those figures into Acas’ suspect formula, we find that just 38% of cases (17 + 25% of 83) were prevented by Acas from ‘going to tribunal’, and that just 53% (38 + 35% of 83/2), not 71%, were prevented by Acas from ‘going to a full tribunal hearing’.

However, even those figures assume that 50% of the cases not settled by Acas through early conciliation proceed to a tribunal claim (i.e. 50% of 69 = 35/100). And, again, we know from the early conciliation updates published by Acas that, in the real world, the actual figure in 2015/16 was just 18%, not 35%. If we also plug that figure into Acas’ suspect formula, we find that just 44% of cases were prevented by Acas from ‘going to a full tribunal hearing'(38 + (18 x 0.35)). Which is a bit different to 71%.

Sure, we can boost that 44% figure a bit, if we accept the research findings that 2% of cases resulted in a private settlement thanks to the involvement at Acas at Stage A, and 7% at Stage B, but it doesn’t make a huge difference. Doing that produces a figure of 40% for ‘prevented by Acas from going to tribunal’, and a figure of 48% for ‘prevented by Acas from going to a full tribunal hearing’ (by Stage B, we’re talking about an extra 7% of just 18 cases, which works out at just over one case). And 48% is still very different to 71%.

In short, the headline figures that emerge from the research study’s unrepresentative sample are way out of whack with the real world. And, as the unrepresentative sample grossly overstates the influence of Acas in resolving cases, the rest of the research findings – such as those about workers’ satisfaction with Acas conciliation, and the importance of fees in workers’ decision-making – must also be taken with a large dose of salt. If you’ve somehow managed to create a sample in which 71% of cases were happily resolved by Acas, the chances are that sample will say they pretty much love Acas.

On Twitter this week, a number of people – including Michael Ford QC and the good people at Thompsons Solicitors – have noted the research finding that ‘20% of withdrawals [of an ET claim] were due to the hearing fee, mostly because the worker could not afford the fee’. However, if my criticism above is correct, the research findings greatly overstate the (positive) role of Acas, and understate the role of fees in workers’ decision-making. That is, the actual figure may well be considerably more than 20%.

All publicly-funded bodies seek to overstate their value, so as to try and impress their funder (the relevant minister). But, to my mind, mistreated and exploited workers deserve better than this from Acas.