The Sally Clark case

The case of Sally Clark is an instructive one when looking at how courts (mis)handle statistics. Wikipedia has a useful summary with good links to some of the problems that come up from it.

The essence of the (statistical) problem was that one of the Prosecution expert witnesses (Roy Meadow) said that the chance of a SIDS (cot death) was 1 in 8,543. Sally Clark's two children had died from cot death. He then took from that that the chances of both children suffering a cot death was 1 in (8543*8543), in other words 1 in 72,982,849 and on that basis the inference was 1 in 73 million is so unlikely that you can discount cot death and conclude that this was murder.

This is completely and embarrassingly wrong. There are two major problems.

The first is that to find the probability of two events happening by multiplying the individual probabilities is only valid if the two events are 'independent'. For example, the Probability of getting a head if you toss a fair coin [P(H)] is ½. The Probability of getting a one if you roll a fair dice [P(1)] is 1/6.

The Probability of getting a head and rolling a one if you toss a coin and roll a dice is ½ x 1/6= 1/12. In other words, for two independent events P (A and B) = P(A) x P(B).

So multiplying the odds is only valid if there are no environmental or genetic factors present in cot deaths. This seems on the face of it unlikely (and the full report that the stats came from indicate that there are environmental factors).

Roy Meadow was made the subject of a complaint to the GMC. This ended up in the Court of Appeal were it was concluded (by a majority) that he was not guilty of Serious Professional Misconduct.

Roy Meadow is obviously an educated man. He is a man of science. Whilst he is not a statistician, this is not complicated in any way. It’s basic maths. One would have thought that someone who is a doctor should have seen this coming a long way off.

But, whilst he got rightly criticised for this, one wonders why nobody in court from amongst the lawyers picked up how obviously wrong this was? Anyone with a GCSE in maths should have spotted the error and asked some questions as to how it was right to assume independence.

The second problem is that is another example of the Prosecutor’s Fallacy (most often seen in DNA evidence). 73 million sounds a large number. 1 in 73 million sounds like very long odds indeed, doesn't it? But even if that figure is correct (which it almost certainly isn’t) what does that tell us?

Because we are very bad at reasoning with numbers, it’s tempting to conclude that the chance of Sally Clark being guilty is 1 in 73 million. It needed to be explained carefully why this wasn’t the case.

What it actually told us is the chance of a person, chosen at random in the UK, suffering two cot deaths is 1 in 73 million. But, the jury was looking at one person where her two children had died and deciding whether it was murder. Yes, two cot deaths are very unlikely, but so are double murders. If anything, a double murder is even less likely.

I think that the best way of looking at it is by looking at what information we have. In assessing the odds of Sally Clark being guilty, we shouldn’t ask what the odds of two children dying from SIDS is, but, given that Sally Clark’s two children have died, what is the probability that they were SIDS? These two questions can sound similar, but the answer is very different and can be very misleading.

The second point is a bit more subtle, but in a murder case really should have been understood. But reading the Court of Appeal judgments (certainly the first one) frankly they didn't seem to get the first point, let alone the second. It's pretty depressing that people can get locked up on this basis.