I've always thought that one of the biggest problems with DNA testing is contamination. It's not factored in to the equations, but here's an example of what can go wrong.
If it didn't have such massive potential consequences, it would be funny. How can you prevent contamination? You can't. Just recognise that it can happen and don't believe anyone that says that their testing methods are 100% accuracte.
When I finally get round to writing up my thoughts on DNA, I shall return to this.
Sunday 29 March 2009
Sunday 22 March 2009
Confessions
The release this week of Sean Hodgson rightly throws up a lot of questions. The case against him didn't seem to share the paucity of evidence that some miscarriages of justice present.
It does however show yet again the problem of relying on confession evidence. It was a feature of many of the high-profile cases from the 70s (Guildford Four/Birmingham Six etc). The introduction of PACE stopped some of the more outrageous ‘verballing’, but the practice still goes on.
Firstly, there hasn’t been enough research (or if there has, it’s not accessible and in front of the courts) as to why people confess and whether these confessions are reliable.
What Sean Hodgson’s case shows is that confession evidence is no substitute for ‘real’ evidence. Isn’t it time to look again at the rules of evidence so that less weight is put on confessions? If the aim of an investigation is to secure a confession (as that is seen as the ‘best evidence’) then isn’t there a clear danger that the police will focus on this, to the absence of pursuing other leads?
Whilst the police don’t beat people any more to get a confession, it is worrying that so many high-profile cases rely, in whole or in part, on ‘cell confessions’. Anyone who reads the Court of Appeal judgment in the Michael Stone case should be appalled that someone could be convicted on such feeble evidence. If I was sent to prosecute an ABH on that evidence, I would be embarrassed to put it in front of a jury. In truth, the case would never even get that far. It’s just too weak. But when it’s a murder, it seems that different rules apply.
It does however show yet again the problem of relying on confession evidence. It was a feature of many of the high-profile cases from the 70s (Guildford Four/Birmingham Six etc). The introduction of PACE stopped some of the more outrageous ‘verballing’, but the practice still goes on.
Firstly, there hasn’t been enough research (or if there has, it’s not accessible and in front of the courts) as to why people confess and whether these confessions are reliable.
What Sean Hodgson’s case shows is that confession evidence is no substitute for ‘real’ evidence. Isn’t it time to look again at the rules of evidence so that less weight is put on confessions? If the aim of an investigation is to secure a confession (as that is seen as the ‘best evidence’) then isn’t there a clear danger that the police will focus on this, to the absence of pursuing other leads?
Whilst the police don’t beat people any more to get a confession, it is worrying that so many high-profile cases rely, in whole or in part, on ‘cell confessions’. Anyone who reads the Court of Appeal judgment in the Michael Stone case should be appalled that someone could be convicted on such feeble evidence. If I was sent to prosecute an ABH on that evidence, I would be embarrassed to put it in front of a jury. In truth, the case would never even get that far. It’s just too weak. But when it’s a murder, it seems that different rules apply.
Friday 13 March 2009
The Robber from Wood Green - overview
This is based on the judgment in the relevant case. I hope that I haven't done any disservice to the arguments put forward or the proper facts. Apologies if that's the case...
Right. The case of Wilson. Here there were seven knife-point robberies over a period of four days by a masked man carrying a knife. A few days later Mr Wilson was arrested after police officers saw him and noted that his clothing matched the description given of the robber by the victims.
He was arrested and said that he was of no fixed abode. This wasn't true and when his flat was searched, property belonging to three of the victims were found. So far so good. It looks at this stage a pretty good case against him. At least there are some pertinent questions to be answered as to what he was doing with this property.
The defence seemed to have a few points; the description of him didn't match the victims descriptions, especially that Mr Wilson had a strong Scottish accent that the robber (or robbers presumably - this doesn't seem to have been explored) didn't seem to display. It seems then that the Prosecution felt that this wasn't enough, so seven months later got some new evidence:
"The issue on this appeal relates to the admissibility of the evidence of Miss Alexis Young, who was responsible for monitoring crime in Islington. The defence disputed the evidence could be adduced. It is the ruling of the Recorder at the trial that it could be adduced which is the subject matter of this appeal. The evidence of Alexis Young was that she had searched the police computer records for similar offences using two criteria, namely "robbery" and "mask" which brought up two other crimes, none of which were similar to the present offences. She also used a criteria "Islington", "lone white male 20 - 30" and "knife". These searches produced no results. She therefore concluded that no similar offences had been committed in the area between the date of the offence and 31st May 2007."
It isn't clear from the judgment whether the report from Ms Young was any more scientific than that, but it would appear not. And that’s it. That’s all she did.
The problem with the last sentence "She therefore concluded that no similar offences had been committed in the area between the date of the offence and 31st May 2007" is that this is a conclusion with absolutely no basis whatsoever in terms of analysis. The report seems to me to be valueless as evidence (before even looking at the maths, see how many cognitive biases there are). The faults with the methodology are so numerous you wonder how it got in front of the jury. If I get bored one day, I might go through all the ones I can see.
The basic point is that evidence of this character could be relevant to whether Mr Wilson was the robber, but only if it was done properly and rigorously (and isn't it really expert evidence?) Without this, it's the legal world's version of homeopathy, it may look good but it's without any substance.
Two other points. From a legal point of view a massive potential problem for the prosecution is that it opens the door to the defence to ask for reams of similar disclosure in a suitable case to prove the opposite (even cherrypicking the terms that produce the best results for them from a database?). This will involve a lot more work for the police (unless there turns out to be one rule for the Prosecution and one for the Defence? Surely not!).
Lastly, on one view there is a heuristic (it's got at least as much validity as the Court of Appeal's view) argument that this evidence actually makes it less likely that Mr Wilson was guilty, but that's for another day...
Right. The case of Wilson. Here there were seven knife-point robberies over a period of four days by a masked man carrying a knife. A few days later Mr Wilson was arrested after police officers saw him and noted that his clothing matched the description given of the robber by the victims.
He was arrested and said that he was of no fixed abode. This wasn't true and when his flat was searched, property belonging to three of the victims were found. So far so good. It looks at this stage a pretty good case against him. At least there are some pertinent questions to be answered as to what he was doing with this property.
The defence seemed to have a few points; the description of him didn't match the victims descriptions, especially that Mr Wilson had a strong Scottish accent that the robber (or robbers presumably - this doesn't seem to have been explored) didn't seem to display. It seems then that the Prosecution felt that this wasn't enough, so seven months later got some new evidence:
"The issue on this appeal relates to the admissibility of the evidence of Miss Alexis Young, who was responsible for monitoring crime in Islington. The defence disputed the evidence could be adduced. It is the ruling of the Recorder at the trial that it could be adduced which is the subject matter of this appeal. The evidence of Alexis Young was that she had searched the police computer records for similar offences using two criteria, namely "robbery" and "mask" which brought up two other crimes, none of which were similar to the present offences. She also used a criteria "Islington", "lone white male 20 - 30" and "knife". These searches produced no results. She therefore concluded that no similar offences had been committed in the area between the date of the offence and 31st May 2007."
It isn't clear from the judgment whether the report from Ms Young was any more scientific than that, but it would appear not. And that’s it. That’s all she did.
The problem with the last sentence "She therefore concluded that no similar offences had been committed in the area between the date of the offence and 31st May 2007" is that this is a conclusion with absolutely no basis whatsoever in terms of analysis. The report seems to me to be valueless as evidence (before even looking at the maths, see how many cognitive biases there are). The faults with the methodology are so numerous you wonder how it got in front of the jury. If I get bored one day, I might go through all the ones I can see.
The basic point is that evidence of this character could be relevant to whether Mr Wilson was the robber, but only if it was done properly and rigorously (and isn't it really expert evidence?) Without this, it's the legal world's version of homeopathy, it may look good but it's without any substance.
Two other points. From a legal point of view a massive potential problem for the prosecution is that it opens the door to the defence to ask for reams of similar disclosure in a suitable case to prove the opposite (even cherrypicking the terms that produce the best results for them from a database?). This will involve a lot more work for the police (unless there turns out to be one rule for the Prosecution and one for the Defence? Surely not!).
Lastly, on one view there is a heuristic (it's got at least as much validity as the Court of Appeal's view) argument that this evidence actually makes it less likely that Mr Wilson was guilty, but that's for another day...
Thursday 12 March 2009
The Sally Clark case
The case of Sally Clark is an instructive one when looking at how courts (mis)handle statistics. Wikipedia has a useful summary with good links to some of the problems that come up from it.
The essence of the (statistical) problem was that one of the Prosecution expert witnesses (Roy Meadow) said that the chance of a SIDS (cot death) was 1 in 8,543. Sally Clark's two children had died from cot death. He then took from that that the chances of both children suffering a cot death was 1 in (8543*8543), in other words 1 in 72,982,849 and on that basis the inference was 1 in 73 million is so unlikely that you can discount cot death and conclude that this was murder.
This is completely and embarrassingly wrong. There are two major problems.
The first is that to find the probability of two events happening by multiplying the individual probabilities is only valid if the two events are 'independent'. For example, the Probability of getting a head if you toss a fair coin [P(H)] is ½. The Probability of getting a one if you roll a fair dice [P(1)] is 1/6.
The Probability of getting a head and rolling a one if you toss a coin and roll a dice is ½ x 1/6= 1/12. In other words, for two independent events P (A and B) = P(A) x P(B).
So multiplying the odds is only valid if there are no environmental or genetic factors present in cot deaths. This seems on the face of it unlikely (and the full report that the stats came from indicate that there are environmental factors).
Roy Meadow was made the subject of a complaint to the GMC. This ended up in the Court of Appeal were it was concluded (by a majority) that he was not guilty of Serious Professional Misconduct.
Roy Meadow is obviously an educated man. He is a man of science. Whilst he is not a statistician, this is not complicated in any way. It’s basic maths. One would have thought that someone who is a doctor should have seen this coming a long way off.
But, whilst he got rightly criticised for this, one wonders why nobody in court from amongst the lawyers picked up how obviously wrong this was? Anyone with a GCSE in maths should have spotted the error and asked some questions as to how it was right to assume independence.
The second problem is that is another example of the Prosecutor’s Fallacy (most often seen in DNA evidence). 73 million sounds a large number. 1 in 73 million sounds like very long odds indeed, doesn't it? But even if that figure is correct (which it almost certainly isn’t) what does that tell us?
Because we are very bad at reasoning with numbers, it’s tempting to conclude that the chance of Sally Clark being guilty is 1 in 73 million. It needed to be explained carefully why this wasn’t the case.
What it actually told us is the chance of a person, chosen at random in the UK, suffering two cot deaths is 1 in 73 million. But, the jury was looking at one person where her two children had died and deciding whether it was murder. Yes, two cot deaths are very unlikely, but so are double murders. If anything, a double murder is even less likely.
I think that the best way of looking at it is by looking at what information we have. In assessing the odds of Sally Clark being guilty, we shouldn’t ask what the odds of two children dying from SIDS is, but, given that Sally Clark’s two children have died, what is the probability that they were SIDS? These two questions can sound similar, but the answer is very different and can be very misleading.
The second point is a bit more subtle, but in a murder case really should have been understood. But reading the Court of Appeal judgments (certainly the first one) frankly they didn't seem to get the first point, let alone the second. It's pretty depressing that people can get locked up on this basis.
The essence of the (statistical) problem was that one of the Prosecution expert witnesses (Roy Meadow) said that the chance of a SIDS (cot death) was 1 in 8,543. Sally Clark's two children had died from cot death. He then took from that that the chances of both children suffering a cot death was 1 in (8543*8543), in other words 1 in 72,982,849 and on that basis the inference was 1 in 73 million is so unlikely that you can discount cot death and conclude that this was murder.
This is completely and embarrassingly wrong. There are two major problems.
The first is that to find the probability of two events happening by multiplying the individual probabilities is only valid if the two events are 'independent'. For example, the Probability of getting a head if you toss a fair coin [P(H)] is ½. The Probability of getting a one if you roll a fair dice [P(1)] is 1/6.
The Probability of getting a head and rolling a one if you toss a coin and roll a dice is ½ x 1/6= 1/12. In other words, for two independent events P (A and B) = P(A) x P(B).
So multiplying the odds is only valid if there are no environmental or genetic factors present in cot deaths. This seems on the face of it unlikely (and the full report that the stats came from indicate that there are environmental factors).
Roy Meadow was made the subject of a complaint to the GMC. This ended up in the Court of Appeal were it was concluded (by a majority) that he was not guilty of Serious Professional Misconduct.
Roy Meadow is obviously an educated man. He is a man of science. Whilst he is not a statistician, this is not complicated in any way. It’s basic maths. One would have thought that someone who is a doctor should have seen this coming a long way off.
But, whilst he got rightly criticised for this, one wonders why nobody in court from amongst the lawyers picked up how obviously wrong this was? Anyone with a GCSE in maths should have spotted the error and asked some questions as to how it was right to assume independence.
The second problem is that is another example of the Prosecutor’s Fallacy (most often seen in DNA evidence). 73 million sounds a large number. 1 in 73 million sounds like very long odds indeed, doesn't it? But even if that figure is correct (which it almost certainly isn’t) what does that tell us?
Because we are very bad at reasoning with numbers, it’s tempting to conclude that the chance of Sally Clark being guilty is 1 in 73 million. It needed to be explained carefully why this wasn’t the case.
What it actually told us is the chance of a person, chosen at random in the UK, suffering two cot deaths is 1 in 73 million. But, the jury was looking at one person where her two children had died and deciding whether it was murder. Yes, two cot deaths are very unlikely, but so are double murders. If anything, a double murder is even less likely.
I think that the best way of looking at it is by looking at what information we have. In assessing the odds of Sally Clark being guilty, we shouldn’t ask what the odds of two children dying from SIDS is, but, given that Sally Clark’s two children have died, what is the probability that they were SIDS? These two questions can sound similar, but the answer is very different and can be very misleading.
The second point is a bit more subtle, but in a murder case really should have been understood. But reading the Court of Appeal judgments (certainly the first one) frankly they didn't seem to get the first point, let alone the second. It's pretty depressing that people can get locked up on this basis.
Intro II
I'd been spouting off in the pub about the misuse of statistics and bad science in general, but there are so many people better qualified than me out there to comment about this. It did strike me though that there was no-one commenting specifically on how science and the law interact. Although, in fairness, it doesn't come up very often, when it does lawyers and courts can get it spectacularly wrong and it seems that this should be pointed out.
The thing that triggered me to start this was when I was flicking diligently through the Criminal Law Review, as all lawyers should, and saw the Commentary on the case of Wilson from last year. On the face of it, this is a frankly extraordinary and quite worrying case. Law (and the criminal law especially) is supposed to be based on the analysis of evidence and reason, a cold and dispassionate assessment of the facts. This case is, I believe, a good demonstration of what can go wrong.
(Almost) all of the comments I will make on this blog are taken from the judgments of the relevant cases. Judgments are supposed to contain all the relevant facts and argument, but as I certainly know from experience, this does not always happen. A blanket disclaimer therefore that apologies galore if I get something wrong, it certainly wasn’t intended.
And lastly, I should say, whilst writing this blog is clearly tragic and self-indulgent, it may hopefully save me a bit of time when colleagues ask me, yet again, to explain DNA evidence. Not much time, but a bit.
The thing that triggered me to start this was when I was flicking diligently through the Criminal Law Review, as all lawyers should, and saw the Commentary on the case of Wilson from last year. On the face of it, this is a frankly extraordinary and quite worrying case. Law (and the criminal law especially) is supposed to be based on the analysis of evidence and reason, a cold and dispassionate assessment of the facts. This case is, I believe, a good demonstration of what can go wrong.
(Almost) all of the comments I will make on this blog are taken from the judgments of the relevant cases. Judgments are supposed to contain all the relevant facts and argument, but as I certainly know from experience, this does not always happen. A blanket disclaimer therefore that apologies galore if I get something wrong, it certainly wasn’t intended.
And lastly, I should say, whilst writing this blog is clearly tragic and self-indulgent, it may hopefully save me a bit of time when colleagues ask me, yet again, to explain DNA evidence. Not much time, but a bit.
Wednesday 11 March 2009
Intoduction
Why this? Do we really need another legal blog? Probably not. So why write this? Well, I’m not a mathematician or a statistician or a scientist of any sort, but I am someone interested in these things. A lot of people get annoyed by misuse of grammar. Most of those people annoy me. What annoys me more is the level of innumeracy in our society.
It is acceptable for people to stand up in Court and state that they have no idea about maths and even take pride in this fact. I haven’t yet had the courage to say “Your Honour, I would read this document I’ve just been handed, but frankly, I was never very good at English. I can barely read and write” but if I did say this, the only thing that would stop me getting taken down for contempt would be that no-one would believe me.
The consequence of this innumeracy is that people are wowed by statistics and swallow bad arguments without thinking (especially, it seems to me, those people who get most upset about the difference between less and fewer). Some of this is annoying but harmless, but in a discipline where the consequences of bad arguments is that someone can get locked up when they shouldn't (or someone who should be locked up walks away) and where logic is supposed to apply, it is depressing how unfamiliar people are with basic maths.
And that’s before even considering the multitude of lies about crime that appear in the newspapers.
What I'm going to try to do in this blog is have a look at where these arise and provide a slight counterbalance to it...
It is acceptable for people to stand up in Court and state that they have no idea about maths and even take pride in this fact. I haven’t yet had the courage to say “Your Honour, I would read this document I’ve just been handed, but frankly, I was never very good at English. I can barely read and write” but if I did say this, the only thing that would stop me getting taken down for contempt would be that no-one would believe me.
The consequence of this innumeracy is that people are wowed by statistics and swallow bad arguments without thinking (especially, it seems to me, those people who get most upset about the difference between less and fewer). Some of this is annoying but harmless, but in a discipline where the consequences of bad arguments is that someone can get locked up when they shouldn't (or someone who should be locked up walks away) and where logic is supposed to apply, it is depressing how unfamiliar people are with basic maths.
And that’s before even considering the multitude of lies about crime that appear in the newspapers.
What I'm going to try to do in this blog is have a look at where these arise and provide a slight counterbalance to it...
Subscribe to:
Posts (Atom)