Thursday, May 03, 2007

You might hope ...

[Update: Read the link associated with the first comment, it does reveal that Furedi was somewhat misreported - or at least reported out of context.]

Ann Furedi is chief executive of the British Pregnancy Advisory Service. You'd hope that someone who was providing advice would base it on evidence of some sort, and that that person would understand evidence. There was an article in today's Guardian, about whether GPs would sign abortion refusal forms - reporting on a study that survey 309 GPs which found that nearly one in five refused to sign abortion refusal forms.

Furedi was quoted in the Guardian as saying "she did not believe the survey accurately reflected GPs' opinions as it polled less than 1% of the UK's 40,000 GPs". But it's not the proportion of people that you sample that counts, it's the number, and it's their representativeness.

Opinion polls do a pretty good job of predicting elections, but they tend to survey around 1000 people. No one says "There are [approx, I'm guessing] 40,000,000 voters in the UK, and they sampled only 1 in 40,000 (or 0.0025%) of them, so I don't believe the results." If we move to a bigger country, say the US, then 1000 people is taken from a (hypothetical voting) population of 200,000,000 - they sample only 1 in 200,000 people, or 0.0005%, but shockingly this is no worse.

Experiments in psychology are doing pretty well if they have a sample of a couple of hundred people, and we're not trying to generalise to a country there, we're trying to generalise to everyone there is - and everyone there might be - that's an infinite sample, so we have, according to this logic, sampled no one at all.

The crucial thing is how representative the sample is. I have tried to have a look for the article, which seems to have been published in Pulse-i so I could check on it, but I couldn't find it. (You seem to need to be a doctor to register as well). If it was like most magazine surveys, it was a volunteer sample, in which case it probably is nonsense, but not for the reasons Furedi said.

Two further points:

First, it's possible, indeed likely, that Furedi was horribly misquoted. There's a good chance that the journalist who was writing the story didn't understand the issue, so took one part of what she said, and quoted it. (Imagine she might have given a 10 minute explanation of sampling, and at the end said ..."and the sample was on the small side", and that's the bit they decide to quote.

Second, the article also mentions Robbie Foy, who's a pal of mine, so I feel sort of famous by association. (Although not as famous as the fact that I used to have a girlfriend whose mother's hairdresser's sister wrote the words to the theme song to Neighbours).