Measurement issues in sex differences in intelligence
There's been a lot of fuss recently over a paper, entitle 'Sex differences in means and variability on the progressive matrices in university students: A meta-analysis', published in the British Journal of Psychology, by Paul Irwing and Richard Lynn. The Guardian and The Daily Mail, (The Great Intellectual Divide) amongst many others have reported this. The journalists report that (basically) the study found that men are more intelligent than women.
However we need to think about this a little more critically, as there are a number of hidden assumptions that are relevant to the interpretation of this study, and they are not made explicit.
The first assumption is that Raven's progressive matrices (RPM) measures IQ. I think that it does - the RPM has one of the highest g-loadings of any tests. If you want a test of IQ, you cannot go far wrong with the RPM.
Of course it cannot be a perfect measure of IQ - there will always be measurement error, but what we need to know is whether this measurement error is random, or is it correlated with sex? That is, does the Raven's progressive matrices test underestimate the IQ of men, or overestimate the IQ of women?
It is quite possible for the correlation to be high (between IQ and a test) but the errors to be non-random. For example it is well known that men are more competitive than women. The RPM is an untimed test - that is, you can spend as long as you would like to on the test. The papers that Irwing and Lynn used did not determine the amount of time taken by people on the test, and did not take this into account. Men are usually more competitive than women, and if people treat the test as some form of competition (and I usually do) the more competitive people might try harder, and take longer. The graph shown below shows this possible effect.
If we used a different test, say one which was time constrained, we might find those lines moved closer together.
The second assumption is that IQ (which is a thing that is measured by IQ tests) is the same thing as intelligence (which is the thing that we have got more or less of) - again, there is a relationship there, and I would suspect it's a strong one, but I don't think it's perfect.
The problem is that the concepts of IQ and intelligence are difficult to define, without using operational constructs, like the actual tests used, and if we start to talk about the actual tests used, we don't know what the relationship is between the test and the construct that we are talking about. It would not be difficult to construct an "IQ" test that men did better on (Hey! We've got one - the Raven's Progressive Matrices!), but would it be possible to construct a test that women did better on? I suspect it would. Would it be possible to determine which one of thoses tests is a better (as in less biased) measure of IQ or intelligence? I suspect it wouldn't.
However we need to think about this a little more critically, as there are a number of hidden assumptions that are relevant to the interpretation of this study, and they are not made explicit.
The first assumption is that Raven's progressive matrices (RPM) measures IQ. I think that it does - the RPM has one of the highest g-loadings of any tests. If you want a test of IQ, you cannot go far wrong with the RPM.
Of course it cannot be a perfect measure of IQ - there will always be measurement error, but what we need to know is whether this measurement error is random, or is it correlated with sex? That is, does the Raven's progressive matrices test underestimate the IQ of men, or overestimate the IQ of women?
It is quite possible for the correlation to be high (between IQ and a test) but the errors to be non-random. For example it is well known that men are more competitive than women. The RPM is an untimed test - that is, you can spend as long as you would like to on the test. The papers that Irwing and Lynn used did not determine the amount of time taken by people on the test, and did not take this into account. Men are usually more competitive than women, and if people treat the test as some form of competition (and I usually do) the more competitive people might try harder, and take longer. The graph shown below shows this possible effect.
If we used a different test, say one which was time constrained, we might find those lines moved closer together.
The second assumption is that IQ (which is a thing that is measured by IQ tests) is the same thing as intelligence (which is the thing that we have got more or less of) - again, there is a relationship there, and I would suspect it's a strong one, but I don't think it's perfect.
The problem is that the concepts of IQ and intelligence are difficult to define, without using operational constructs, like the actual tests used, and if we start to talk about the actual tests used, we don't know what the relationship is between the test and the construct that we are talking about. It would not be difficult to construct an "IQ" test that men did better on (Hey! We've got one - the Raven's Progressive Matrices!), but would it be possible to construct a test that women did better on? I suspect it would. Would it be possible to determine which one of thoses tests is a better (as in less biased) measure of IQ or intelligence? I suspect it wouldn't.
0 Comments:
Post a Comment
<< Home