When surveys contradict one another
Two surveys produced conflicting results about public support for gender identity services. But neither took account of scientific evidence.
How come two surveys gave apparently contradictory answers on “what the majority think” about a highly sensitive topic?
An IPSOS survey in 2023 asked: “How much do you agree or disagree with the following? With parental consent, transgender teenagers should be allowed to receive gender-affirming care (e.g., counseling and hormone replacement treatment).”
59% of New Zealanders strongly or somewhat agreed with that. 26% disagreed and 15% weren’t sure.
This survey was cited in a trans-friendly news item in The Post.
Around the same time, a Curia survey asked: “Some people have proposed banning puberty blockers, cross-sex hormones, and physical sex-change surgeries for children under the age of 18 who identify as transgender. Would you support or oppose this kind of ban?”
54% supported this proposed ban. 27% were opposed and 19% unsure.
This survey was cited – and commissioned – by the conservative Family First NZ.
Majorities supported two apparently contradictory statements.
Well, not exactly. The two propositions put to the public are differently worded and don’t directly contradict one another in all respects.
The IPSOS survey asks about “gender-affirming care” including counselling and one of a number of possible medical interventions. It makes the significant qualification of parental consent, but doesn’t mention medical judgement or scientific evidence.
The Curia item asks about a “ban” (for under-18s) on three kinds of medical intervention, and doesn’t mention any psychological interventions such as counselling. “Banned”, however, is clearly the opposite of “allowed”.
It looks as if one survey shows majority support for allowing hormone replacement treatment for under-18s with parental consent, while the other shows majority support for banning it outright regardless of parental consent or medical opinion. That implies, in theory, that some people (maybe around 13% or more) are able to express contradictory opinions on this topic, depending on who’s asking and how they ask. And up to 19% of people may be unsure either way.
The problem’s in the survey design. The IPSOS question uses “soft” terms such as care, affirming, counselling and consent. The Curia question uses “hard” medical terminology, such as blockers, hormones and surgery. The language used gives differing impressions.
And then there’s “priming”: our response to one thing will be affected by how we responded to others, especially when closely connected in time.
The question that appeared to precede the one mentioned above in the IPSOS survey was: “How much do you agree or disagree with the following? Transgender people should be protected from discrimination in employment, housing, and access to businesses such as restaurants and stores”. 84% of New Zealanders agreed with that one.
The preceding question in the Curia survey was: “Do you believe that primary age children should be taught that they can choose their ‘gender’ and that it can be changed through hormone treatment and surgery if they want it to be?” And 76% said “no” to that.
The IPSOS survey led with a question about rights (to be free from discrimination) that all respondents hold, and this set a more positive frame for the next question.
The Curia survey, in contrast, had primed respondents with a question that evoked protection of “primary age children” from premature exposure to delicate and complex issues including sex-change surgery. It was more likely to put the respondent (especially if they’re a parent) in a frame of mind to disapprove of medical intervention in the next item.
There’s no perfect survey that’s devoid of priming effects. Indeed, this very subject-matter may be too complex and highly charged to be fit for simple survey questions on a five-point scale.
The apparent anomaly in results arises partly from differing survey designs (the wording and order of the questions “primed” respondents) and partly from the fact that most people (including me) lack the specialised clinical knowledge to be able to judge. Those who said they weren’t sure were perhaps just being honest. These are difficult questions for a layperson to answer.
When it comes to medical interventions, normally I ask, “Who’s done the research and what do they conclude?”
There’s been a very thorough review of gender identity services undertaken for the NHS in England by Dr Hilary Cass. She found that “there is not a reliable evidence base upon which to make clinical decisions, or for children and their families to make informed choices”.
Despite the assumption about “parental consent” in the IPSOS survey, parents can’t give fully informed consent if there’s insufficient evidence on which to advise them about outcomes and risks. But a lack of evidence hasn’t deterred a lot of people – on both sides – from reaching firm conclusions anyway.
Dr Cass recommends, for example, “extreme caution” and “a clear clinical rationale” if considering “masculinising/feminising hormones” from 16 to 18. So it could be provided as part of a research programme under multi-disciplinary medical and psychological supervision. Her review concludes with neither “ban” nor “allow”. It’s not that simple.
But she also notes the high level of “toxicity” that characterises debate about these issues, and sternly says that the name-calling and bullying “must stop”.
Getting back to surveys, neither the IPSOS nor the Curia survey took account of scientific evidence and clinical advice. They each had a built-in bias.
Public opinion surveys can help us to understand the politics around sensitive issues, but, as always, we should interpret them with a critical mind. They’re not to be taken as objective facts about a social reality “out there”.
An opinion survey is often cited, however, as if it were implicitly an informal plebiscite. And our democratic ethos could assume that, if a majority agreed with a proposition, then that’s what should be done. We leap from thinking “this is what most people think” to “therefore the government should see to it”.
If a majority agreed with a proposition in one apparently “scientific” survey, does that necessarily mean it’s right?
It’s reasonable for a survey to ask people which political party they’d vote for. But is it wise to ask about such complex and sensitive matters as gender identity services?
National Party in sneakers, displaying their rainbow side at the Pride parade:
