A RECENT REPORT from the Thomson Reuters Foundation made headlines with its conclusion that the United States is one of the 10 worst countries on earth in which to be female. According to the authors of "The World's Most Dangerous Countries for Women," life in America is more violent, cruel, or unfair for women than it is in such grim places as Iran, North Korea, Myanmar, or Iraq.
My first thought on seeing the report was that no one with a working brain could possibly take such a ludicrous conclusion seriously. American women are among the safest, wealthiest, healthiest, best-educated, longest-lived, and most fortunate members of their sex in all of human history. Hundreds of thousands of women from all over the world immigrate to the United States each year — and millions of additional women would like to.
Yet the findings of the Thomson Reuters report were covered in prominent news outlets, including USA Today, CBS, Newsweek, and Fortune, as though they were verifiable facts that had been documented through rigorous scientific analysis. "The U.S. and Syria tied for third [place] for the risks women face in terms of sexual violence, harassment, sexual coercion, and women's lack of access to justice in cases of rape," Fortune told its readers.
Really? The United States and Syria are equally dangerous for women? Syria is a war zone, where more than 400,000 people have been killed in recent years. At least 5 million Syrian refugees have fled the country to escape its horrors. Great swaths of Syria were until recently controlled by Islamic State, which employed mass rape as a weapon of control, and abducted tens of thousands of girls and women to coerce into sexual slavery. In what universe are the lives of women in the United States remotely comparable to that?
Did no journalist at Fortune or the other news outlets that ran with the Thomson Reuters report think to question the authors about their methodology? If they had, they would have seen at once that "The World's Most Dangerous Countries for Women " doesn't even pretend to be objective or scientific. It is based on a "global perception poll" of several hundred "experts in women's issues." When Christina Hoff Sommers, a scholar at the American Enterprise Institute, tried to find out who those "experts" were, Thomson Reuters wouldn't say. In a letter to Sommers, the organization wrote: "We gave an assurance to the experts that their answers would be confidential to allow total honesty."
As Sommers points out in a new video, part of her "Factual Feminist" series, that is a very strange reply. "Are they suggesting their experts might misrepresent the truth if they spoke on the record?" she asks. "Since when do professionals demand anonymity when giving expert opinion?"
Rank-ordering countries is "always tricky and never fully scientific," acknowledges Sommers. But there are organizations that scrupulously assemble and quantify data on the condition of women in different parts of the world — among them, the WomanStats Project, the World Policy Analysis Center, the US State Department, and the World Health Organization. Their reports and findings, some of which Sommers calls attention to, are based on publicly available data.
The same cannot be said of the Thomson Reuters poll. Yet the foundation rolled out the survey as if it were serious social science. And they say that the poll was developed in collaboration with a "global team of data scientists." It speaks of "representative samples," "response rates," "weighted scores." And American reporters were taken in.
Reuters is one of the most esteemed and venerable names in journalism, but this survey is a travesty. It lacks transparency and credibility. It conveys misinformation, which only encourages cynicism about journalism. It fuels charges of "fake news."
There is nothing new about tendentious, axe-grinding "studies" that come to predetermined "findings." Nor is there anything new about research based on insufficient or narrow inputs, or that simply reflects existing bias. One of the most valuable pieces of research ever published, in my humble opinion, was John P. A. Ioannidis's 2005 paper in PLoS Medicine, "Why Most Published Research Findings Are False ." Time and time again, a new study generates a result that is in some way surprising, or newsworthy, or eye-opening. And time and time again, that study eventually gets discredited.
 Women in Syria have faced the horrors of a ghastly civil war, mass rape, and desperate poverty. Hundreds of thousands have been killed; millions more have fled as refugees. Yet a new Thomson Reuters report says Syria and the United States are equal in their treatment of women. |
Here's another example: A study released this spring by Temple University's Hope Center found that — as the New York Times put it in a headline — "Nearly Half of College Students Surveyed in a New Report Are Going Hungry ." The researchers reported that upwards of 4 of every 10 college students in America are at hunger's doorstep: "More than half of survey respondents from two-year institutions and 44 percent of students from four-year institutions worried about running out of food," the report said. "Nearly half of students could not afford to eat balanced meals."
The Times took the report's findings at face value, and played up their significance:
In the coming weeks, thousands of college students will walk across a stage and proudly accept their diplomas. Many of them will be hungry.
A senior at Lehman College in the Bronx dreams of starting her day with breakfast. An undergraduate at New York University said he has been so delirious from hunger, he's caught himself walking down the street not realizing where he's going. A health sciences student at Stony Brook University on Long Island describes "poverty naps," where she decides to go to sleep rather than deal with her hunger pangs.
These are all examples of food insecurity, the state of having limited or uncertain access to food. Stories about college hunger have been largely anecdotal, cemented by ramen and macaroni and cheese jokes. But recent data indicate the problem is more serious and widespread, affecting almost half of the student population at community and public colleges.
A survey released this week by Temple University's Hope Center for College, Community and Justice indicated that 45 percent of student respondents from over 100 institutions said they had been food insecure in the past 30 days.
No one can doubt that there are students at US colleges who struggle to get enough to eat, and no decent person would minimize either the precariousness of their situation or how difficult academic success can be for students with so little money that they constantly worry about food. When my wife attended Syracuse in the 1980s, she held down three part-time jobs to cover her tuition, rent, and living expenses. To this day, she vividly remembers the panic she felt on one occasion when she somehow lost a $10 bill. It was all the money she had budgeted for the week's meals, and she knew there would be no more until the next payday. Food insecurity is not a trivial thing for those who face it.
But is it really true that "nearly half" of American college students can't afford to eat properly? That bottom line struck me as unlikely — so unlikely that it made me wonder about the methodology that generated it.
In an Appendix, the Hope Center provides details of how it conducted its survey.
It sent online invitations to students at 123 colleges, using language calculated to appeal to those feeling stressed:
Making it in college these days can be tough. We want to help. Colleges and universities need to know about the lives of real students like you so that they can offer more support. After you complete the survey, you can enter a drawing to receive a $100 award. . . .
Click here to share your story!
Everything will be kept confidential, so tell the truth. Share your challenges. Help us find solutions.
Nearly 1.5 million such invitations were issued. They were sent early in the fall semester, the report notes, in order to reach students who might be at risk of "dropping out of school later in the year." But of the 1.5 million students contacted, fewer than 86,000 participated. In other words, more than 94 percent of the students contacted by the researchers ignored the invitation. Only a sliver of students, 5.8 percent, responded. Presumably those students were disproportionately receptive to an emailed pitch offering "to help" those for whom college "can be tough."
And what were the characteristics of that sliver?
More than 9 out of 10 (93 percent) were attending public institutions, a far higher share than in the nation's overall college population. Just over 70 percent were female, 36 percent were Hispanic, and 60 percent were not being supported as dependents by their parents or by someone else. Half identified themselves as having some kind of disability, medical condition, or psychological disorder.
This was not, in short, a representative sample of America's population of college and university students. Those on whom the Hope Center's conclusions were based were a self-selected group of respondents, statistically much more likely to be facing financial stress, and therefore more likely to experience food insecurity. Nearly half of those college students may indeed find themselves at times without the food they ought to have. But that's a far cry from the conclusion that nearly half of college students don't have enough money for food.
 Some new studies that reach startling conclusions are correct. But don't bet on it. |
To repeat, there is nothing new about research studies yielding worthless or dubious results, yet being given a respectful reception they don't merit. I am currently reading The Guarded Gate , Daniel Okrent's riveting new history of the anti-immigration movement of the early 20th century, which deployed the "science" of eugenics to make the case that most foreign populations were biologically defective and should be excluded from the United States. Among the most influential immigration restrictionists of the era was the psychologist Henry Goddard, who conducted a "scientific" study of the intelligence of immigrants at Ellis Island.
He published his results in 1917. This is from Okrent's account:
"One can hardly escape the conviction," wrote Goddard, "that the intelligence of the average 'third class' [i.e., steerage] immigrant is low, perhaps of moron grade." How low? His initial results indicated the 83 percent of the Jews were either "morons" (mental age 8 to 12) or "imbeciles (mental age 3 to 7). Italians came in at 79 percent, Hungarians at 80. . . . Goddard put [the results] through an additional statistical screen and managed to conclude that his calculations were off by a factor of two — it was really only 40 percent, all told, who could be considered feebleminded. Yet more alarming . . . was a column of numbers showing that only six of the nearly 200 who were tested could be considered "normal."
Such results defied logic, but they were good enough for the restrictionists. Editors of The Survey — the leading social work journal of the day . . . — claimed that the 40 percent figure was sufficient to prove that if a visit to Ellis Island selected one person at random from "the great mass of 'average immigrants,' " then "you would very likely have found that your choice was feebleminded." It was as simple as that.
Even the best-intentioned studies can yield worthless results; a fortiori those conducted by activists with an agenda. That's true whether the agenda is to intensify #MeToo awareness, to make the case for a larger welfare state or tuition-free college, or to curb the flow of immigration. Or, for that matter, to grab 15 minutes of media attention for the latest eyebrow-raising scientific or nutritional conclusion.
If any phrase should trigger skepticism, it is that familiar preamble "A new study shows." Whatever it shows has a better than even chance of turning out to be quite wrong. Remember always: Studies show that many studies are false.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
ICYMI
My Sunday column disputed the claim that conservative radio host Dennis Prager and his very popular video portal, Prager University, are being censored by YouTube. In reality, hundreds of PragerU videos are posted on YouTube, where they rack up a billion views every year. Along with its parent company, Google, YouTube has helped Prager earn the biggest audience of his career. His whole complaint is that a few dozen PragerU videos are hidden from YouTube users who choose "Restricted Mode," which filters out content with sensitive material. But fewer than 2 percent of YouTube users opt for Restricted mode — by even the broadest definition, that isn't censorship. YouTube is a private company, and should be as free to set the rules for the content on its site as Prager is for his radio show or in his books. Big Tech censorship of conservatives may be a legitimate problem, but PragerU isn't an example of it.
In Wednesday's column , I noted a milestone in Israeli politics: Benjamin Netanyahu has become the longest-serving prime minister in the country's history, surpassing the legendary David Ben Gurion. How has someone so controversial achieved such longevity in office? In a nutshell, by being successful. On Netanyahu's watch, Israel has grown stronger and more prosperous. Despite a rising tide of global anti-Semitism, the Jewish state is more secure than it has ever been. It is less isolated diplomatically than in the past. And while the Middle East remains notoriously violent and unstable, the decade since Netanyahu's 2009 return to power has been the most peaceful in Israel's history.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
The last line
"Whatever our struggles and triumphs, however we may suffer them, all too soon they bleed into a wash, just like watery ink on paper" — Arthur Golden, Memoirs of a Geisha (1997)
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
(Jeff Jacoby is a columnist for The Boston Globe).
-- ## --
Follow Jeff Jacoby on Twitter.
"Like" Jeff Jacoby's columns on Facebook.
Want to read more Jeff Jacoby? Sign up for "Arguable," his free weekly email newsletter