How I Got Banned From Psychology Today’s Facebook Page: An Important Reminder of Why We Must Demand Higher Standards of Science Reporting in the Media
Dr. Justin J. Lehmiller Posted on Wednesday, February 1, 2012 at 8:00AM 
A Psychology Today blogger recently published a post entitled Gender Differences in Personality are Larger than Previously Thought. The post began by stating “a new study confirms that men's minds come from Mars and women's from Venus” [emphasis added]. It then went on to discuss the “huge differences” that exist between the sexes and how “men and women belong to two different species.” Psychology Today was not alone in publishing such provocative claims—many other media outlets reported the same thing. But are these assertions warranted? That’s debatable. The research that formed the basis for the Psychology Today post1 has been questioned by many in the psychological community, who have voiced concerns about the methods used to reach these conclusions. Furthermore, this particular study does not mesh with most published research in this area, which indicates that men and women are far more similar than they are different when it comes to personality.
As a result, when the Psychology Today blog post appeared in my Facebook newsfeed a couple of weeks back, I felt the need to provide a well-reasoned rebuttal. I was among the first to comment and stated that a single study should hardly be taken as “confirmation” of anything because true science is built upon replication—that is, we need to see major findings repeated before we can even think about drawing sweeping conclusions like those proposed in this case. I also mentioned that we need to interpret this study in the context of other research conducted in this area. To that end, I cited a 2005 review paper published in a highly respected journal that analyzed the results of 46 different studies.2 The pattern across these studies revealed that the sexes are very much alike with respect to personality and many other psychological traits. So why should this one new study negate the findings of dozens of others? (Note: For a more elaborated discussion of why you should be skeptical of this "men are from Mars, women are from Venus" idea, see here)
A few hours after posting this comment, I logged back onto Facebook to find that not only had it been deleted, but the Psychology Today Facebook page went so far as to block my ability to make future comments! Needless to say, I found it perplexing that a magazine founded on the principle of accurately conveying psychological research to the public would dismiss my data-driven critique. Although I cannot say with certainty why my commentary was stifled, if the motivation was to protect a sensationalized research report from criticism, I find that to be alarming.
As much as I am concerned about this turn of events, I must say that I am even more troubled that the results of this research were even publicized so widely to begin with because the journal in which this research was originally published is a questionable source, at least in my opinion. The journal is PLoS ONE, which only appeared on the scene about five years ago. Although PLoS ONE claims to utilize a rigorous peer-review process, they publish the vast majority of submissions they receive, with a reported acceptance rate of 69%. This high number indicates that PLoS ONE is not incredibly selective about what makes it into print. For comparison purposes, the top journals in the field of psychology typically accept just 15-20% (or fewer) of all submissions they receive.
It is also worth noting that none of the top journals in psychology charge authors to publish their work; in contrast, PLoS ONE requires authors to pay a $1350 “publication fee.” Thus, as long as an author is willing to front the cash, PLoS ONE will publish almost anything. PLoS ONE publishes a huge amount too, including 14,000 papers in 2011 alone! One might reasonably wonder whether there were really 14,000 worthwhile discoveries last year across all of science, let alone in a single journal. Also, at $1350 a pop, this translates to almost $19 million dollars in revenue from publication fees! One has to wonder where all of that money goes, especially considering that PLoS ONE is an online-only journal and therefore does not have the same expenses as traditional print journals…
In my opinion, a scientist should never have to pay in order to get good quality work published. I also feel strongly that journals should be highly selective about what they print and that not every article makes enough of a scientific contribution to merit publication. However, not everyone shares this view, which is why journals like PLoS ONE were created—they exist to ensure that virtually every article ever written will ultimately be published, irrespective of merit. This is highly problematic because it serves to clutter and cloud the literature, making it harder for the public and the media to distinguish good from bad research. Granted, this is a symptom of a larger problem in the world of academia, where professors are pressured to publish an arbitrary number of papers each year and there is a bias toward rewarding publication quantity over quality. The roots of this problem go very deep and, unfortunately, are not likely to be changed any time soon.
However, until something is done about this, we should demand more responsible reporting of scientific research from the popular media. Media reports of research routinely fail to mention journal quality, let alone the quality of the methods and samples utilized in a given study. By ignoring this information, most science news reports give the impression that almost every study is sound, which is both dangerous and misleading. Journalists have an obligation to evaluate the quality of their source material before publishing anything, whether it is scientific, political, or something else entirely. Just because a new study generates a provocative headline is not reason enough to run it on the front page. If there are questions about the reputation of the journal or there are legitimate concerns about the quality of the study, perhaps it should not be covered at all.
I realize I have focused largely on the role of media here, and I should note that scientists bear some responsibility in correcting this problem too. We need to work with college and university administrators to modify the system of rewards to favor scientific quality over quantity. We need to work with the media to help reporters better understand our work and what makes for sound research to ensure that science is accurately reported. We also need to take a more active role in sharing our research with the public, such as by guest writing for popular media outlets and updating relevant Wikipedia entries. As scientists, we are certainly responsible for what we put into our journals, but we must learn to be responsible for what comes out of them as well.
Any way you look at it, the current system of publishing and reporting scientific research is not serving the public interest and we all need to work together to fix it.
Want to learn more about The Psychology of Human Sexuality? Click here for a complete list of articles or like the Facebook page to get articles delivered to your newsfeed.
1Del Giudice, M., Booth, T., & Irwing, P. (2012). The distance between Mars and Venus: Measuring global sex differences in personality. PLoS ONE 7(1): e29265.
2Hyde, J. S. (2005). The Gender Similarities Hypothesis. American Psychologist, 60, 581-592.
Image Source: http://i337.photobucket.com/albums/n369/cammy1989/banned.jpg
Reader Comments (14)
Great post. Shocked at this but not surprised. What is alarming is when I hear supposedly reputable psychologists wittering on about men and their caves. Meta-analyses simply do not support this "science-fictional" wide gulf. Keep up the good work.
Thanks for your post. I wish more people in this field would speak out about Psychology Today's scientific and ethical shortcomings.
By the way, your link in this sentence appears to be broken: " (Note: For a more elaborated discussion of why you should be skeptical of this "men are from Mars, women are from Venus" idea, see here)"
Hi Justin,
Thanks for this important discussion of what I know was a controversial publication in PLoS ONE. I just wanted to clarify a few things.
First, I've checked with everyone on staff, and we certainly did not ban you from Facebook. I'm not sure what happened there. But you'll see on our Facebook page at least two comments in response to this article that raise serious issues with it. We're delighted when people raise intelligent objections and instigate discussion about something that appears on our site -- that's one reason we're on Facebook to begin with. There are also comments on the post itself that disagree with its conclusions, and those have been left intact as well. We would never find reason to delete reasonable comments; in fact, we welcome them.
Second, just to clarify: The post in question was written by one of our hundreds of bloggers. He is a professor at the University of Chicago who is sharing his interpretation of the study. This was not a reported piece that was published in the magazine. Part of the purpose of our blog network is to create a space for many different perspectives -- this is just one.
Please do try commenting on Facebook again -- or on the post itself -- if you'd like to add to the discussion. You're also more than welcome to email me directly if you have further questions.
Hope this clarifies things.
Here, here! I sometimes feel like Psychology Today has gone the way of a lot of our "news" outlets by only promoting the information that will get them the most hits. Thanks for hitting back.
Hi, Lauren.
Thanks for your comment. As I mentioned in this article, I cannot explain exactly what happened on Facebook. What I can tell you is that I did indeed post an early comment on this particular post that was deleted shortly thereafter. I was also prevented from the ability to post new comments after that. I did not make the comment from my personal Facebook account, but from “The Psychology of Human Sexuality,” which is a Facebook page I run myself and use as a forum for talking about research and professional issues. If my comment was deleted in error, you have my sincere apologies—but please note that I am still not able to make new comments.
Also, while I understand the distinction that you’re making between the bloggers and the magazine, I would like to point out that “Psychology Today” appears very prominently at the top of each blog page and in the URL. Thus, although you may perceive a distinction between the blog and the magazine, the average reader will likely infer that they are one and the same, or that the magazine at least endorses the content of the blog. If you would like to make this separation apparent to the average reader, you may wish to consider adding a statement to each blog page saying so.
Thank you again for your prompt reply and for sharing your perspective.
Sincerely,
Justin Lehmiller
PLoS is a non-profit that has been a pioneer of "open access" journals. Instead of charging thousands of dollars per subscription and $30 for an individual paper like traditional academic publishers, PLoS makes scientific papers available for free to everyone and charges a nominal fee to cover expenses. PLoS also allows you to fill in a form to apply to waive the fee if it's an economic hardship and, from what I've heard, it's pretty much automatically granted.
Compared to the 1.1 *billion* dollars of profit that Elsevier brought in in 2010, the paltry 19 million dollars of revenue that PLoS has is hardly something to be outraged over.
It seems like your post here has two main points. The first is that news reporting of science should make a much greater effort to present new research findings in the context of existing research. This is a major issue with current science reporting, and I'm glad you've flagged it.
The second point you raise is to challenge the validity of PlosOne as a legitimate publication outlet and question whether it is legitimate to pay for publication. I agree that journals have legitimately earned reputations and that such reputations can be a proxy for quality. That said, there are many ways that the philosophy of PlosOne is superior to that of some other journals. (1) publications in PlosOne can be freely accessed by anyone on the web; in contrast, most journals limit access. There's currently a major ground swell of support to move towards journals that provide free access (e.g., http://gowers.wordpress.com/2012/01/29/whats-wrong-with-electronic-journals/). (2) The philosophy of PlosOne is to publish articles that are technically valid regardless of whether the findings are exciting or not. This can help to overcome issues of publication bias associated with only publishing interesting or counter-intuitive findings.
My point was not to suggest that open access journals are inherently bad—they aren’t. In fact, I agree with the general philosophy of making research articles more widely available so that people outside of university settings can easily access them. Perhaps if scientists knew that their work was going to be distributed to a more general audience, they would make more of an effort to write papers that are accessible and understandable to the world at large. This would go a long way toward increasing the relevance of our research, it would likely reduce misunderstandings with the media, and it would encourage the public to consult the original source material. Also, I would be thrilled if journals placed less emphasis on publishing only new and “sexy” findings and stopped ignoring “boring” things, such as simple replications and failures to replicate (those are incredibly important pieces of scientific information too!).
That said, I do stand by my criticisms of PLoS ONE because that journal has an extremely high acceptance rate, publishes an incredibly large number of papers per year, and charges excessive “publication fees.” As someone who has reviewed for multiple journals for several years, I can say with confidence that nowhere near 69% of the papers I’ve reviewed are suitable for publication—I would put that number at less than 25%. My feeling is that there is too much pressure on academic scientists to publish an arbitrary number of papers per year, regardless of how good or important they are. I think we would do the world of science a favor by telling academics to publish less and focus on quality and contribution instead. As for the issue of “publication fees,” I do not see how $1350 per paper is justified for an online only journal. I fail to see how this fee is “nominal” by any standards, given that there is virtually no overhead when publishing something online only—it costs next to nothing. Also, the comparison of PLoS ONE to Elsevier is hardly fair because PLoS ONE is a single journal and Elsevier is a publisher of over 2,000 journals. If Elsevier made $19 million per journal, their profit would be almost $40 billion!
In short, while I do not disagree with the concept of open access journals, I do not think PLoS ONE is implementing this idea appropriately.
I think the high acceptance rate fr PLoS ONE doesn't necessarily show problems: For most journals, how much they can publish is limited by page count. Freed from such a restriction, it would be possible for a lot more to be published. This doesn't necessarily mean you're wrong, but it does mean that more evidence is needed to convince me.
Hi Justin, I am surprised that this happened and find it very frustrating. I read badly described reports of psychology (and other scientific) articles all of the time, and it would be nice if more (not fewer!) scientifically-minded psychologists could interpret them for the world at large. I know nothing about PLoS ONE. I agree that a 69% acceptance rate seems high (25% of the papers you review are of publishable quality!?! You must review for better journals than me! I would say maybe 10% of the articles I review are of publishable quality!).
I did want to comment on the issue of paying for publication, however. There are many journals outside of the psychological arena (predominantly in heavily grant-funded disciplines and/or medical research) that require authors to pay for publication (though $1350 sounds a bit high). Often these fees can be waived if you do not have current grant funding. One example is the journal, Alcoholism: Clinical and Experimental Research. It is a well-regarded publication in the addictions field (I think the impact factor is over a 3.0), but few psychologists publish there due to the fees associated with publishing. I think a more important question to ask regarding pay-to-publish journals is whether they ask you to pay before or after your article has been peer-reviewed. It would be unconscionable to request fees before peer review has occurred (due to the potential for bias). I do not think there would be any more bias involved in paying after an acceptance decision has been reached than in not paying and waiting for library and personal subscriptions to kick in. I am not sure that paying an author's fee for publication is the direction I want our journals to go - but I would hate for someone in, say, social neuroscience to be adversely affected in hiring and promotion decisions within psychology departments for publishing in pay-to-publish journals.
Adam: I think you definitely have a point that some journals are overly selective because of page limitations. However, even if freed of those limitations, I don’t think acceptance rates would ever balloon up anywhere near 70%. As someone who sits on several editorial boards and reviews regularly for a number of journals, I can confidently say that the vast majority of papers submitted just do not merit publication and everyone else in the field I’ve ever spoken to about this shares a similar perspective. I have a hard time believing that the situation would be different at PLoS ONE (i.e., that most of their submissions would be publication worthy), especially given that they are a brand new journal and they charge a hefty fee to publish.
Jaye: You are right that there are some well-respected journals that charge publication fees, and I thank you for pointing that out. For example, the Journal of Marriage and Family charges a $25 submission fee—however, that journal is incredibly selective and very well regarded. Thus, charging a small fee is not inherently problematic. Where it becomes a problem (in my opinion) is when a journal accepts almost every submission they receive AND publication is contingent upon the author paying a very large sum of money.
Nice work, Dr. Lehmiller! I think Psychology Today has been quite a joke for while, but I completely stopped reading it when they published "The Truth About Beauty" back in 2010. http://www.psychologytoday.com/articles/201010/the-truth-about-beauty So incredibly biased and unnecessarily insulting language about "Naomi Wolf and her feminist sob sisters." I think the publication is a joke, and hardly worthy of inclusion in the reporting of empirical science.
I am Associate Editor at PLoS ONE. Richard Lippa (http://psych.fullerton.edu/rlippa/) wrote a post for our blog that quite clearly lays out the arguments in favour of Del Giudice et al.'s position (http://blogs.plos.org/everyone/2012/01/04/multivariate-versus-univariate-conceptions-of-sex-differences-let-the-contest-begin/). We recognized that the study might generate controversy when it was submitted, so I monitored the review process; our academic editor obtained two good, insightful reports from researchers expert in personality structure. We gave Janet Hyde the opportunity to see the paper and prepare a response ahead of publication. We would welcome further comments about the merits (or demerits) of the study in the comments section of the article (http://www.plosone.org/article/comments/info%3Adoi%2F10.1371%2Fjournal.pone.0029265).
One difference with PLoS ONE's approach to peer review compared to most journals is that our academic editors do not take into account subjective measures of perceived or predicted "impact", interest or importance when making decisions, focusing instead on technical soundness (http://www.plosone.org/static/information.action). We do not have a target acceptance level, and publication is not contingent on payment as waivers are granted. Academic and staff editors have no access to information about individual article payment (http://www.plos.org/publish/pricing-policy/publication-fees/).
Hi, Matt:
Thank you very much for your reply. I am very glad to hear you say that publication decisions are made independently of fee payment. I also appreciate the forum that you have provided for discussion of the research your journal publishes--I wish more journals would do the same.
That said, I do still have some concerns about the approach to publication adopted by PLoS ONE and a number of other journals that have emerged in recent years. First, if the publication fee is often waived and is not a requirement for publishing the paper, then why does it exist and why is it so steep? I certainly realize that there are some costs involved in running a journal, but many open access journals raise the revenue they need without charging authors, which helps to avoid any perception of potential scientific impropriety.
Second, while I do agree that it can be scientifically unproductive to set arbitrary acceptance rates at journals, it strikes me as odd when a new journal has an acceptance rate that is five, six, or seven times higher than the rate at most traditional journals. When a journal publishes almost everything it receives, it is not doing any favors to the public or to science. Let me explain: when a paper is published, it gains legitimacy, regardless of its quality or contribution. To people outside of the field (e.g., journalists) who may not understand the methods or statistics, they have no way to judge the merits of the work and will assume based on heuristics that a published paper must be solid science. That’s where we run into potential danger and is why merit must be considered in the review process.
Related to this point, just because a paper is “technically sound” does not mean that it makes a contribution to the literature. You seemed to imply that a focus on “technical soundness” is more objective than a focus on “impact.” However, a determination of “technical soundness” is every bit as subjective as a determination of “impact.” For example, as noted in Richard’s blog about the Del Giudice article (http://blogs.plos.org/everyone/2012/01/04/multivariate-versus-univariate-conceptions-of-sex-differences-let-the-contest-begin/), “there are no God-given prescriptions for proper metrics of effect size.” The same could be said for a lot of other methods and statistics—different scientists have different ideas about what is and is not “sound” procedure. Thus, whether you focus on technical aspects or impact, you cannot take subjectivity out of the review process. Both are important things to consider in making an editorial decision, and when you remove impact from the equation, the end result is that the scientific literature gets watered down.
I do sincerely appreciate your reply, and I should reiterate that I think there are some really great things to be said for the idea of open access and encouraging more public discussion of and interaction with science.
-Justin