10 Reasons Academic Journals Are Filled With Junk Science

Science is held in unique esteem in our society. When we see the words “a study shows” in a newspaper, most people will just accept that whatever follows is the truth. When we see that an article cites an academic journal, we step back and applaud that finally, somebody is giving us the facts.

And that’s actually a good thing—or, at least, it should be. We should be able to trust that these studies and papers were created by people dedicated to the pursuit of the truth.

But there are a few dirty secrets behind the studies we like to trumpet as the truth. Because the reality is that the academic journals that publish these studies are running rampant with bad science—and there are a few things in place that allow it to continue happening.

10Fake Academic Journals Are Becoming Common

Anyone who went to a university knows how much academic journals are held in high esteem. They are treated like the holy grail of sources. Whatever is written in an academic journal is usually accepted as an absolute truth because the papers within them are written by professors and diligently reviewed by highly respected colleagues.

Well, some of them are anyway. Other journals, though, will just publish anything you send them as long as you slip them a $100 bill.

As the Internet has boomed, it has become easier and easier to make your own academic journal—even if you don’t have any qualifications. And people do it. There has been a growing number of pay-for-publication magazines that don’t review the articles they’re sent but label themselves as academic journals anyway.

The number of these fake journals has been on the rise since the Internet began. In fact, from 2010 to 2013, the list of fake journals ballooned from 20 to 4,000.

People can’t tell the difference between the real ones and the fake ones, either. These journals look like peer-reviewed journals. They’re shelved in the same spaces, organized in the same categories, and repeated by the same sources. Ideas printed in these things will get reprinted in the popular media just as easily as ideas in prestigious journals. And even those prestigious ones will repeat these findings as if they’re fact.

But they’re not. These magazines will publish anything you send them without taking as long as one second to review what you’ve written.

9You Can Get Literally Anything Published

You can write literally anything at all and get it published in an academic journal. And when we say “literally” here, we mean literally.

A pair of computer scientists, frustrated with unwanted spam from a pay-for-publication journal, submitted a joke article. It was 10 pages of the same seven words repeated over and over: “Get me off your f—king mailing list.”

They formatted it like a real journal article. It was splashed with charts and tables, all of which just read “get me off your f—king mailing list.” It had those seven words patterned across every page, but it was an article that anybody would have recognized as a joke if they had so much as glanced at it.

The thing is, nobody ever looked at it. Instead, the journal’s computer just sent them an automated response telling them that they’d written a masterpiece and asking them to send $150 to get it published.

The pair, curious to see what would happen, sent the $150. Sure enough, the journal sent out its next issue a short while later with a 10-page article entitled “Get Me Off Your F—king Mailing List” proudly printed within its pages.

So, yes. When we say they will publish “literally anything,” we’re definitely being literal.

8Newspapers Will Repeat Anything From A Journal

Once you’ve paid your $150 and put your article into an academic journal, the world will accept whatever you said as fact. Even if you don’t believe it yourself.

One journalist put this to the test by conducting an absolutely terrible experiment. He wanted to see what would happen if he gave newspapers weak proof that eating chocolate makes you lose weight—something that should be obviously untrue.

He gathered together 15 people, gave chocolate to five of them, and measured their health in as many categories as he could. He believed that if he tested a small enough group of people on enough different things, the people eating chocolate would have to become healthier in at least one way just by sheer random chance.

Sure enough, his results let him say that chocolate was a weight loss tool. He paid to have his terrible science put into a journal and then sent his conclusions to newspapers.

The response was incredible. His faked findings were reprinted or reported on by Cosmopolitan, The Huffington Post, news networks, and morning talk shows. And not a single person talking about them mentioned his deliberately bad methodology.

7Academics Will Repeat Anything From A Journal

It would be nice to be able to say that newspapers only reprinted these things because they didn’t know any better. But they’re not the only ones that do it. Academic journals have been caught reprinting lies that are just as obvious.

The British Medical Journal puts out a joke issue each Christmas. They fill their pages with studies that make ridiculous claims, expecting their readers to appreciate a little levity as a break from the usual fare.

Except that not everybody gets the joke. Like their report that analyzed the effect of retroactive prayer. They tried praying for people who were sick 10 years ago and then checked whether or not they got better. If they got better, they chalked up the recovery to the people in the future praying for them. It was a ridiculous premise. But a few years later, their joke findings were cited as supporting evidence in an article published in a peer-reviewed journal.

That’s not even an isolated incident. Their most successful joke article, which analyzed how many calories kids burn playing video games, has been cited 400 times by various journals.

So it’s not just the mass media that gets suckered into this and not just con artists who sucker them. Once an idea is in a journal, it becomes an idea worth repeating—even to the people who write for the journals.

6Articles With Shorter Titles Get Cited More Often

Even when academics are citing good studies, their rationale for picking them isn’t what you’d imagine. We like to think that these papers are being written by people who have checked every source out there and only used the best ideas. But that’s not necessarily true.

When academic journals choose which ideas they’ll use in their articles, the process isn’t that different from how teenagers decide which BuzzFeed article to read next. As it turns out, the headline makes a huge difference.

A study of 140,000 academic articles found that one of the biggest factors that decides what ideas get cited in other articles is the length of the title. Time and time again, short, simple titles get cited more often than long ones.

This is a big deal because when a paper gets cited, its ideas spread further. So a short, snappy title might have a bigger impact on whether an idea gets accepted in the scientific community than the use of actual, proper research.

It’s a big deal for scientific careers, too. Citations are usually accepted as a benchmark for how well an academic’s ideas have been accepted. So a person who can create a short title might end up with more respect than his or her peers, too.

5Most Experiments Can’t Be Reproduced

The most important part of the scientific process may well be reproducibility. The results of an experiment are only meaningful if other people can get the same ones. Otherwise, all you’ve witnessed is some weird fluke that you don’t yet understand.

The thing is, an incredible number of scientific experiments can’t be recreated. The numbers vary from field to field, but most of them are astounding. Only 36 percent of psychology experiments can be reproduced. This is bad but nothing compared to cancer research, where only 11 percent of what they discover can be replicated by others.

It’s a problem that most people don’t realize exists—but of which the academic community is completely aware. In a survey, more than half of all researchers called the reproducibility problem a “crisis” and an even higher percentage admitted to having failed to recreate an experiment themselves.

In a way, this is a good thing. The fact that we’re trying to reproduce these experiments means that vital quality control is being conducted. But when the percentage of failed reproductions is as high as 89 percent in some fields, it suggests that a lot of people might not be doing these experiments properly in the first place—and that a lot of ideas are getting out there that simply are not true.

4Scientists Hide Faked Data By Using Big Words

A lot of the time, the scientists putting out these misleading studies know exactly what they’re doing. They’re not just getting a few numbers wrong, they’re flat-out lying and they’re trying to hide it.

One group looked at 253 studies that have been retracted from journals and noticed a pattern. Whenever the scientists put in fake data, they’d write in the most convoluted language possible. The articles were filled with weird jargon, complicated sentence structures, and abstract ideas.

Basically, they make their articles as hard to understand as possible in the hopes that you’ll just give up and move on instead of realizing that what they’re saying isn’t true.

In theory, this should make it easy to know which articles you shouldn’t trust. If you have to struggle to understand what they’re talking about, there’s a good chance they’re lying.

But the problem is that these journals are filled with incomprehensible language anyway, purely because it’s part of the culture. Academics will actually throw in unnecessarily difficult language just because they’re afraid that their writing will sound “like a magazine” if it’s not at least a little confusing.

So the data might make it a bit hard to separate who’s lying and who’s just using big words to sound important. Still, it makes one thing abundantly clear—the people who are lying know that they are doing it.

3Drug Companies Bribe Academics

During his career, Harvard professor Joseph Biederman published a lot of strange, poorly designed articles about childhood bipolar disorder. He kept pointing to one conclusion—that children could be bipolar, that they needed to be treated with drugs, and that it didn’t matter how young they were.

His writing had a huge impact. He was a major influence on changing the way the psychiatric community viewed childhood bipolar disorder, and because of his recommendations, doctors started doping up kids who were as young as two years old.

When people started to look into what he was saying, they noticed a few things. First, just like the journalist who claimed that chocolate was a weight loss tool, Biederman set up his experiments so that they’d prove anything he wanted them to. Second, he’d been paid $1.6 million from the drug companies he kept praising.

Everybody accepted what he said because he was a Harvard professor writing for scientific journals. But his ideas got a lot of kids put on medication that they probably shouldn’t have taken. It was all bad science, and he wrote every word because he was paid to do it.

2Professors Need To Publish Or Perish

There’s a phrase that gets tossed around in the academic world: You either “publish or perish.” In other words, if you don’t get something printed in an academic journal each year, you can count on being out of a job.

Some studies have argued that this might be the biggest reason for so much bad science out there. Scientists have to work at their careers. They have to get their funding somewhere to keep their jobs alive, and so they have to get something publishable into print. If they don’t have any ideas, they start putting out anything they can, as fast as they can. And this leads to studies with small sample sizes and articles in pay-to-publish journals.

One group of neuroscientists called out this practice. After identifying studies that they believed were conducted by people who were just writing to keep their careers alive, the neuroscientists tried to recreate the studies. As expected, when they did the experiments themselves, they got wildly different results.

Some scientists have called this out as the major problem in the scientific community. When scientists feel the need to get something out there, it leads to a lot of the experiments out there just being wrong.

1Experts Are Prone To Being Closed-Minded

A major change in the academic world could fix a lot of these problems. But these academics are experts in their fields, and that makes it harder to change the way they see the world. According to one study, just being an expert naturally makes a person more closed-minded.

A group of psychologists quizzed random people on politics. Some were given easy questions that made them feel like they knew everything while others were given extremely difficult questions that left them feeling stupid.

Afterward, they were tested on their willingness to consider other viewpoints. Those who had been manipulated into feeling stupid were extremely open to seeing the world from another point of view. But those who were told they had expert knowledge shut out any ideas that were different from their own.

It’s a troubling revelation because it means that the experts who influence our idea of reality are just as closed-minded. If there’s a major problem in the scientific world, it will take radical changes to fix it. And those radical changes are going to need open minds.