“But It’s Not From a Peer Reviewed Journal!”

I see this nonsense more and more as days go by. The “I Fucking Love Science” and the Black Science Man fetishists are especially guilty of this.

The elevation of “Peer Reviewed” journal articles to almost holy writ directly from the mouth of God.

Infallible, dogmatic, and “How dare you question SCIENCE!” narrow-mindedness of those that have neither experienced nor witnessed first-hand the “peer review” process.

While peer review, ideally, is a good thing, in practice it can be abused or “gamed.”

I’ll tell you a story. When I was in Grad School, one of my colleagues wrote up a paper on his chemistry for submission. After the usual back-and-forth between him and the boss (and lots of red ink), they submitted it to a fairly high impact journal.

And they waited.

And waited some more.

Finally, the results came back, the article was rejected for publication in that journal. Guess the reason why.

Was it because there was something wrong with the chemistry? No.

Was it because my colleague tried to handwave his way through his mechanism without hard data? Nope.

Take a moment and think about what merited a rejection. The chemistry was sound and the conclusions were backed up with data.

Think for a second, I’ll wait.

The real reason why his paper was rejected was because one referee had issue with the manuscript because my colleague didn’t include all the “necessary” citations for “prior art” in his introduction. Because of this small, petty thing, the paper was rejected out of hand.

Here’s the kicker: the reason why this referee got so butthurt about the citations not being there was because he was slighted that the “prior art” – meaning his own research – was not mentioned.

Petty and weak isn’t it?

My colleague did eventually publish his paper in another journal, but it goes to show that while peer review is supposed to focus on the validity of the work as a whole, the pettiness of Academics subvert reality.

There have also been some cases where some very poorly written papers skate through the review process, because of the Big Name in the authors list. Even if the chemistry is meh so-so (*coughcough*K.C.Nicolau*coughcough*) or yet another paper where “we increased the yield of this well-known named reaction by 0.5% (99% ee) using a catalyst that takes 10 steps to make (and only works with our “special” bottle of the metal salt starting material).” It doesn’t matter. If you were a referee and declined a Big Name’s paper with a “How is this suitable for [Journal Name]?!?” you most likely end up blacklisted for essentially doing your job.

Good luck publishing your own stuff when the vindictive, petty tyrants that make up Academia deem you persona non grata.

The long and the short of this rant is this: Peer Review is a good thing, but it isn’t the end all be all denoting that a work is either Scientific or Correct.


8 thoughts on ““But It’s Not From a Peer Reviewed Journal!”

  1. Ah, Peer Review! So many stories.

    Like when a famous professor was asked by a top journal to contribute because they wanted to have one more paper on cell cycle in their issue, he sends in a premature study, the reviewers reject it, the professor complaints and the journal accepts the manuscript after all.

    And how do you become a “peer reviewer”? By becoming professor. And how do you become professor? Through great connections (and also good science, of course). Does this mean professors are always the best? You do the math.

    I think in total, the peer review process is ok, questionable decisions are maybe 20%. But over all, it is not the be-all-end-all method.

    In the same vein, a book is never good just because it is published by one of the big East Coast booksellers.

    Then there is the point of WHICH studies get mentioned in the news. If the professor is media-savvy, then he sells his study to the newspaper, even though the journal it is in is only 2nd or 3rd tier. Peer-reviewed, yes, but not necessarily very impactful. If you are not a scientist, it is difficult to judge the significance of a study.

  2. Being a wannabe science nerd I am glad to see this post. I have always wondered if the science article I am reading is using a study that was done properly.

    Do I research if the study was done with the utmost delicacy and respect for the scientific method? Nope.

    Question: how can a “science laymen” (although I love reading science articles) like me trust that what I am reading is citing a study that was done properly? Is there a, “be-all-end-all method” that can be done while conducting research that makes a study most valid?

    • As Florian said above: “If you are not a scientist, it is difficult to judge the significance of a study.”

      The problem with being able to notice BS or handwaving arguments is that you yourself need to have knowledge of the field (knowing the theory behind things, and also lots of reading). I know synthetic chemistry (Organic and Inorganic/Organometallic) decently enough to see it, but if you showed me a Bio or Biochem article I’d shrug my shoulders and say “OK, I don’t know enough about this to comment on.”

      • It’s getting harder to trust whether what I am reading is legit.

        Branches of science:
        1. Natural
        2. Social
        3. Formal
        4. Applied

        Natural – “Natural science is a branch of science that seeks to elucidate the rules that govern the natural world by applying an empirical and scientific method to the study of the universe.”

        Social – “The social sciences are the fields of scholarship that study society.`

        Formal – “The formal sciences are the branches of knowledge that are concerned with formal systems […]”
        -Information Theory
        -Philosophy (Logic)

        Applied – “Applied science is the application of scientific knowledge transferred into a physical environment.”
        -Computer Science
        -Applied Physics
        -Applied Math

        Each apply the scientific method when conducting research to come up with empirical evidence (I had to look that word up hah).

        But does a Chemist conduct a study the same way a Psychologist conducts studies?

        I think because I read about so many different sciences I seem to get the idea that they all do things the same way. And are all in it ‘just for the sake of science’ and don’t have any hidden agendas. My ‘delicate science bubble’ has been popped.

        I guess nothing I read these days can be safe from fraud or manipulation of the results.

        (info on sciences taken from https://en.wikipedia.org/wiki/Branches_of_science)

      • Most “results” of social “science” “studies” can be tossed in the trash can because of one thing: irreproducability.

        They may follow the Scientific Method, but when there’s no way to independently confirm said findings, it becomes a GIGO (Garbage In, Garbage Out) scenario.

        The Natural Sciences at least give a method for you to reproduce the original author’s work (which is mainly where one finds his “preps” for making compounds). The Applied Sciences are self explanatory: a bridge will fall apart if not built right or a patient might die if misdiagnosed.

  3. A guy a twitter shared this article: http://www.firstthings.com/article/2016/05/scientific-regress

    “When a study fails to replicate, there are two possible interpretations. The first is that, unbeknownst to the investigators, there was a real difference in experimental setup between the original investigation and the failed replication.

    The other interpretation is that the original finding was false. Unfortunately, an ingenious statistical argument shows that this second interpretation is far more likely.”

    I have heard of taking TRP for women and society but I never thought to look through my RP lens and take a glance at science. My precious, precious science.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s