I am often asked if skeptics and skeptical organizations should undertake first-hand investigations. Of course, it depends upon what your goals are. But I think the question can be re-phrased to mean – is there any value or benefit to first hand investigation, and to this the answer is a definite “yes.”
But this is not to denigrate the value of skeptical review from the comfort of your computer chair. This kind of activity has sometimes been referred to as “armchair skepticism” – meant to be derogatory. While I see the value in going out into the field, armchair skepticism has a valuable and complementary role to play.
In fact, these two activities mirror what real scientists do, and are roughly analogous to peer-review vs experimental replication.
The community of scientists keep each other honest, and keep the process of science grinding forward, in various ways – only one of which is going into the lab to replicate a study or do follow up research. When a colleague publishes a paper, or presents a paper at a meeting, his colleagues provide analysis and criticism. Ideas are examined for logic, internal consistency, and plausibility. Other options, perhaps neglected by the researcher, are explored. And existing research, perhaps not taken into account by the researcher, is brought up and discussed.
This feedback is provided without ever doing any actual investigation. When skeptics perform the exact same service to paranormal or fringe claims, this should not be denigrated at all, but seen as providing in our area of expertise the same kind of analysis that scientists provide in theirs.
This “peer-review” takes several forms. First, the term “peer-review” is often used to refer to the formal process of reviewing a paper that has been submitted for publication. I am not referring to this formal peer-review (which I do not think has any analogy in skeptical activity), but rather to the informal peer-review that collectively refers to all the efforts of the scientific community to hammer errors and flaws out of scientific thinking.