If you are going to test arguments, you need information from experts to measure the truthfulness of statements that you hear or read. Where can you get reliable information? One of the magical things about the world we live in today is that much of human knowledge that has been generated from antiquity until the present day is online and can be reached using a smartphone or computer in a matter of seconds. Stop and think about that! That is wonderous. Humans today would be considered wizards and sorcerers by those living a hundred years ago. This information comes in many forms that can be sorted into four groups: primary literature, secondary literature, tertiary literation, and the popular press. Let’s define each class.
Primary literature is research articles published in peer-reviewed journals. These articles will investigate rather specific empirical questions and contain data that supports or refutes them. The article will report the materials and methods that were used to generate the data, present the data in tables or graphs, and provide arguments for or against the question at hand.
Primary literature is the closest to the data since it will discuss data generated by the authors of the paper and analyze it. This interpretation is often of high quality because the scientists working on it are experts in their fields and have exceptional mastery of the subject. Also, the experiments and analyses have undergone peer-review and were thought to be worth publishing. There is no filtering through interpretation and summation as in other forms of information. However, that doesn’t mean the primary literature is the unquestioned truth. Every human has bias and blind spots. Pet theories of the scientists can cloud their interpretation of the data, and flawed experimental designs can skew the data in ways the scientists doing the research may not understand. Peer review can help to mitigate these errors, but sometimes weak papers still get published. Despite these caveats, primary literature is the most trustworthy.
Also, be wary of what I like to call abstract readers. These are found in many fields, but I have run across many of them in the health and fitness field. These “experts” will cite articles they have “read” to back up behaviors they are promoting. In reality they have only read the abstract, or even worse, the title of the article.
Secondary literature is an interpretation of primary sources. The audience for secondary literature is often the groups of scientists working in the field of study that it reviews. Secondary sources do not report data they generate, but rely on information in the primary literature and summarize it. They contribute commentary by experts on the current subject and discuss evidence. Most secondary literature is peer-reviewed and contains numerous citations to the primary literature. Examples of secondary literature include review articles, monographs, and in some cases, even books devoted to one topic. Non-peer-reviewed secondary literature includes editorials, opinion pieces, commentary, histories, or perspective papers.
Secondary literature is also highly trustworthy but comes behind the primary literature. While experts write these articles, they did not carry out every experiment and thus are farther from the data. Personal viewpoints may hinder honest interpretation in areas of dispute, and they can sometimes advocate for only one side of controversies.
Tertiary literature is a further distillation of secondary and primary sources and often has a wider audience that includes non-scientists. Tertiary literature usually does not give credit to any one particular author, but may highlight those who have made significant contributions to the field. Often they will provide more generalized coverage and have a broader subject than secondary literature. Experts in the field of study still frequently write tertiary literature. Examples of tertiary literature include this book, textbooks, dictionaries, manuals, and Wikipedia. Yes, Wikipedia! This resource has grown up over the last decade and much of the information there can be trusted but should be scrutinized like any other information that you read.
Tertiary literature comes in third in trustworthiness. While experts write these articles, they are covering broader topics, and it is impossible for them to have first-hand experience with much of the experimental data and methods. Interpretations can be incorrect, and the process of making complex, detailed experiments understandable to a target audience can distort the facts. A careful reading of the literature and thoughtful editing and writing can minimize these errors, and most tertiary literature is again, very trustworthy.
The popular press will often cover important scientific topics and attempt to inform the general public about them. They may refer to a specific primary research article, or to a few, but they do not have the long citation list of a secondary research article. These articles are most often not written by experts.
When it comes to science in the popular press, it is the least trustworthy. Most popular press articles, while they may pass through an editorial process, are not peer-reviewed and thus have less credibility. Also, the authors of these pieces are more subject to bias. Finally, the popular press has the goal of attracting eyeballs to their work and will often sensationalize or lose the nuance of the original scientific research with the goal of making the findings more exciting.
The most dangerous place to get information on any topic is from blog posts or other internet sites written by amateurs. The authors often have no training in the matter they are discussing and a considerable bias. While it is possible to find exceptional content about many different topics in a blog post be wary of random opinions on the internet, they are no more reliable than that crazy guy you used to live by who was always shouting about lizard men.