If you are going to test arguments, you need information from experts to measure the truthfulness of statements that you hear or read. Where can you get reliable information? One of the magical things about the world we live in today is that much of human knowledge that has been generated from antiquity until the present day is online and can be reached using a smartphone or computer in a matter of seconds. Stop and think about that! That is wonderous. Humans today would be considered wizards and sorcerers by those living a hundred years ago. This information comes in many forms that can be sorted into four groups: primary literature, secondary literature, tertiary literation, and the popular press. Let’s define each class.
Primary literature is research articles published in peer-reviewed journals. These articles will investigate rather specific empirical questions and contain data that supports or refutes them. The article will report the materials and methods that were used to generate the data, present the data in tables or graphs, and provide arguments for or against the question at hand.
Primary literature is the closest to the data since it will discuss data generated by the authors of the paper and analyze it. This interpretation is often of high quality because the scientists working on it are experts in their fields and have exceptional mastery of the subject. Also, the experiments and analyses have undergone peer-review and were thought to be worth publishing. There is no filtering through interpretation and summation as in other forms of information. However, that doesn’t mean the primary literature is the unquestioned truth. Every human has bias and blind spots. Pet theories of the scientists can cloud their interpretation of the data, and flawed experimental designs can skew the data in ways the scientists doing the research may not understand. Peer review can help to mitigate these errors, but sometimes weak papers still get published. Despite these caveats, primary literature is the most trustworthy.
Also, be wary of what I like to call abstract readers. These are found in many fields, but I have run across many of them in the health and fitness field. These “experts” will cite articles they have “read” to back up behaviors they are promoting. In reality they have only read the abstract, or even worse, the title of the article.
Secondary literature is an interpretation of primary sources. The audience for secondary literature is often the groups of scientists working in the field of study that it reviews. Secondary sources do not report data they generate, but rely on information in the primary literature and summarize it. They contribute commentary by experts on the current subject and discuss evidence. Most secondary literature is peer-reviewed and contains numerous citations to the primary literature. Examples of secondary literature include review articles, monographs, and in some cases, even books devoted to one topic. Non-peer-reviewed secondary literature includes editorials, opinion pieces, commentary, histories, or perspective papers.
Secondary literature is also highly trustworthy but comes behind the primary literature. While experts write these articles, they did not carry out every experiment and thus are farther from the data. Personal viewpoints may hinder honest interpretation in areas of dispute, and they can sometimes advocate for only one side of controversies.
Tertiary literature is a further distillation of secondary and primary sources and often has a wider audience that includes non-scientists. Tertiary literature usually does not give credit to any one particular author, but may highlight those who have made significant contributions to the field. Often they will provide more generalized coverage and have a broader subject than secondary literature. Experts in the field of study still frequently write tertiary literature. Examples of tertiary literature include this book, textbooks, dictionaries, manuals, and Wikipedia. Yes, Wikipedia! This resource has grown up over the last decade and much of the information there can be trusted but should be scrutinized like any other information that you read.
Tertiary literature comes in third in trustworthiness. While experts write these articles, they are covering broader topics, and it is impossible for them to have first-hand experience with much of the experimental data and methods. Interpretations can be incorrect, and the process of making complex, detailed experiments understandable to a target audience can distort the facts. A careful reading of the literature and thoughtful editing and writing can minimize these errors, and most tertiary literature is again, very trustworthy.
The popular press will often cover important scientific topics and attempt to inform the general public about them. They may refer to a specific primary research article, or to a few, but they do not have the long citation list of a secondary research article. These articles are most often not written by experts.
When it comes to science in the popular press, it is the least trustworthy. Most popular press articles, while they may pass through an editorial process, are not peer-reviewed and thus have less credibility. Also, the authors of these pieces are more subject to bias. Finally, the popular press has the goal of attracting eyeballs to their work and will often sensationalize or lose the nuance of the original scientific research with the goal of making the findings more exciting.
The most dangerous place to get information on any topic is from blog posts or other internet sites written by amateurs. The authors often have no training in the matter they are discussing and a considerable bias. While it is possible to find exceptional content about many different topics in a blog post be wary of random opinions on the internet, they are no more reliable than that crazy guy you used to live by who was always shouting about lizard men.
In the early days of the information superhighway (yeah that is what we old-timers used to call it) it was laborious to find information if you didn’t know where to look. Helpful individuals, very often working in academia and government, began posting long lists of excellent links on topics. These were valuable because they gave you maps and signposts to a subject you might be interested in, and a real human curated them. But curated lists have huge downsides. You were subject to the whims of that person and what peaked their interest. They also could not cover everything. Very rapidly, an obvious solution arose: search engines. It may surprise you, but scientists have been thinking about and working on search engines since just after World War II. Vannevar Bush published an article entitled As We May Think in The Atlantic Monthly in July 1945 where he laid down the first ideas on how human knowledge could be indexed and retrieved. I don’t have space to go through the long history of search engines, but their rise dramatically increased the ease of information retrieval. These engines work by sending out bots or spiders (little programs crawl the web and collect the text of web pages, sending it back to a central server). The information on the web page is indexed and then ranked. How this ranking takes place determines where on a search results page the information appears. When you search, they first match what you ask for to various pages and then rank the pages based on a complicated algorithm. The specifics of this algorithm are secret (to prevent webmasters from gaming the system), but Google has a helpful set of pages describing what they deliver to you. Every search engine algorhytm does it differently, and that is part of their brand. Search engines evaluate the popularity of your page (how many other pages are linking to your page), how often your search terms appear on those pages, whether your pages has a good user experience, among many other factors.
The go-to search engine for most people is Google. You know you have made it when your business becomes a verb. In order of rank, the top other sites are Bing, Yahoo, Ask, and AOL Search. All of these sites index and search all web pages, unless the webmaster asks otherwise. Many sites track what you search and use that information to show you advertisements, but others make it a point not to (duckduckgo.com). By the way, search engines also keep that information and may sell it to third parties. No matter what search engine you use, the results returned can be authored by anyone, and while search engines do their best to provide relevant data, they cannot discern the truth or quality of any page. If you are looking for a place that searches only information written by experts that is from peer-reviewed journals, several search engines cater to that subspecialty. The dominant service provider is the National Center for Biotechnology Information (NCBI) that is funded by the National Institutes of Health (NIH). The NIH is a part of the US Department of Health and Humans services and fund scientific research. As part of that mission they created the NCBI to facilitate the research process. The search engines of journals (PubMed and PubMedCentral) are a subset of NCBI’s mission, but a particularly important one. They catalog, index, and provide a search interface for over 27 million citations in biomedical literature. If you want to find primary and secondary literature about any topic in biology or medicine, PubMed is a great place to look. Search engines from Google and Microsoft also have available search functions that only search through scholarly articles, Google Scholar and Microsoft Academic. Many of the articles found through these services are open access, meaning anyone can read them, but others are behind subscription walls. Several organizations consolidate their collection of journals and make them available to the public for searching. I list a few of my favorites, but there are many more. Science Direct, Directory of Open Access Journals, Plos One, and Biomed Central. All of these organizations sponsor open access journals that anyone can read for free.
There are multiple ways to find information that interests you. When you want answers to a topic, pick one you like and use it! Before you had to trust authorities in positions of power, newscasters, doctors, scientists, and politicians who most often reached you through the news media and popular periodicals. Today, you can check the facts yourself, but how do you sift through the B.S. and find the kernels of truth? You have to discover for yourself what works, but here are a few suggestions.