Oct 06

How to Evaluate Any News Content for Reliability and Bias

October 6, 2020
Woman evaluating and assessing an article in a newspaper

News article on laptop and in newspapers with headline "Internet Is Sweeping the World Faster Than Electricity!"The modern information landscape is flooded with thousands of news and “news-like” sources. The sheer volume of these sources presents a unique challenge to each one of us being news literate ourselves.

A common complaint these days from individual news consumers (including educators) is “I don’t know what to trust anymore.” However, none of us (especially educators) have the luxury of simply giving up—rather, we have the responsibility to learn what to trust, as challenging as that may be.

To be truly news literate today, it is no longer sufficient to be familiar with just a handful of news sources you trust solely based on their reputations. Nor is it sufficient to rely on a handful of shortcuts or “tips and tricks” to determine what to trust. Today’s news literacy requires being able to evaluate the unfamiliar. It requires acquiring, developing, and continuously practicing keen information evaluation skills.

My company, Ad Fontes Media, rates the news for reliability and bias and teaches others to do the same for themselves. We have developed a framework that our analysts use to take any piece of news or news-like content, evaluate it methodically, and give it a score for reliability and a score for bias. If you are unsure about the best way to approach news content from an unfamiliar source, you can use this same framework to assess how much you should trust it.

There are many factors you can use to assess the reliability and bias of a piece of news content, but we have found that a few key factors can be consistently evaluated in nearly any piece of content, and that by making assessments about these factors, you can make a reasonable determination about both reliability and bias. When you can reasonably determine both reliability and bias, you can reasonably infer a level of trust.

Reliability

First, to assess reliability of a piece of news content, consider each the following three factors independently:

  1. Expression
  2. Veracity
  3. Headline/Graphic

Expression

Even before you try to verify whether individual facts in an article are ultimately true, the expression of an article can provide valuable indications about its likely reliability. Consider whether an article is expressed as mostly fact statements, analysis statements, opinion statements, or very highly opinionated statements. Those that are expressed as mostly opinion statements or very highly opinionated statements should be considered inherently less reliable than those with mostly fact and analysis statements. But although articles with more fact and analysis statements tend to be more reliable, it is important to look further to confirm.

Veracity

It is impossible for any one person to fact check and verify every statement in every article, but it is important to try to determine the veracity of certain key facts. In particular, if any statements in an article are surprising, fishy, or shocking to you, and sound like they would be “big, if true,” it’s time to start digging.

Fortunately, we have a robust media ecosystem full of professional fact-checkers and journalists at thousands of news organizations across the country whose jobs are to investigate questionable claims as quickly as possible. If you come across a “fact” that you have reason to doubt, verifying it is as easy as entering the questionable fact in a search engine and scanning the results that come up.

You may find a “fact check” or a “debunking” that can conclusively tell you if something is true or false, but many statements do not warrant a full investigative fact check. More often what you find is one of two options: 1) another reputable organization has corroborated the fact or 2) no other reputable organization has corroborated the fact. If you can’t find the fact corroborated in a source you are familiar with and trust, it is best to be skeptical. At the very least, wait before sharing the article on social media.

Headline and Graphic

Comparing a headline and graphic to its corresponding story is another simple but effective factor to consider in assessing overall reliability. Sometimes headlines and graphics give much different impressions of a story than the actual article indicates after reading it.

If you determine that an article’s headline and corresponding graphic do not correspond to the content of the article itself, you should consider the overall article less reliable than one in which each matches the other.

Bias

Evaluating degrees of bias is also important in today’s news landscape, because there is a big difference between analysis and opinion content that merely advocates or argues for different points of view and extremist, polarizing content that vilifies and dehumanizes one’s political opponents.   

You can evaluate the level of political bias of an article by considering the following three factors independently:

  1. Language
  2. Political Position
  3. Comparison

Language

As a first step, examining various types of linguistic indicators can provide clues about the article’s degree of bias.

A first type of linguistic indicator is the words used to describe political positions. Notice if the article uses words normally associated with one side or another, such as “pro-choice” vs. “pro-life” or “illegal aliens” vs. “undocumented immigrants.” Each side has milder and more extreme terminology across political issues. These terms can be helpful clues, but they do require that the reader be somewhat familiar with which side uses which terms.

A second type of linguistic indicator is the words used to characterize one’s political allies and opponents. For example, calling a politician smart or savvy may indicate mild positive bias toward that politician’s party, while calling a politician heroic or brilliant may indicate a stronger positive bias. Conversely, calling a politician crafty or sneaky may indicate a mild negative bias, while calling that same politician stupid or senile would indicate an extreme negative bias.

Political Position

Capitol building with blue left and red right sides

This step is what comes to mind for most people when they think about bias; it involves evaluating whether certain political positions are advocated for or presented in an imbalanced way in an article.

In our Media Bias Chart content analysis methodology, we anchor the left/right spectrum upon the contemporary policy positions of elected officials. That is, the middle of the right side of the chart represents median Republican policy positions and the left side represents median Democratic policy positions. One can place an article’s bias relative to whether it advocates or represents political positions more or less extreme than those of the median left and right positions of elected officials.

Comparison

Often, the best measure of the bias of an article is one that can only be made in comparison to other articles about the same topic. Reading laterally—meaning in other news sources—can show you how the present article compares with others in terms of what it focuses on and what it leaves out.

Another way to use the comparison factor is to ask yourself, “How could this article have been written in a less biased way?” or “Could I imagine a more biased version of this article?” These questions can help you determine if the article is just a bit left/right biased, significantly left/right biased, or extremely left/right biased.

Practice Develops Skill

It is important to understand that practice is an important part of being good at news literacy. Going through articles and thinking about their reliability in terms of their expression, veracity, headline, language, political position, and comparison may seem overwhelming at first, but once you start looking at them that way, it becomes easier.

These skills are less like learning to ride a bike, or that 3 x 3 = 9, or that the Declaration of Independence was signed in 1776; that is, they are not something that people can learn by being taught once. Learning news literacy is more like learning to read and write well—the more you practice, and the more techniques you implement, the better you get over time.

Fortunately, we have no lack of practice material to evaluate. Open up your news websites, scroll your social media feeds, and apply these evaluation factors to the articles you come across, and you’ll get more proficient at news literacy all around!

High school student evaluating news articles on her phone and tablet

Want more on news literacy? Check out the next chapters in this series:

See also:

About The Author

Vanessa Otero is the creator of the Media Bias Chart®, the viral image that prompted many lively discussions around the internet about where people get their news. She is the CEO of Ad Fontes Media, the company she founded in 2018 to fulfill the need revealed by the popularity of the Chart—the need for an easy-to-understand, reliable news rating system. Vanessa is a native of San Diego, CA, and currently resides in Westminster, CO. She has a B.A. in English from UCLA and a J.D. from the University of Denver Sturm College of Law, and is a practicing patent attorney. As the founder of Ad Fontes Media, she seeks to impact the way consumers, educators, and organizations deal with today’s complex media landscape.