Join FAIR's e-mail list & receive news, action alerts and much more...
A FAIR Research Memo: May 14, 1992
Study of Bias or Biased Study?
The Lichter Method and the Attack on PBS Documentaries
By Jim Naureckas
The Center for Media and Public Affairs, a conservative media research group, timed the release of its study of public TV programming to coincide with the congressional debate over public broadcasting reauthorization. The group’s report lends what appears to be empirical support to those who claim that PBS is biased to the left: "On the social and political controversies addressed by PBS documentaries across a full year of programs," it concludes, "the balance of opinion tilted consistently in a liberal direction."
An examination of the group’s findings, however, demolishes this conclusion. The study relies on methodology that ignores the overwhelming majority of material in PBS documentaries. It then draws sweeping conclusions based on the remaining, out-of-context material, and frames these conclusions in ways that are often misleading or deceptive.
The Center for Media and Public Affairs
The Center for Media and Public Affairs was founded in the mid-’80s by Robert and Linda Lichter, two academics who have made a career out of claiming to document leftist bias in the news media. Their stated mission was "to conduct scientific studies of how the media treat social and political issues," and they put great stress on their claim to non-partisanship. "It’s not in a scholar’s blood to have an ideology," Robert Lichter told the Washington Post (2/10/92).
The Lichters’ funding and history belie this stance of objectivity. From 1986 to 1988, Robert Lichter was a fellow at the conservative American Enterprise Institute. Fund-raising letters for the launch of the Center for Media and Public Affairs contained endorsements from leading right-wing figures like Ronald Reagan, Pat Buchanan, Ed Meese and Pat Robertson.
Robert Lichter’s writings and public statements also indicate a conservative worldview. At a conference sponsored by Accuracy In Media after the Gulf War, according to an AP report (4/27/91), "He said he was disappointed in statements by [Peter] Arnett upon his return from Baghdad that he was in the enemy capital on behalf of all CNN viewers, not just Americans. ‘I see a trend toward journalists seeing themselves as citizens of the world’ rather than patriotic Americans, Lichter said."
Funding for the Center has come from the most prominent foundations of the right, including Smith Richardson (at least $298,000), Olin Foundation ($250,000), JM Foundation ($100,000) and the Coors Foundation ($55,000). (Smith Richardson gave the Center $40,000 in 1987 for its study on PBS.) These foundations also contribute heavily to more overtly right-wing media pressure groups like Reed Irvine’s Accuracy In Media, L. Brent Bozell’s Media Research Center, and David Horowitz’s Committee on Media Integrity.
The Scaife Foundation, another major right-wing funder, gave the Lichters money for their book, The Media Elite, which argued that journalists’ personal political biases made their work unreliable. (The same argument, of course, could be made about academics like the Lichters.) The study featured in the book, based on interviews with journalists conducted in 1980, was widely criticized by scholars for methodological flaws. (See Columbia Journalism Review, Nov/Dec ‘85, March/April ‘87; Journalism Quarterly, Winter ‘87; Journal of Communication, Spring ‘88.) Although it has long been touted as proof of liberal journalistic bias, the study (based on a small, dubiously representative sample) failed to prove much of anything: On only 11 out of 20 questions cited in The Media Elite did a majority agree with the "liberal" response—e.g., "The government should not attempt to regulate people’s sexual practices."
Mainstream reporters initially tended to report, based on the Lichter’s right-wing funding and their predictable claims of leftist bias, that the Center was "conservative" or "right-wing." Lately, however, journalists seem to be giving the Center’s claims to be apolitical more credence. The L.A. Times’ Tom Rosenstiel praised their "non-partisan" approach in an interview in the D.C.-based City Paper (2/30/90). USA Today (6/28/91) also called them "non-partisan," and Newsday (3/4/92) referred to them as "non-ideological."
The Lichter Methodology
Despite the Lichters’ objective posture, the methodology used in most of their research is not scientific. They have used it in the past to "prove" entirely dubious claims, such as the idea that Jesse Jackson was the candidate with the most positive news coverage in 1988, or that George Bush got as much negative coverage as Saddam Hussein during the Gulf War.
In analyzing media coverage, the Lichters single out what they judge to be "thematic messages"—explicit statements of opinion or evaluation. Usually the Lichters determine that such statements make up a very small proportion of the statements found in news reporting—yet proceed to generalize about coverage as a whole based on this tiny percentage.
The Lichters’ tendency to generalize from a narrow sliver of data is the main way that their studies end up supporting their preconceived conclusions of left bias. Take the Center’s report on Gulf War coverage (Media Monitor, 4/91) and its widely cited claim that "nearly three out of five sources (59 percent) criticized U.S. government policies during the [Gulf] War." This, of course, is not 59 percent of all 5,915 sources, but of those 249 sources (4.2 percent) who in the Lichters’ judgment stated an explicit position. This leaves only 148 sources, or 2.5 percent of all sources, who made explicit criticisms of U.S. policy (from the left, right or center).
On what basis can you generalize from the 4 percent of sources who supposedly expressed overt opinions to the 96 percent who didn’t? Doing so results in absurd claims, such as, "Surprisingly, the U.S. government fared little better than its Iraqi counterpart in the soundbite battle." That would be surprising, considering that 44 percent of total news sources were from the U.S. government, according to the Center’s own research.
The Lichters have also been known to stress partial data when a more comprehensive statistic would not prove the bias that they seemed to be looking for. For example, the Center’s report on abortion coverage (Media Monitor, 10/89) trumpeted this finding on the front page: "Pro-choice activist sources outnumbered their pro-life counterparts by a five to three margin." What wasn’t noted on the front page is that the anti-abortion position was often represented by government officials and other non-activist sources (who may speak with more authority than activists to the average news consumer). There is a statistic in the report that includes viewpoints from all sources: "On our summary measure of views on abortion policy, the pro-choice side had a slight edge (53 percent to 47 percent)." This is the more inclusive but less dramatic statistic—and it was buried on the last page.
Under the guise of revealing patterns of bias, what the Lichters really uncover are patterns of rhetoric. The Center’s abortion study found that 75 percent of media sources on abortion favor abolishing Roe v. Wade, yet 66 percent think abortion should be legal. Are these sources schizophrenic? No: The Lichter method simply picked up on the way activists talk. Pro-choice people favored the slogan "keep abortion legal," while anti-abortion forces rallied around "overturn Roe v. Wade."
Yet the Lichters constantly treat such semantic differences as if they indicated real biases in the media: "The pro-choice side dominated the legalization debate. But the pro-life side won out in the debates over Roe v. Wade’s status, government funding, morality and the outset of life." (For more on the Center’s abortion study, see FAIR’s research memo, "Do the Media Have a Pro-Choice Bias?")
The PBS Study
The Center’s study of PBS looked at 225 documentary programs, which took up 222 hours of airtime between April 1, 1987 and March 31, 1988. (The broadcasts are nearly five years old because the Center abandoned the study when it failed to get sufficient funding, then picked it up again when PBS became a hot political issue.)
The Lichters’ study of PBS is notable for what it leaves out: It excluded talkshows such as William F. Buckley’s Firing Line and Morton Kondracke’s American Interests, news reports like the MacNeil/Lehrer NewsHour, and business programs like Louis Rukeyser’s Wall $treet Week. The Center claims this is to ensure "a group of programs that were similar in style and content, to maximize the comparability of judgments."
The study’s focus, however, removes those PBS shows most often criticized for having a conservative slant—programming that takes up more of the PBS schedule than the documentaries that the Center’s study is limited to. Firing Line and American Interests —programs underwritten by the Center’s biggest funders—provided approximately 50 hours of programming a year between them.
The Center’s researchers broke down 225 documentary programs into 35,094 segments, an impressive-sounding number that features prominently in the summary of the report distributed to the press. However, most of these 35,094 segments were not analyzed for political content. Only 614—1.7 percent of all segments— "clearly stated a thematic message," and these were the basis of all the Center’s conclusions about the politics of PBS documentaries. The other 98.3 percent of statements in documentaries have no bearing on political slant, according to the Center’s methodology. A "thematic message" occurs, on average, approximately 2.7 times per PBS documentary, according to the Center’s study.
Often, most segments with a clear "thematic message" on a particular issue the Center examined (such as nuclear power, or the right to privacy) come from one or two programs or a single series—an indication that the segments do not say anything meaningful about the general drift of PBS programming.
The claim that PBS has a liberal bias is argued in a section of the report called "The Battle of Ideas," made up of nine subsections ("War,'' "The Environment," etc.). The "empirical" basis for each section is an interpretation of the relevant "thematic messages," and in each case is based on an extremely small number of segments, and/or on a misleading, sometimes deceptive presentation of those segments. Frequently, the descriptions of findings contained in the executive summary of the report are at odds with the more extended descriptions in the full report.
War:
In this category, the most significant finding, in terms of number of segments analyzed, was that war was more often described as "a personal tragedy" rather than as "a geo-political event." But in expanding on what this finding means, the report states that war was most commonly described as "a personal rite of passage or a moment of horror successfully survived"—not at all equivalent to "a personal tragedy."
All other conclusions on PBS’s "bias" about war are based on a mere 18 segments—only 0.4 percent of the 4,042 total segments about war, and 11 percent of the "thematic messages." In general, the report argues, PBS has a pacifistic bent—even though 1,309 military personnel appeared as sources on documentaries during the period studied.
The report complains that "there were no programs in our sample that set out to justify war." The view that war is generally bad and is to be avoided appears to the Lichters as an example of liberal bias.
Environment:
According to the Center’s summary, on the question of "what balance (if any) could be struck between human needs and protecting imperiled ecosystems," 61 out of 100 segments analyzed said that "the environment must be preserved above all else." But that is not at all what the data showed. According to the full report, "those who argued that environmental protection took precedence over human needs usually offered one of two rationales. Most common were arguments that preserving habitats and biodiversity were beneficial to mankind because of the potential medicines and other products that might be found among unknown species.... The other major rationale was that squandering our resources would eventually imperil our very existence." Clearly, these two rationales are not arguments that preserving the environment takes precedence over human needs; they are arguments that human needs depend on preserving the environment.
According to the study, "preservation and conservation...were the cornerstones of PBS environmental documentaries." However, over the course of a year, the Center found only 100 "thematic messages" dealing with the environment—some opposed to environmental protection—on 47 shows dealing with nature and environmental protection. That amounts to about two messages (most pro-environment, some anti-) on each show—hardly a drumbeat of propaganda.
Disadvantaged Groups:
"Disadvantaged Groups" is the heading that the Center uses to discuss PBS coverage of minorities and women. Ironically, the section claims that PBS coverage is biased because it acknowledges that women and minorities are disadvantaged.
"Racial discrimination was described as a condition of American society 50 times without a single dissenting opinion," according to the report’s summary. Actually, discrimination was described as a former condition of U.S. society in 37 of these 50 segments—only 13 segments dealt with contemporary U.S. racism. And the study must be read carefully to find that both the 50 and 13 figures include people who approve of segregation, or "criticized efforts to increase integration." Whether they said it’s good or bad, they all acknowledged that discrimination exists, so they’re counted as "liberals."
The report implicitly criticizes a statement from an African-American: "I think we need to do for ourselves. We need to build our own institutions and our own businesses and our own jobs, so that we can change the conditions we’re in." The sentiment echoes the rhetoric of black conservatives like Clarence Thomas who criticize federal programs aimed at helping the poor, yet it is used by the Lichters as evidence of liberal bias.
All analysis of women’s issues is apparently based on just 13 segments. This is remarkable evidence of how seldom women’s issues were discussed on PBS.
The Constitution:
The Lichters’ examination of PBS discussion of the Constitution provides a case study of how the Lichter method detects not media slant, but standard patterns of rhetoric. Thus by a 7-1 margin, sources who professed an opinion supported free speech and a free press—neither liberals nor conservatives like to portray themselves as opponents of free speech. By a 9-0 landslide, sources took the non-controversial position that "the Constitution is a good tool for governing." The Lichters found a broad right to privacy supported in six out of seven cases—all but one of which occurred in an interview with Supreme Court Justice Harry Blackmun. Similarly, an interview with Robert Bork and Edwin Meese provided nearly all of the soundbites that led the Lichters to declare that strict constructionism won out over loose constructionism, 6-1. In reality, of course, the views expressed in one or two five-year-old documentaries say nothing about the overall slant of PBS programming, either then or now.
Health Care:
Health care was the topic of 3,066 segments in PBS documentaries, according to the Lichters. Their report points to 20 segments (0.7 percent) that have a "liberal" slant (questioning doctors or the medical industry), and a further 15 (0.5 percent) that have a pro-medical spin. Some of the examples the report cites to show "liberal" bias are peculiar—for instance, a Catholic priest questioning the morality of in vitro fertilization.
Religion:
The study found 20 segments arguing that "religion should advocate social change." The study noted that this sentiment came from both conservative and progressive religious sources, though it presented no data about the political breakdown of such sources. No other statistical information is presented about PBS’s depiction of religion, though the topic is presented as if it backed up the claim that on PBS, "the balance of opinion tilted consistently in a liberal direction."
Foreign Topics:
"The only foreign country to receive extensive treatment was South Africa," the study’s summary claims, indicating that PBS might have a disproportionate interest in a subject that might be considered a left or liberal cause. Yet the study’s own data shows that South Africa, the subject of five documentaries, did not receive exceptional attention. The Soviet Union was featured in 12 programs, while Japan and China were the subject of five each. Western Europe as a region was the subject of 24 programs, while Eastern Europe was the focus of 11. It is impossible to square this data with the Center’s claim that "no other country or issue received extensive treatment on the order of South Africa and its apartheid system."
The treatment of South Africa is quite revealing of the Lichters’ underlying politics. The study makes the claim that "friends and allies of the United States were targeted for criticism more than four times as often as enemies or unfriendly nations." "Most" of this criticism of "friends and allies," the study goes on to state, was directed at South Africa, a nation then facing sanctions from the U.S. aimed at altering its system of government. A significant amount of the remainder of criticism of "friends" was directed at "the Philippines under Marcos," a ruler whom the Reagan administration helped to depose. To use criticism of Marcos and apartheid as evidence of anti-Americanism says more about the Lichters’ bias than that of PBS documentaries.
The report notes that in programs on South Africa, apartheid "was condemned by over two out of three sources (69 percent)," then goes on to disclaim the finding: "Even this division may be misleading, since the statements from apartheid’s defenders tended to be so extreme as to lack credence within the American political culture." In other words, PBS already shows a "bias" against apartheid, and the bias is "even" worse than it appears statistically because the defenders of apartheid were too extreme to be taken seriously. Besides implying that there is a credible, moderate case to be made for apartheid, the statement points out the weakness of the Lichters’ entire methodology: If a simple tabulation of pro and con statements about apartheid does not indicate the actual balance of the debate, then how can any similar statistic, taken out of context, prove bias?
Four of the five criticisms the Center could find of "unfriendly" (i.e., left-wing) countries occurred in one documentary, a conservative critique of Angola. That the 31 programs on the Soviet Union, China, Eastern Europe and Nicaragua only contained one statement criticizing those governments for "undemocratic politics or conditions of economic hardship" strains credulity.
PBS Sources
The Lichters prominently acknowledge that those who speak on PBS are predominantly white and male, and that women in particular are greatly underrepresented in comparison with the general population. (Only 14 percent of program participants were women, and 17 percent were people of color.)
The Lichters put less emphasis, however, on a statistic from their research that greatly undermines their thesis. They would have one believe that the agenda of liberal groups controls PBS. Yet what the study terms "special interest groups"—"the feminist movement, the environmental movement, pro- and anti-nuclear power groups and organized labor"—were heard in only 223 out of 35,094 segments (0.6 percent). In comparison with these "special interests," PBS viewers were six times as likely to hear from corporate representatives (1251 segments) or military personnel (1309 segments), and nine times as likely to see government officials (2101 segments). These ratios, rather than the highly dubious sampling of "thematic messages," may provide a truer picture of the slant of PBS documentary programming.