Another look at a misleading graph. Part 1: Critique

On 2015-09-29 in a televised Congressional hearing, Representative Jason Chaffetz, Chairman of the US House Committee on Oversight and Government Reform, challenged Cecile Richards, president of the Planned Parenthood Federation of America (PPFA), to respond to this graph,

At the hearing, Ms. Richards responded to the graph’s misleading message, saying “It doesn’t feel like we’re trying to get to the truth here.”

Interviewed two days later, Chairman Chaffetz denied the graph was misleading. He ignored questions about the visual message and disingenuously focused on the text, saying, “I stand by the numbers.”

Chaffetz’s denial notwithstanding, displaying numbers visually conveys a visual message. And the message of this graph was crafted by Americans United for Life (AUL), an organization dedicated to restricting women’s reproductive rights in the US and abroad.

In these posts, I’d like to focus on the stories in these data. Here, in part 1, I critique the misleading graph and redesign it to explore the stories in the data more fully (as others have done, for example, Linda Qiu, Keith Collins, Tessa Dignum, or Kevin Drum). In part 2, I review the historical context that frames this event as yet another episode in a long struggle around birth control legislation in the US.

The rhetoric of the original graph

The vertical scales for the two arrows are different (a “dual y-axis”), implying that 327,000 is greater than 935,573.  To conceal the problem, the authors purposefully omit the vertical scales while retaining the horizontal time scale.

Note the hypocrisy: the authors use numbers and a conventional horizontal time scale to create the perception that their message has a rational basis in the data, yet they visually distort the relationships among these numbers to support their predetermined message. Visually, abortions appear to exceed cancer screenings by a ratio of more than of 3:1.

The authors promote their views further by selecting value-laden colors. Pink, associated in the US with breast cancer awareness, is good (“life-saving” in the graph subtitle), while red is bad, what the AUL calls “life-destructive”, implying blood and death (the AUL casts Planned Parenthood as “Abortion, Inc”).

The arrowheads distort the story, suggesting a “strong continuation of these trends, which we don’t know to be true” (Noah Iliinsky, interviewed by Linda Qiu).

The time scale implies that 8 years of data are displayed. However, numerical values are shown for only 2006 and 2013. And data markers — a visual convention of the scatterplot form — are not shown at all. Edward Tufte has argued that graphics like this have achieved a “graphical absolute zero, a null data-ink ratio.” (The Visual Display of Quantitative Information, 1983, p. 95)

Furthermore, that the lines cross creates the not-so-subtle visual implication that the trend in abortions is a dominant trend—which is the entire point of the deception.

Here, I show the four data markers, omit the arrowhead drama, and emphasize the text (the numbers that Chaffetz “stands behind”). The revised graph forces us to see how little information is being conveyed and how the relationships have been visually distorted.

Redesigning the graph

In redesigning the graph, as others did immediately following the hearing, my goal is to discover what undistorted stories these data tell. I obtained the data from Planned Parenthood annual reports (e.g., 2016, 2015, 2014, etc.), the same sources claimed by Jason Chaffetz and the AUL.

I start by showing the four numbers on the same vertical scale and applying a color scheme that is not value-laden.

Next, I include the data for all available years, including 2014 data which has become available since the time of the hearing.

Already the story has a nuance. The data provoke the question: what caused the decline in cancer screening and prevention services after 2009? The AUL claims the decline is evidence of Planned Parenthood’s changing business model, from a provider of “life-saving” to “life-destructive” services. A more likely explanation is the enactment of the Affordable Care Act (ACA) in 2010, providing more women access to health insurance thereby potentially reducing their reliance on Planned Parenthood. In addition, especially since 2003, the medical community has recommended less frequent procedures for the early detection of cancer (American Cancer Society).

The Planned Parenthood annual reports indicate that abortion and cancer screening and prevention are not the only services they provide. The next figure adds contraception, STI/STD testing and treatment, pregnancy and prenatal care, and “other” services: all the services in the data set.

This graph suggests we might want to find out what caused the increase in STI/STD testing and treatment or what’s behind the gradual decline in contraceptive services. The graph also shows that compared to the number of cancer services, between two and four times as many STI/STD services were provided.  A similar ratio obtains for cancer services compared to contraception. Moreover,on this common scale, abortion services appear to be fairly constant, especially compared to the dramatic changes seen in other types of service.

So why did the original graph compare abortion to cancer screening only? Cherry picking (ignoring inconvenient data): the decline in cancer screening and prevention services is the only trend in these data that supports AUL’s argument that Planned Parenthood is abandoning “life-saving” services.

In a final refinement of the data, I account for population increase. To avoid a distorted view of time-series data (especially over long time periods, which is not the case here), numbers of services should be studied on a per capita basis — in the same vein as accounting for inflation when studying dollar amounts over time. Using total population data from the US Census, the graph below shows the Planned Parenthood data on the basis of the number of services per 1000 people.

STI/STD services have seen a slight decline since 2011, a trend not seen in the earlier figure. Otherwise, nothing stands out as particularly noteworthy compared to the earlier graph.

Lastly, because Jason Chaffetz and the AUL are pointing out how many abortions Planned Parenthood provides, the rational comparison in these data is between abortion services and non-abortion services. The next graph supports Planned Parenthood’s assertion that abortion services account for a fairly constant 3% of all services they provide. (However, accounting for “services” rather than numbers of people served is problematic, as discussed by the Washington Post. Nevertheless, I use numbers of services here because those are the data values Chaffetz and the AUL quoted.)

What stories do the data tell?

Looking at this data in this way raises a new question. While per capita abortion services have remained fairly constant, non-abortion services have dropped off in recent years. Why? To study this question, we have to depart from the Planned Parenthood data that counts numbers of services rather than numbers of people.

The ACA was enacted in 2010. Between 2010 and 2014, the US Census Bureau reports that the percentage of the US population who were uninsured decreased from 16% to about 12% — a 25% decrease in this fraction.

Because a high percentage of Planned Parenthood’s patients can be assumed to be uninsured (in 2014, 75% had incomes at or below 150% of the federal poverty level), we might expect a correlation between decreases in the percent uninsured and the percent using PPFA services. And so it is. Between 2010 and 2014, the annual number of Planned Parenthood patients decreased from 3 M to about 2.7 M — on a per capita basis, a decrease from roughly 0.97% to 0.85% of the population — a 12% decrease in this fraction.

I conclude that these data, in context, tell these stories:

1. As the percentage of the population having health insurance increases due to the ACA, the percentage who are PPFA patients decreases.
2. Abortion services, as a per capita fraction of all PPFA services, remains fairly constant and small from year to year.
3. The total number all Planned Parenthood patients is less than 1% of the US population. And only a fraction of those are abortion patients.

Thus the visual argument of the original graph is contradicted by the very data they used: the PPFA is not predominantly a “life-destructive” organization.

Finally, Planned Parenthood receives approximately $500 M in federal funds (out of a $3.65 T budget in 2017) — that’s a minuscule 0.014% of the federal budget that serves about 1% of the population. What motivates AUL to create a lying graphic and Jason Chaffetz to use it in a public harangue against Cecile Richards when the dollar amounts and numbers of people are so small?

I examine that context in part 2.