Will the True Unemployment Rate Please Stand Up? Misclassification in the May 2020 Jobs Report

By Erica L. Groshen

June 10, 2020

The Bureau of Labor Statistics’ (BLS) May 2020 jobs report surprised pretty much everyone. After two months of historically unprecedented decline in labor market conditions, instead of more slippage we saw a partial rebound. Employment rose and unemployment fell. (See my post about it here.) Their expectations defied, many observers rushed to question the integrity of the numbers. They are wrong on all counts.

This article delves into the heavily tweeted and rebutted contention that the bureau mishandled a misclassification issue that it reported in the news release. Some critics see the issue as a sign of political tampering or incompetence. My comments here benefit hugely from what I learned while serving as the bureau’s commissioner from 2013 to 2017.   

One strand of doubt this week that I won’t dwell on are unfounded accusations of some unspecified tampering. I discuss and dismiss this issue generally in my explainer on the jobs report during the COVID-19 era. For May, let me also note that the rebounds seen in the independent employer and household surveys that form the basis of the jobs report are mutually reinforcing. Manipulating either without detection would be basically impossible, and so much more so to tamper with two.

To be clear, everyone is entitled to verify and offer constructive criticism of the bureau’s  methods. After all, statistical agencies are accountable to the public and have a duty to be transparent. Statistical agencies must earn the public’s trust in order to achieve their mission of informing public and private decisions. But that doesn’t mean that all criticisms are correct. Below I answer five questions many people may be asking right now.

What is the misclassification issue?

This is how the BLS describes the issue (see question 12 in this note) with the Current Population Survey (often called the “household survey” in the jobs report):

“Other than those who were themselves ill, under quarantine, or self-isolating due to health concerns, people who did not work during the survey reference week (May 10–16) due to efforts to contain the spread of the coronavirus should have been classified as ‘unemployed on temporary layoff.’ However, as happened in April and March, some people who were not at work during the entire reference week for reasons related to the coronavirus were not included in this category. Instead, they were misclassified as ‘employed but not at work.’”

Essentially, the category of “employed but not at work” refers to workers who are away from the job for an employee-initiated reason, such as illness, vacation, family leave, jury duty, or national guard service. Employees not at work because of COVID-19-related shutdowns belong in the category “unemployed on temporary layoff.” They do not need to search for other work to be in this category.   

The point is that neither the questions nor the interviewers have changed. Instead, when circumstances changed to something no one has ever encountered before, people responded to the survey in a way that led them to be misclassified.

Back in October 2013 (while I was there), the bureau first encountered this misclassification problem during the federal government shutdown. This 2013 note from the bureau explains the situation with nearly the same language and follows the same procedure used in 2020. Of course, the duration and magnitude of the COVID-19 shock means that this issue is far more consequential now than in 2013 or ever before.

How did this happen?

COVID-19 rapidly altered many things. Most BLS methods for tracking the labor market have proven solid, despite these disruptions. In retrospect, it’s no surprise that some survey questions did not withstand the extreme stresses imposed by the pandemic. Indeed, it’s a credit to the bureau that all the other Current Population Survey questions proved unproblematic.

For a picture of how things unfolded, I refer you to successive BLS notes for March, April, and May and this summary table:

My reading is as follows. In advance of the March survey, remembering the misclassifications in October 2013, the bureau reviewed lessons learned and issued the special COVID-19 guidance to the Census Bureau interviewers who field the survey. The purpose of the guidance was to ensure that temporary layoffs would be properly counted as such.

When the March survey responses arrived, BLS staff checked the data and saw a problem. Their best estimate is that about 1.4 million workers on furlough answered questions as if they were employed but out on some kind of leave. With the benefit of hindsight, this makes sense if many employers and employees had informal arrangements (with no name to call the situation), or agreed to charge the time to some leave category. Many probably had no prior experience with temporary layoffs. Self-employed workers or contractors with no business may also have answered in this way. And, many interviewers may not have seen this situation before. Combined with the 2.2 million workers correctly classified as on temporary layoff, about 39 percent of workers on temporary layoff were likely misclassified in March.  

In preparation for the April data collection, the BLS transmitted concern to interviewers and reiterated its previous COVID-19 guidance. No doubt, this information was received amid many other disruptions during that chaotic time. Unfortunately, the intervention failed to resolve the problem. Employers temporarily laid off a huge share of the workforce in April. Although a smaller share of temporary layoffs appear to have been misclassified than in March (30 percent), the scale of layoffs was much larger in April, so 7.5 million temporary layoffs were likely misclassified.

For May, the bureau redoubled efforts, describing it as follows:

“Due to the unusual circumstances related to the pandemic, Census Bureau interviewers were given additional training prior to collecting data in May. Supervisors held all-interviewer training sessions and reviewed the guidance that had been provided in March and April on how to record answers to the three survey questions detailed below.”

Yet, the problem persisted in May. The share of temporary layoffs likely misclassified declined further to 25 percent, but that still resulted in about 4.9 million likely misclassifications. The bureau plans urgent research to get to the bottom of these difficulties and craft a solution. No doubt, they will investigate differences between workers who were and were not misclassified, such as occupation, demographics, firm size, traditional employee, or contractor. All these answers can help the bureau craft better guidance or questions in the future. Sadly, in this fraught environment, some observers have latched on to this problem as evidence of tampering or incompetence.

Why doesn’t the BLS apply its estimated adjustment to “fix” the official unemployment rate?

Most notably, the misclassifications have introduced a consequential underestimate of the official unemployment rate, particularly for April. Table 2 shows the official unemployment rate along with the bureau’s estimate of the impact of the misclassifications by month. The figure below illustrates the impact visually:

As Table 2 and Figure 1 show, the impact of the misclassifications is consequential in both levels and changes. Yet, the bureau has announced that it will not use its misclassification estimates to alter the official unemployment rate for March, April, or May. There are at least three reasons for this decision:

• First, the bureau notes that changing respondents’ answers violates its practices. Changing answers carries the real risk of setting an unwelcome precedent, putting the bureau at the edge of the proverbial slippery slope. The future could see a rash of recoding demands for less legitimate reasons. In addition, the public could rightfully be less certain that published statistics reflect the underlying data collected.
• Second, the bureau’s estimated impacts cannot identify exactly which respondents made the errors. If it were to recode responses, staffers would have to select whose answers to change. So, any recoding would be essentially arbitrary and risk biasing other measures throughout the report, such as detail by race, sex, or region. Alternatively, if the agency adjusted the top line without changing underlying microdata, many other statistics would no longer add up to the total properly or be internally consistent.
• Third, the bureau’s method for estimating the impact of misclassifications has not been vetted. Official statistics methods are normally subject to rigorous review before they are used in government reports. However, in these circumstances, they performed no cognitive research with respondents, published no papers, and consulted no advisory committee. Experienced, world-class professional staff produced useful and credible impact estimates in a short amount of time. But this is not the normal process for producing an official statistic.

Which unemployment rate should you use, official or adjusted?

There is no one answer because it depends on the use you have in mind. Whatever your intended use, you can and should ask the bureau for guidance on how to proceed.

Using the official rate is probably the best advice in most cases. This will avoid confusion and ensure consistency with local or demographic subgroup unemployment rates, because demographic and local rates will not be adjusted for misclassification.

However, policymakers need to be aware that the official rate understates the severity of the recession for three months. To understand the depth of the downturn, the April “total” unemployment rate of 19.5 percent is probably better than the official one when comparing to the 1938 Great Depression maximum of about 25 percent or the 2009 Great Recession high of 10 percent. This comparability to historical statistics implies that some econometric modelers and researchers may opt to use the adjusted rate in their work. 

Going forward, I expect misclassifications to subside. For one, the bureau indicates that it intends to work closely with the Census Bureau to quickly reduce misclassifications. Also, when the recovery picks up steam, the number of people on temporary layoff will decline anyway, lessening the potential for misclassifications. 

What is the bottom line, and what lessons were learned?

The misclassifications were caused by COVID-19-related severe and unusual circumstances—not incompetence or tampering. The main consequence may be that some data users will forever add an asterisk to these three months’ national unemployment rate. That’s undeniably awkward, but chalk it up as one more cost of the COVID-19 pandemic.

I conclude with a few lessons to draw from this episode.

• Official statistics are critical infrastructure. No other source of labor market information was broad and authoritative enough to have alerted observers to May’s rebound before the jobs report came out. The ensuing debate and concern about the quality of the data was so fierce because this information really matters to decision makers throughout the country. Dollars invested in official statistics have a large payoff.  
• Making gold-standard statistics is difficult, especially during extraordinary times. National statisticians must be as transparent as possible and always be modernizing and preserving continuity. Even as the economy changes, they must accurately collect, process, and vet huge amounts of data on a very tight time frame. And, everyone is watching.
• Public trust in official statistics matters. A key contributor to preserving data integrity and public trust is maintaining the independence of statistical agencies from political interference. The many practices that preserve the BLS’s independence have served the agency well and helped to direct most people’s attention to the real issues.

Erica Groshen is an Upjohn Institute research fellow. She served as Commissioner of Labor Statistics, from 2013 to 2017. As commissioner, Groshen headed the Bureau of Labor Statistics, the independent, principal fact-finding federal government agency for statistics on labor markets conditions, inflation, and productivity.