CBO Overestimates Total Revenue By 1.1 Percent On Average With A Mean Absolute Error Of 5.2 Percent

From The Congressional Budget Office, November 10, 2015 Report, "CBO’s Revenue Forecasting Record:"

How Accurate Have CBO's Two-Year Revenue Projections Been?
On average, CBO has overestimated total revenues by 1.1 percent in its two-year projections—those that provide estimates of revenues for the fiscal year following the year in which they are released. A misestimate of that size in its January 2015 baseline projection, for example, would amount to $37 billion out of the roughly $3.5 trillion in total revenues that CBO projected for fiscal year 2016. Overestimates and underestimates offset one another in the mean error measure, so that average overestimate of 1.1 percent over the past three decades includes projections for years in the latest recession for which CBO overestimated revenues by as much as 25 percent and projections for the late 1990s and the mid-2000s for which CBO underestimated revenues by nearly 10 percent (see the figure below). The calculation of those errors—and of all such measures cited in this report—includes an adjustment to remove the estimated effects of legislation enacted after the projections were produced. That adjustment is necessary because the baseline projections incorporate the assumption that current laws governing taxes will generally not be modified by future legislation.

Source: CBO

***In addition to the mean error, CBO employs two other commonly used measures to evaluate the accuracy of revenue projections: the root mean square error (RMSE) and the mean absolute error. Unlike the mean error, the mean absolute error is the average of the errors without regard to direction (the negative signs are removed from underestimates before averaging), so errors in different directions do not offset one another. The RMSE, the calculation of which involves squaring the errors (thus removing the negative signs), also measures the size of errors without regard to direction, but by squaring the errors, it places a greater weight on larger deviations. The mean absolute error is an easier measure to understand, but the RMSE may be a more useful measure of forecast errors for revenue projections because larger forecast errors may have a disproportionately greater cost for policymaking than smaller ones.

For CBO's two-year revenue projections made since 1982, the mean absolute error is 5.2 percent, and the RMSE is 7.4 percent (see the table below). A mean absolute error of that magnitude would correspond to an error of about $180 billion in the revenue estimate for 2016 that CBO released in its January 2015 baseline projections. Because a disproportionate share of the misestimates occurred in projections made in years immediately preceding recessions, both the RMSE and the mean absolute error are roughly one-third smaller when the four two-year projections (out of the 32 included in this analysis) that were produced at or near peaks in the business cycle are excluded.

Source: CBO

***How Efficiently Has CBO Incorporated New Information?
CBO has tended to revise consecutive revenue forecasts in the same direction, suggesting that the agency does not efficiently incorporate new information into its forecasts. That tendency was less pronounced in the past 15 years than it was in the previous period, although the limited number of forecasts that can be assessed makes it difficult to conclude that CBO has improved its use of new information. That tendency, furthermore, has varied significantly over the entire history of CBO's forecasts, and CBO's forecast accuracy would not have been systematically improved had the agency incorporated into its forecasts what was known at the time about that tendency; such modifications would have over adjusted the forecasts in many cases.