How we measured U.S. Forest Service wildfire prevention work

By George LeVines (The California Newsroom), Emily Zentner (CapRadio)

Nearly two decades ago, the United States Forest Service identified the hillside town of Grizzly Flats, California, as being at high risk of burning in a wildfire. Over the following years, the agency devised and approved a plan to protect the town and the surrounding habitat by cutting down trees, removing brush and using controlled burns on 15,000 acres of its land to the south and east of Grizzly Flats. They called it the Trestle Project.

Work on the ground did not begin until 2018. In August 2021, the Caldor Fire reduced most of the town to ash and rubble. When the fire ignited, the Forest Service had completed just 14% of the Trestle Project. Here’s how we came to that conclusion.

Background

The Forest Service primarily tracks work that counts toward wildfire mitigation projects in two different datasets within its Forest Service Activity Tracking System (FACTS) database. One is the Hazardous Fuel Treatment Reduction dataset and the other is called the Timber Harvests dataset. Both are required to get a robust picture of wildfire mitigation work. 

The hazardous fuels data track a variety of landscape treatments designed to reduce the fuel (trees, plants, etc.) available to a forest fire. Examples of those activities include piling, thinning, prescribed burning, chipping, rearranging, crushing and making fuel breaks. (Technical definitions are available here.) The timber harvest data include activities that involve the commercial sale of timber including cutting the larger trees in an area and salvage logging after a fire has burned through.

The records are incomplete, inconsistent across jurisdictions, and contain errors, according to the Forest Service.

During the review of our methodology the agency discovered several treatments in FACTS that had been erroneously labeled “complete,” leading to confusion about how much work was actually done. It turned out the Forest Service had completed less work than it was reporting. The agency said it corrected the data on August 10, but for our analysis we used data provided directly by the Forest Service via email, rather than the FACTS figures available on the agency website. 

The quality and timeliness of the data entries by the Forest Service also vary widely from forest to forest and district to district. For example, in the case of the Trestle Project the agency made additions and adjustments to records of treatments throughout our nine-month investigation, even though those treatments were completed months prior. Additionally, some districts track planned treatments while others do not, and some forests use one column to calculate acreage while others use another column. 

The Forest Service aims to keep its records up to date and consistent across the entire agency, but it’s difficult, said Eldorado National Forest’s District Ranger Scot Rogers.

“We barely have enough folks to get the work done on the ground as it is,” Rogers said. “So when the question comes of, ‘Am I going to have someone in front of the computer filling out a spreadsheet — particularly in the middle of summer — or … continuing to do fuel work,’ I'm going to default to having them out there in the woods.” 

When not in the woods, Forest Service employees will manually enter treatments into the databases, doing so in a way that makes measuring the total number of acres covered challenging. When multiple treatments happen on the same parcel of land, each treatment gets its own record. It’s not uncommon to see “hand or machine piling” in one row of the data followed by “burning of piled material” in another row, both referencing the exact same geographic area. Earlier in August, NBC published an investigation into the long-standing criticisms regarding how the Forest Service often uses data that gives the impression it has treated much more land than the reality on the ground.

Handling overlapping treatments

The Trestle Project boundary is a singular patch of more than 20,000 acres in the Eldorado National Forest, within which 15,000 acres of treatment was proposed and approved. We wanted to know how many of those 15,000 acres were completed on the ground, where those treatments overlapped, what type of treatments were performed and when the work was done.

To get those numbers we used several geographic information systems (GIS) transformation techniques  in a software program called QGIS, namely merging, dissolving and clipping. This combination had the effect of reducing any overlapping projects into a single layer. We then calculated the area of that layer. We found the Forest Service treated 2,137 acres of the 15,000 planned by the time the Caldor Fire burned; the layer looked like this.

We ran our analysis past several experts in the field and the Forest Service itself. Experts and the agency validated our method.

The Forest Service did, however, dispute our classification of what counts as “prescribed fire.” The agency said it considers any fire it lights intentionally a prescribed fire. However, this practice can mislead because it lumps together high impact treatments and lower impact treatments in a way that retired two-decade Forest Service Senior Ecologist Hugh Safford even described as “white washing” the data.The agency also wanted our investigation to factor in all Grizzly-Flats-area treatments performed in the 20 years leading up to the Caldor Fire. But most experts we talked to said treatments are typically only effective for 10 years, while a 2019 Government Accountability Office report said eight to 12 years in mixed-conifer landscapes such as the Eldorado National Forest. Taking all of this into account, our analysis considered treatments in the 15 years leading up to the Caldor Fire, the absolute maximum considered reasonable by any of the experts we talked to.

We used this same methodology to calculate total treated acres in other parts of the state, which will be published later in this series.

If you’re interested in reproducing these findings for your area, step by step instructions are available here. Please contact George LeVines with any questions or feedback.