In the press release for their recently released report on the EPA’s 2022 draft formaldehyde assessment under the Integrated Risk Information System (IRIS) program, a committee of the National Academies of Sciences, Engineering, and Medicine (NASEM) claimed that the EPA “follows the advice of prior National Academies reports and that its findings on hazard and quantitative risk are supported by the evidence identified.” Members of Congress applauded “the IRIS program on receiving this stamp of approval from the Academies.”
However, a closer read of the report reveals that the NASEM committee was constrained in what they reviewed and yet still raised dozens of substantive scientific and methodological criticisms of EPA’s work. For example:
- The NASEM committee admits that it did not conduct an independent assessment and was limited to responding to EPA’s questions and only reviewing material provided by EPA.
- The NASEM committee was required by EPA to exclude from their review other comprehensive assessments of formaldehyde (including by the World Health Organization, European Chemicals Agency, and the European Commission’s Scientific Committee on Occupational Exposure Limits) as well as key peer-reviewed studies (as with EPA’s draft assessment, more than 70 key studies or reviews of formaldehyde were fully ignored or dismissed) as well as public comments.
- The NASEM committee acknowledges that there is no “static benchmark” to evaluate whether EPA’s methods or conclusions have improved. Moreover, they did not evaluate if EPA’s assessment meets legal requirements for the use of the “best available science.” Instead, the NASEM committee indicates that many EPA methods were “consistent with EPA’s state-of-practice approach,” a meaningless characterization for evaluating EPA’s scientific quality, and did not review whether changes in the 2022 EPA draft assessment addressed 2011 recommendations from NASEM.
- The NASEM committee provided detailed critiques of EPA’s approaches to systematic review protocols, toxicity values for myeloid leukemia, study evaluation and inclusion, transparency in methods, causal determinations and terms (like whether evidence “demonstrates” a particular health outcome), biological plausibility, exposure evidence, potential types of bias, use and reanalysis of raw data, need for a more rigorous “reality check” on certain health claims, and characterizing uncertainty.
- The report acknowledges that the committee could not replicate key aspects of EPA’s approach for the one study it fully evaluated.
Given the extremely limited scope of NASEM’s charge, and therefore a lack of a comprehensive review of the best available science, we believe that the report is not fit for use in the regulatory decision-making process. Furthermore, the press releases from NASEM, EPA and Members of Congress are misleading and a deeper dive is needed to bring to light the problems identified.
Because of this narrow scope and major inconsistencies with legal and scientific standards for regulatory science, ACC believes that EPA must take the time to fully evaluate and address all of NASEM’s recommendations, the scientific considerations that have been raised by other peer review and regulatory bodies, and the substantial docket of public comments that have been submitted.
The Substance of the Report Contradicts Summary Message
NASEM’s favorable summary of EPA’s draft assessment and the reception from certain stakeholders is at odds with the substance of its own report, which contains multiple criticisms about various aspects of the draft assessment accompanied by over 40 recommendations. Chief among NASEM’s concerns was the overall lack of accessibility, reproducibility, and transparency of EPA’s draft assessment, noting that the agency needed to revise its assessment to “ensure that users can find and follow methods used in each step of the assessment for each health outcome.”
Nine other NASEM recommendations focus on transparency and related issues, all of which underscore ACC’s long-held concerns surrounding the overall lack of transparency in EPA’s draft IRIS assessment process.
The unresolved recommendations are referenced but summarily dismissed in NASEM’s latest document, stating that “2022 Draft Assessment responds to the broad intent of the 2011 NRC report.” It’s worth noting that the report also mentions that “the present committee did not review specific changes in the 2022 Draft Assessment against the recommendations in the 2011 NRC report…” One wonders how NASEM could have arrived at a satisfactory conclusion about EPA’s progress on the committee’s 2011 recommendations without reviewing their progress against the 2022 draft assessment. [See Insufficient EPA Responses to 2011 NAS Recommendations.]
The numerous criticisms and recommendations detailed in the report are downplayed in support of its favorable sounding headline and conclusion. This only reinforces the concerns ACC has raised over the last year and paints the picture of a rushed, rubber-stamped and pre-determined process that is ignoring scientific rigor in favor of EPA’s desired outcome.
Disregard for the Best Available Science
NASEM’s ability to provide a comprehensive review of the body of scientific literature was constrained by EPA, which imposed a narrow and rigid set of charge questions for NASEM to answer and constrained its ability to exercise independent scientific judgment or consider contrary evidence.
This narrow scope precluded any consideration of the best available science.The Federal Advisory Committee Act requires that EPA cannot use the advice of NASEM unless the report is “the result of the Academy’s independent judgment” and the review process is not under the control and management of the sponsoring agency.
Furthermore, NASEM followed EPA’s lead and ignored or dismissed over 70 key peer-reviewed studies, most international formaldehyde assessments, and comment from authors of key studies.
Over the last decade, more than 40 peer reviewed studies have demonstrated safe thresholds for formaldehyde exposure, including ground-breaking methodologies published in 2021, award-winning re-analysis of key data, and several lines of evidence demonstrating the safe use of formaldehyde under normal conditions of use. Many of these high-quality studies and reviews are ignored, briefly dismissed, or misused by EPA in the draft assessment (as noted in comments from the Small Business Administration).
Significantly, NASEM also fails to fully address the validity of the toxicity values in EPA’s 2022 draft IRIS assessment, stating “the committee did not conduct an independent hazard evaluation or dose-response assessment, and therefore does not recommend alternative hazard identification conclusions or toxicity values.” This limitation is a significant failure and disregards the scientific evaluations from global health agencies, regulators, and other international authorities who have used decades of scientific evidence by universities and independent scientists to support a safe threshold for formaldehyde exposure and no causal association with leukemia.
The potential consequences of EPA’s draft formaldehyde assessment dictate that the agency take time to fully evaluate and address all the recommendations in the NASEM report as well as other scientific considerations that multiple peer review and regulatory bodies have raised. A rushed process that ignores the broad body of scientific studies around formaldehyde could unnecessarily cause public alarm and lead to inaccurate decision-making, as well as other unintended, adverse effects. As ACC outlined in great detail in March 2022, this failure by EPA and NASEM to fully resolve these recommendations may also run afoul of numerous legal requirements.
Report Criticizes EPA’s Failure to Follow Process Resulting in a Faulty Systematic Review
NASEM clearly states in its report that, “[t]he IRIS Handbook specifies the publication of an assessment plan, which describes what the assessment will cover, and a systematic review protocol, which describes how the assessment will be conducted (EPA, 2022).” Yet, inexplicably, formaldehyde is the only chemical under review by the IRIS Program for which EPA has not developed an assessment plan or systematic review protocol. [See Inconsistencies Between FA Assessment and IRIS Handbook.]
NASEM criticized EPA for deviating from its own guidelines, noting that “the assessment does not satisfactorily follow recommendations for problem formulation and protocol development. EPA did not develop a set of specific protocols for the 2022 Draft Assessment in a fashion that would be consistent with the general state of practice that evolved during the prolonged period when the assessment was being developed... prepublished protocols are essential for future IRIS assessments to ensure transparency for systemic reviews in risk assessment.”
Failure to Consider Public Input
NASEM similarly ignored all public comments noting that it did not consider other interpretations of scientific information relevant to the hazards and risk of formaldehyde, or alternative opinions, because of the narrow scope of their review. However, NASEM acknowledges the more than 40 individuals who shared their perspectives during the public comment periods and expressed gratitude for “these valuable contributions to its work.” The September 2021 contract between EPA and NASEM established, prior to public comment or solicitation of panel members, that the committee “shall not conduct an independent assessment separately from the IRIS document nor shall the NAS comment on the broader aspects of the of the IRIS program” and the constraint that the committee “shall be limited to responding to the materials provided by the EPA” with a directive that other background information provided to the Committee by others “shall not be reviewed by the NAS.” This prohibition including information provided by public commenters or other authoritative bodies.
ACC is not alone in highlighting concerns with this formaldehyde assessment, with broad scientific criticism from authors of key studies, other federal agencies, Members of Congress, former government officials and advisors, and numerous stakeholder groups. It’s no wonder, then, that the current review has precluded questions about the IRIS process and whether prior peer review recommendations were fully addressed.
ACC Takes a Stand
ACC has made clear that we are concerned about EPA and NASEM’s failure to comply with the Federal Advisory Committee (FACA) transparency, balance, and independence requirements in the peer review process. Absent any corrective actions, ACC, on behalf of a working group of ACC members, took legal action to try to ensure that NASEM and EPA comply with their obligations. The complaint, outlines how this review process fundamentally fails to follow FACA and the basic standards for scientific integrity, independent peer review, and governmental transparency.
In conclusion, ACC and our members hope to restore balance and objectivity to the NASEM process, improving the integrity resulting from this third-party peer review.
We need strong, independent, science-based assessments as the basis for federal regulations—and this report does not pass the test.