ARC logo

AAAS:
On the OMB Proposed Peer Review Bulletin

Whereas the Office of Management and Budget (OMB), Executive Office of the President, through its “Proposed Bulletin on Peer Review and Information Quality,”1 is now considering requiring that all federal agencies institute a system of “peer review”;

Whereas OMB’s proposed "peer review" procedures differ markedly from those used and strongly endorsed by the scientific community;2,3,4,5

Whereas there is no evidence that proposed new procedures are likely to improve the quality of science used in the regulatory process;6,7,8

Whereas the proposal unnecessarily limits agency discretion; is likely to result in delays in protecting the nation’s health, safety and environment;9,10,11,12,13and may constrain public health officials from reacting quickly in times of national emergency;14,15and

Whereas the proposal has conflict of interest provisions that could restrict many of the nation’s leading scientific experts from serving as peer reviewers16, 17;

Therefore, be it resolved that the AAAS Council is strongly opposed to the proposal, requests that OMB withdraw the proposal, and encourages other scientific organizations to oppose this proposal.

REFERENCES

1 Office of Management and Budget. Proposed Bulletin on Peer Review and Information Quality Bulletin. September 15, 2003. Available at: http://www.whitehouse.gov/omb/fedreg/030915.pdf

2 NAS President Bruce Alberts wrote to John Graham that “the highly prescriptive type of peer review that OMB is proposing differs from accepted practices of peer review in the scientific community, and if enacted in its present form is likely to be counterproductive.” http://www.whitehouse.gov/omb/inforeg/2003iq/iq_list.html

3 “Calling their [the OMB] process peer review seems strange to us here” wrote Donald Kennedy, editor of Science magazine in “Disclosure and disinterest” (303 Science 15; January 2, 2004.) Kennedy is former President of Stanford University President and a former Commissioner of the Food and Drug Administration. He also chairs the National Academy of Sciences’ Science Technology and Law Program.

4 According to Sheila S. Jasanoff, Pfozheimer Professor of Science and Technology Studies, John F. Kennedy School of Government, Harvard University, the OMB authors of the proposal fail to understand the difference between regulatory science and research science. “The reliability and success of regulatory, or policy-relevant, science cannot and should not necessarily be measured according to the same criteria as the reliability and credibility of ordinary research science, which is investigator-initiated or ‘curiosity-driven.’ …[T]he success of regulatory science includes its capacity to provide timely answers to pressing policy questions; research science operates under no comparable time pressures. Correspondingly, the procedures used to ensure the reliability and credibility may reasonably differ from one scientific context to another.” [Letter to OMB, page 2] Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/iq/159.pdf

5 Referring to its potential impact on FDA approvals of new drug applications, the Pharmaceutical Research and Manufacturers of America (PhRMA) asserted that the proposed requirements “would contribute little value and would add to the time and expense of a gatekeeper function that has historically been criticized for obstruction and delay.” [Letter to OMB, page 7] Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/iq/118.pdf

6 Michael R. Taylor, Senior Fellow and Director, Risk, Resource, and Environment Division, Resources for the Future; and former Deputy Commissioner for Policy, Food and Drug Administration, noted “…I do think it is striking that it offers peer review as a solution, but it doesn’t define the problem that it sets out to solve…[and] the proposal does not provide any examples…of cases in which the lack of adequate peer review has had important consequences for the regulatory process or society.” Available at: http://www7.nationalacademies.org/stl/Peer_Review_Transcript.pdf [transcript p. 210]

7 John C. Bailar III, Professor Emeritus, Department of Health Studies, University of Chicago remarked, “I would like to see a list of situations in which peer review would have been a substantial help. If nobody can produce such a list, to create such a list, that in itself would be very informative.” Available at: http://www7.nationalacademies.org/stl/Peer_Review_Transcript.pdf [Transcript p. 269]

8 The Association of Medical Colleges (AAMC), representing the nation’s 126 accredited medical schools, 400 affiliated teaching hospitals and 94 medical societies, and the Federation of American Societies for Experimental Biology (FASEB) with more than 60,000 member “…we see no public benefit from mandating an additional layer of OMB imposition, peer review and public comment…” [Letter to OMB, page 3] http://www.whitehouse.gov/omb/inforeg/2003iq/iq_list.html

9 Two professors from Rutgers University, Stuart Shapiro (who had been a desk officer at OMB under Dr. Graham) and David Guston, provided an estimate of the costs and benefits of the proposal, applying the methods OMB would use if it were a proposal sent by an agency. Professors Shapiro and Guston’s best estimate of the cost of the proposed peer review system is $325 million annually: the monetary value of the effects of delaying regulation that saves lives, protects the environment or produces other societal benefits. Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/iq_list.html

Michael R. Taylor, Senior Fellow and Director, Risk, Resource, and Environment Division, Resources for the Future; and former Deputy Commissioner for Policy, Food and Drug Administration noted: “Public health agencies act all the time on the basis of emerging information, and in the face of scientific uncertainty. That is what we want them to do. This doesn’t mean they should act capriciously or without careful review of the information before them. The stakes are too high to accept anything less than great care and allegiance to sound scientific practice. But if we want our public health agencies to act like public health agencies, we should not impose formal review requirements that may preclude their doing so. Thus, any OMB review mandate should include an explicit exception for cases in which the FDA commissioner, not the OIRA administrator, determines that action must be taken to protect public health.” Available at: http://www7.nationalacademies.org/stl/STL_Peer_Review_Agenda.html [Transcript p. 218]

10 Sheila Jasanoff, (noted above, and the author of The Fifth Branch: Science Advisors as Policymakers, Harvard Univ. Press, 1990) and one of the nation’s leading thinkers on the use of science in public policy warned against using peer review to “…offset the need for public-health type judgment or governmental judgment on the behalf of people, I think would be a serious mistake.” Available at: http://www7.nationalacademies.org/stl/STL_Peer_Review_Agenda.html [Transcript p. 87]

11 Professor Sheila Jasanoff noted further in written comments to OMB, “Through its wide-ranging application and the stringency of its requirements, the proposal is likely to have significant impact on the time and cost of policy development—and, by extension, on the capacity of regulators to effectively protect public health, safety and the environment.” Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/iq_list.html

12 Criticizing OMB with unusually direct language, the Pharmaceutical Research and Manufacturers of America (PhRMA) asserted that the proposed requirements[on FDA] “would contribute little value and would add to the time and expense of a gatekeeper function that has historically been criticized for obstruction and delay.” [Letter to OMB, p. 7] Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/iq/118.pdf

13 Warren M. Washington, Chair, National Science Board; Senior Scientist and Head, Climate Change Research Section, National Center for Atmospheric Research noted: “…in some of the emerging areas of innovative research, we are finding that we only have a limited number of peers to serve as reviewers. And therefore, we have to be fairly careful about overusing some of these people. …And so, we are quite seriously concerned about imposing another layer of review burden on the science community.” [Transcript, p. 232] Available at: http://www7.nationalacademies.org/stl/Peer_Review_Transcript.pdf

14 Michael R. Taylor, Senior Fellow and Director, Risk, Resource, and Environment Division, Resources for the Future; and former Deputy Commissioner for Policy, Food and Drug Administration said, “this is why I characterize this particular element of the bulletin as radical, because it does put in the hands of the OIRA administrator, the decision whether there is enough evidence to act on a public health issue. And that arguably [is] not the job there.” Available at: http://www7.nationalacademies.org/stl/STL_Peer_Review_Agenda.html [Transcript page 222]

15 The Association of Medical Colleges (AAMC), representing the nation’s 126 accredited medical schools, 400 affiliated teaching hospitals and 94 medical societies, and the Federation of American Societies for Experimental Biology (FASEB) with more than 60,000 members, expressed concern about “…the proposed requirements’ likely interference with timely, responsible public health announcements to the detriment of the public weal. …[A]gencies in the Public Health Service must be acknowledged to have special prerogatives for evaluation and announcements of timely information important to the public health.

16 Jeffrey M. Drazen, MD, Editor-in-Chief of The New England Journal of Medicine, criticized the proposed bulletin because it would disqualify the most expert scientists from serving as peer reviewers. He notes, “In the United States, where the Federal Government is the single largest source of funds for cancer research, it would make no sense to disqualify scientists who had received federal funds to support their research from the peer review process.” Available at: http://www.whitehouse.gov/omb/inforeg/2003iq/83.pdf

17 Robbins A. Science for special interests. Boston Globe, December 7, 2003.

[Approved by AAAS Council on March 9, 2004.]