key questions surrounding the broader impacts criterion
• The Question of Clarification. Calls for the “clarification” of the broader impacts criterion persist, most notably in the reports of various NSF Committees of Visitors. Calls for clarification are a natural reaction to confusion. Note, however, that the term “broader impact” is not in and of itself grammatically or conceptually less clear than “intellectual merit” – at first blush, it makes as much sense to ask what one means by “intellectual merit” as to ask what one means by “broader impact.”
If it is not simply a matter of not understanding the words “broader impact,” what, then, are we to make of the scientific community’s reaction to the broader impacts criterion? Is the broader impacts criterion in need of clarification? If so, how could such clarification be accomplished? Are NSF’s suggestions about the sorts of activities that would satisfy the broader impacts criterion sufficient to clarify things? If the broader impacts criterion is not in need of clarification, then why do calls for clarification persist?
• The Question of Interpretation. Confusion surrounding the interpretation of the broader impacts criterion may be rooted in the fact that, in contrast to the intellectual merit criterion, there exists no disciplinary framework to provide shared standards of evaluation for the “broader impacts” of proposals. Whereas geo-scientists focused on deep earth processes share some disciplinary standards regarding what constitutes good deep earth geo-science that allow them to assess the “intellectual merit” of proposals, their disciplinary standards are essentially useless in deciding whether, say, a proposal that includes the participation of 3 members of under-represented groups has “broader impacts” than another that proposes the integration of research and teaching across disciplinary lines. What role does disciplinarity (or interdisciplinarity or non-disciplinarity) play in interpretations of the broader impacts criterion? Is there a way to provide for the broader impacts criterion the kind of shared framework disciplinary standards provide for the intellectual merit criterion? If so, how?
• The Question of Application. Confusion surrounding the application of the broader impacts criterion, i.e., confusion as to how important the broader impacts criterion should be in determining funding decisions, may be rooted in underlying assumptions about the best or proper relationship between techno-science and society. Such underlying assumptions may or may not be tied to one’s disciplinary standards, may or may not be explicitly held, and may or may not be widely shared with the rest of the scientific community. Moreover, policy makers and other stakeholders may have very different background beliefs and underlying assumptions about the proper relationship between science and society. Can bringing such background beliefs and underlying assumptions to light help answer the question of how important the broader impacts criterion should be in determining funding decisions? If so, how?
• The Question of Contextualization. . Questions concerning the clarification, interpretation, and application of the broader impacts criterion have not arisen in a vacuum. NSF is not alone in trying to raise the profile of the societal benefit of funding techno-scientific research: the Government Performance Results Act applies to all Federal agencies, and the idea of developing ways to assess the social impact of research is a hot topic both in the U.S. and abroad. Moreover, recent moves by U.S. policymakers have continued to reinforce the notion that funding decisions will be tied to demonstrable results.
For instance, the 21st Century Nanotechnology Research and Development Act, which put into law the activities of the National Nanotechnology Initiative, now funded at over $1 billion per year, mandates that research on the societal implications of nanotechnology be “integrated” with scientific and technical nanotechnology research (Fisher, 2005). Also, the American Competitiveness Initiative, which was announced in President Bush’s January 31st State of the Union Address, and which would double the Federal commitment of funds to basic research over the next ten years, is routinely tied to the “responsible management” of those funds through the process of peer and merit review (see, e.g., comments by Presidential Science Advisor John Marburger during a Press Briefing by Administration Officials on the American Competitiveness Initiative, February 1, 2006).
Does placing so much responsibility on scientists and engineers to assess the societal impacts of their research inhibit their freedom? Can placing the broader impacts criterion into a larger policy context help resolve confusion surrounding its interpretation and application? Can lessons learned from examining issues surrounding the broader impacts criterion be applied in other contexts? Can lessons learned from attempts in other contexts to assess the societal impact of techno-scientific research help resolve some of the issues surrounding the broader impacts criterion?