Devin FinnBy Devin Finn for Denver Dialogues
In a recent seminar at the Josef Korbel School of International Studies, Sam Popkin quoted Jack Kemp: “Nobody cares what you know until they know that you care.” Politicians’ demands that academic studies have something “to do with science” overlook the reality that scholars are addressing social and political problems and crises about which they are passionate. Among researchers, the idea that compelling, policy-relevant work demands that we engage more thrives. We must achieve relevance to gain the legitimacy necessary for state and private research funding—and to warrant the engagement of the public. The very question of how to have impact on policy and politics involves personal judgments about values and ideas. Researchers’ roles, then, are by nature, political. Celestino Perez, for example, argued that an ethical imperative mandates that United States military officers wielding violent means possess a responsibility to prepare intellectually for the task. Still, for researchers of international and comparative politics, a sense of humility may be central to communicating our findings with one another and with policymakers.
Lately, missing from the conversation about fruitful connections between academic research and policy relevance are the implications that these two-way ties have for the hierarchy of knowledge construction in the social sciences. Scholars have discussed the tension latent in the narrative about bridging the gap between the objective rigor considered intrinsic to scientific investigation and the production of research of public import. But there is little consensus on what “science” means, and we do not always distinguish between ontology, methodology, and method. As Rogers Smith argues, political science now enjoys “intellectual pluralism, though as much by default as by conviction.”
Over the last several years, a group of senior researchers worked to establish Data Access and Research Transparency (DA-RT), a set of guidelines for archiving data and producing research that “improves the value of social science.” Their aim is to engender greater transparency and openness in research, an idea which became a set of principles adopted by the APSA Council and included as part of its Code of Professional Ethics in 2012.
The DA-RT leaders have summarized their efforts in on online statement, and Jeffrey Isaac synthesized the events that led to the recent adoption of the standards by over twenty political science journal editors. APSA leadership issued a statement endorsing DA-RT in November 2015.
DA-RT proponents argue that authors making evidence-based knowledge claims must also provide comprehensive information about how the claims were produced, until now a shortcoming which has hampered readers’ ability to evaluate them. From a report on a DA-RT workshop held in March 2015:
Because transparent research is more credible and more legitimate, new opportunities for transparency promise substantial benefits for the advancement of knowledge. In addition, shared data provide public goods that pay extraordinary dividends for entire research communities and society at large.
This approach makes certain assumptions: that to make the discipline more scientific is a shared objective, and that transparency is the means to a more scientific end. Insistence on and adherence to standards of transparency and replicability do not make our interpretations of evidence value-neutral. By making uniform decisions about when, with whom, and exactly how our data are shared, proponents seem to want to deny that we make decisions in analyzing data that are political and value-driven.
The original intention of the DA-RT movement may have been to eliminate the use of unsystematic, sloppy research employed to advance normative claims about policy and strategy. But the initiative has polarized qualitative scholars, some of whom have used their disagreement with DA-RT to stimulate dialogue, including through a web site and a special edition of the newsletter of the APSA Organized Section for Qualitative and Multi-Method Research.
At stake are a few pivotal issues and potential problems:
Some negative effects of DA-RT are pragmatic and professional. The efforts required to meet the standards, including transcription, translation, and data management, are considerably taxing. For researchers without access to the resources available at elite institutions, these activities are costly in time and money, particularly for those who spend considerable proportions of their days on teaching activities. Junior scholars, who may be spending significant amounts of time in difficult-to-access, non-digitized archives, are being asked to bear disproportionately the costs of the policies.
Ethical and legal challenges constitute serious obstacles. Some researchers, including Jennifer Piscopo, cite “networks of trust” built over years of research through non-elite and elite interviewees—that must be protected through anonymization, confidentiality, and cautious control over interview material and field notes. For interpretivists, analytical processes linking evidence to findings are irreducible to an expanded footnote, a digital image, or a series of linked citations.
Further, in some violent and non-democratic political contexts, revealing sources (and the ways in which one contacts and communicates with sources) as a research practice is dangerous, unethical, and may put one’s interlocutors at risk. There may, in fact, be legal consequences in handling particularly sensitive testimony and information, as the battle over access to oral history records at Boston College for possible criminal prosecution in the U.K. shows.
DA-RT leaders say that through the standards, they will rely on and respect the provisions of researchers’ IRB-approved protocols with regard to “human subjects” research. But researchers in different disciplines have raised many questions about whether IRB oversight may be applied effectively to non-experimental research.
Beyond IRB concerns, Jeffrey Isaac urges “serious attention to the ethical relationship between our inquiry and the subjects of our inquiry”:
It is the old question of the value-basis of all political science. How do our research problems get defined, and what is the ethical and political significance of our choices? From where does our funding come? What values animate our work, individually as scholars and collectively as a discipline? What values should animate our work? And in situations where values conflict, what kinds and forms of judgment and institutionalized practice best help us to navigate them in a responsible way?
DA-RT’s singular view of science may not engender conversation among researchers. By invoking the irrefutable values of rigor and transparency, DA-RT proponents articulate a vision of science that in fact elevates and enforces a particular epistemology. For some researchers, DA-RT calls to mind King, Keohane, and Verba’s (1994) discipline-shaping push for the application of causal inference techniques to qualitative research. The standardization of research practices is intended to be a “multi-epistemic” effort that will help editors achieve “credibility and legitimacy aspirations.” But will DA-RT help researchers working on very distinct logics of inquiry to be in productive dialogue?
The commitment to DA-RT is an epistemological statement: if fully implemented, it affects how the social world is conceived and how evidence about it is collected and analyzed. Decisions that affect the lens through which researchers apprehend the stuff of reality explicitly involve power and political ideas.
Some journals have begun implementing the DA-RT conventions for submission and publication. The potential changes demand that researchers be attentive to the hierarchy of knowledge production that DA-RT may institutionalize. As guidelines for research practice, the standards affect the questions that are asked and the evidence that we marshal to answer them. In turn, they shape how researchers respond to the imperative to contribute to public policy design and implementation. Perhaps the conversation that DA-RT has sparked will catalyze an open discussion of the practices of politics that our participation in building knowledge about the social world entails.
Devin Finn is a post-doctoral research fellow at the Sié Chéou-Kang Center for International Security and Diplomacy at the University of Denver.
