Why the Method Matters

By Thomas Zeitzoff. 

A recent article in the Chronicle discusses anthropologist Scott Atran’s work in Iraq and Syria. The article provides a fascinating portrait of researchers seeking to understand the motivations for why members of ISIS or the Kurdish Peshmerga are willing to commit violence or sacrifice themselves for a cause or a group. Atran’s central thesis comes from recent interviews in Syria and Iraq, along with decades of interviews and interactions with religious extremists from around the world. He argues that individuals who engage in terrorism are driven by two things: 1) sacred values—moral imperatives such as religion or family, that drive human behavior regardless of the cost, and 2) unyielding devotion to a core group, or band of brothers.

A large part of the article describes Atran and his team’s quest to interview, face-to-face, members of the Peshmerga and ISIS, including one ISIS member in a Peshmerga prison. Atran’s drive to interview ISIS members stems from his belief summed up by this quote in the article from Andrew Silke, Director of Terrorism Studies at the University of East London, “ ‘An awful lot of research on terrorism is done by people who have never met a terrorist,’ Silke says. ‘There is good work you can do at long distance, but the insight and understanding you develop from speaking with them is much more impressive.’”

Atran and his colleagues place a premium on “on-the-ground” knowledge gained through these interviews. They provide rich data and nuances that give crucial insights into the motivations of individuals engaged in political violence (others have also made similar arguments). Yet, it’s important to acknowledge their shortcomings. Below, I elucidate three ways we must be careful in weighing knowledge gained from these interviews.

1) The public vs. private face

As Timur Kuran observes, many times our privately held views are very different than our public opinions. Systematic biases are likely to creep in when people are asked sensitive questions face-to-face. Humans are social creatures, and a desire to be seen in a positive light keeps us from sharing “socially undesirable” attitudes. In survey research and polling, the answers to directly questioning attitudes on racism, willingness to vote for women, support for abortion, and willingness to discriminate against rival ethnic groups are all underreported compared to indirect questions, or computer surveys. Furthermore, interviewers of a different gender or from different ethnic groups elicit less truthful and more desirable responses than those done by interviews of the similar group. Setting all of these biases aside, more than sixty years of psychological research on cognitive dissonance has shown that motivations for human behavior are complex, and may run in the opposite direction. Sometimes stated motivations for actions are accurate, and sometimes they are ex-post justifications. In other words, behavior may explain the stated motivation rather than the reverse.

Why does this matter? Face-to-face interviews with individuals engaged (or formerly engaged) in political violence are likely to heighten all these biases. Thus, information gleaned in face-to-face interviews should be scrutinized in light of three things: 1) Who is conducting the interview and are they likely perceived as a member of the ingroup, neutral, or an enemy or outgroup member? 2) What are the incentives of the interviewee to lie or misrepresent—e.g., is he/she a person talking under their own free volition, or are there external pressures being placed upon him/her? And 3) is there a chance that the stated motivations are simply ex-post justifications?

2) The systematic problem

Survey research is built upon the idea of using a representative selection of people to gain insights. Yet getting access to people involved or formerly involved in political violence is notoriously difficult. By definition many of these people are engaged (or were formerly engaged) in covert actions, and do not readily volunteer for research. So researchers are left to find people through referral via contacts, or fixers (a so-called “snowball” sample), interviewing prisoners (ethical issues discussed below), or other non-random techniques. Past research has shown, information from biased samples can result in wildly inaccurate portraits of conflict. Any insights gleaned from interviews must be weighed in light of the population they come from. Are these people really representative of participants in political violence (see excellent new research from Grant M. Gordon’s on military control and civilian abuse in the Democratic Republic of Congo (DRC) for a great example of this kind of research)? Or is there something special/different about people willing to be interviewed, prisoners, or individuals in unit “X”?

3) The ethics of on-the-ground

Research in active conflict zones involves considerable risk to researchers and participants. As Christopher Blattman convincingly argues in an excellent series of pieces on doing research in war zones, “does your conversation (research) pose any risk to the informant, by arousing past traumas, increasing stigma, or putting them in danger? Is it even likely to make them uncomfortable? If the answer to any of these questions is ‘yes’, you need to think twice about the questioning (research).” Prisoners may be a valuable source of information, but extra safeguards are needed for working with this vulnerable population, who may not be able to voluntarily consent to the research. This is doubly true with prisoners captured on the battlefield, and in active conflict zones.

The ethics of research also depends on its quality and potential impact. Risky research is permissible and even encouraged if it promises a large potential benefit — especially to participants. Something that promises to unlock the code of why people commit horrible acts in the name of religion, or prevent it from happening in the first place, has a bit more latitude with respect to risk than the nth study on topic x. Researchers have an ethical obligation, particularly when conducting risky research in conflict zones, to have it be of the highest quality. They must make sure the knowledge gained is solid, and worth the risk to participants.

The method used to draw conclusions about political violence matters. On-the-ground research by researchers like Atran provides important insights into the motivations of those who commit political violence and about decision-making in conflict. Yet this micro-level research is not a panacea. It’s important to recognize the potential biases and ethics associated with it. Furthermore, individuals operate within a broader political context. Focusing on motivations for joining ISIS is important, but ignoring the macro political instability in Iraq and Syria is a mistake. The context in which individuals are making decisions to join groups such as ISIS is an equally important part of the puzzle. We should not narrowly focus or privilege details gained from on-the-ground research in place of broader trends or factors about the Syrian Civil War (such as the number of foreign powers intervening, or number of rebel factions). Marrying on-the-ground findings with broad-based, macro research provides a more complete picture of political violence, allowing researchers and policymakers to better understand and, hopefully, prevent it.