The Social Scientist Who Knew Torture Wasn’t Worth the Game

When the U.S. Senate Intelligence Committee – or at least its Democratic half – reported that torturing prisoners didn’t provide useful information for fighting terror, the CIA responded forcefully that in fact its enhanced interrogation absolutely, positively might have been really valuable.

Talking to reporters after the blockbuster report came out earlier this month, CIA Director John Brennan said it was “unknowable” what among the data collected by the CIA had torture in its provenance, and furthermore, “Let me be clear: We have not concluded that it was the [enhanced interrogation techniques] within that program that allowed us to obtain useful information from the detainees subjected to them.”

As pundits debated with greater conviction the benefits of enhanced interrogation (“It did in fact produce actionable intelligence that was vital in the success of keeping the country safe from further attacks,” said Dick Cheney) or the negatives of torture (“[It] “produces more misleading information than actionable intelligence,” said John McCain), they drowned out one frustrated political scientist’s fervent, “I told you so.”

That was important to do because the only imaginable justification is that [torture] is effective. That may not be a sufficient condition – we may reject it for moral reasons – but if it’s not effective it can’t possibly be justified

Even as the program of enhanced interrogation conducted by Americans at their Guantanamo Bay lockup for purported terrorists was being dismantled, John Schiemann had been applying game theory to model the interaction between a detainee and an interrogator. Game theory, as it is generally defined, creates mathematical models to describe decisions made by rational actors in various scenarios. And despite the sometime unfortunate connotation of that first word, “game,” the technique has proven both flexible and powerful. Some 11 Nobel in economics have been given to game theorists, for example.

Schiemann’s own work – published in 2012 — demonstrated that, by logic alone, we can know that torture is not going to be effective means to routinely elicit useful intelligence. The information will not only be unreliable, the associate professor of political science at New Jersey’s Farleigh Dickinson University found, but that torture will be increasingly brutal and even more frequent than its proponents imagined.

That’s what he predicted.

This month’s Senate report found him correct in every particular. (Well, almost. “My model may have predicted more valuable information would result than actually resulted.”)

For example, the ideal outcome – that we’ll get lots of sterling information, no one who isn’t a baddie will get tortured, and those who do have good intelligence will give it up – didn’t occur, according to the Senate report. And of the 119 detainees who were subjected to “conditioning,” which, while not waterboarding, was much more brutal than anything even the most heinous American criminal suspect would ever (legally) face, 26 were not guilty of terrorism. They still were treated harshly in large part to maintain the threat of torture should anyone withhold –or appear to withhold — information.

Not only did torture not produce reliable information, it didn’t even produce information reliably; the Senate report found that seven of the 39 detainees subjected to harshest techniques produced zero intelligence reports after interrogations.

“The Senate report tells us what happens when you introduce torture into an interrogation room,” he summed up. “I’d like to think my work explains why what happens happens.”

Ironically, the biggest finding from the real-world test was that interrogational torture doesn’t work, likely damning any subsequent efforts to harness it as a sincere investigational tool.

He rejects “ticking time bomb” justifications in large part because enhanced interrogation itself is based on the calendar and not the wristwatch. “The ticking time bomb is a complete red herring on a number of levels,” he said. He points to the case of Abu Zubaydah, who was the first detainee the CIA itself –as opposed to its designates — used enhanced techniques on. FBI agents initially questioned a wounded Zubaydah in the hospital, and started receiving a flow of useful intelligence. Those agents and their less brutal techniques were withdrawn — over their protests – and the CIA started a regime developed by two contract psychologists. The first step? Zubaydah was placed in isolation for 47 days, with no one, friend or foe, speaking to him during that period.

“That’s just absurd if you’re thinking that the reason we have to do this is for the ticking time bomb scenario. … On [the contract psychologists’] own theory, it can take 30 days for a detainee to become ‘compliant.’

“The lie to this is that somehow the traditional rapport-based technique is slower. It’s not. And secondly that somehow the enhanced interrogations are going to be quicker. They’re not – on their own theory they’re not.”

John Schiemann

Schiemann had used game theory in some previous academic work before starting on what he called “the torture project.” He traces his academic interest in the subject to 2006, when stories about “enhanced interrogation techniques” started breaking. “Like everybody else, I was wondering about their effectiveness,” he recounted.

“I started poking around, and saw there was no solid empirical data to answer the question with,” Schiemann said. “Whatever empirical evidence exists it was all anecdotal, it was unsystematic, there was no way to test it.” His solution was to model the torture equation, basing it on the prescriptions and definitions set by enhanced interrogation’s proponents: “I follow their own brutal information calculus and logic, and I trace out the implications of that logic and see how it corresponds to what we see in the real world.”

His original article, titled “Interrogational Torture: Or How Good Guys Get Bad Information with Ugly Methods,” came out in the journal Political Research Quarterly in 2012; drafts of that article have been circulating widely since 2009. (And he’s working on a book covering the subject that he hopes is out in a year or so.)

Meanwhile, outside of the ivory tower, away from any meddling institutional review boards, there was an active natural experiment into enhanced interrogation that offered a chance to compare outcomes once its classified results became public. “The CIA program is the best empirical attempt in the real world to implement a very limited, controlled model of interrogational torture,” according to Schiemann. “I think they really did try to limit – and one could argue they shouldn’t even have done it to begin with, and I’d agree with that – that they did believe they could create a program with very limited, controlled torture that would generate lots of information.”

It didn’t, at least according to the Senate report: “The use of the CIA’s enhanced interrogation techniques was not an effective means of obtaining accurate information.”

Or as Schiemann‘s journal article concludes, “The debate over this question suggests that this reality needs probing, and the probing offered here suggests that torture games have no winners.”

Criticisms

Several criticisms from both side of the debate arise from Schiemann’s exercise. For example, how can a chalkboard exercise match the real-world laboratory the interrogators inhabited.

“It may appear tempting to policy makers favoring torture to dismiss the analytical model as too abstract and too removed from “the real world” to be of any use or value,” Schiemann writes in the paper. “But of course these policy makers are relying, if only implicitly, on their own model of interrogational torture.”

“There’s this idea that the fact that I put in a bunch of letters and symbols and do some math – and not very complicated math, I would say – that that’s different from what [the CIA] do[es],” Schiemann explained in an interview. But his model is their model, albeit with a few underlying assumptions thrown out:

The Dick Cheney interrogation model is a complete and perfect information game with one type of detainee (knowledgeable and weak) and one type of interrogator (pragmatic). The detainee knows for certain that the interrogator will not torture after full information is provided (q = 1), and the interrogator knows for certain that the detainee has information (i.e., there is no chance he is innocent; p = 0). The interrogator also knows exactly how much information the detainee possesses (θ = 1) and recognizes information as valuable with just as much certainty (ω = 1). Such a model is much further removed from ‘the real world’ than the model elaborated here.

“Because it’s so vivid and visceral, and the game theory model is by definition very abstract and logical, it’s difficult for that to have an effect. It’s easier for people to pick it up and say it has nothing to do with the real world, it’s just a bunch of math, and so dismiss it. … All the formal abstraction really does is make very clear how you’re arguing.”

But Schiemann knows that all the omegas (ω) and thetas (θ) in the world won’t win any arguments with policymakers. “The way to do this isn’t through giving policy makers 30-pages with an appendix on game theory,” he admitted, “but instead by giving them vivid narratives on how exactly that plays out.”

Meanwhile, some question whether it’s acceptable even to consider the effectiveness of an immoral act.

In a rebuttal to Schiemann’s journal articlealso published in Political Research Quarterly, Dustin Ellis Howes at Louisiana State University suggested that creating a venue for asking these questions adds legitimacy to what should be an illegitimate undertaking. “Propagating the falsehood that torture is primarily a game,” Howes wrote, “obscures the fundamental character of torture and invites policy makers and social scientists to use formal models to improve or perfect torture.”

Schiemann said he respects and even sympathizes with the strong moral arguments that arise from examining torture’s effectiveness. But … “My goal is to address the four in 10 Americans, based on a survey we did here at Fairleigh Dickinson, who say, ‘Well, I don’t like torture, and if it’s ineffective, we can’t use it. But if it’s effective, we have to use it sometimes.’ One can disagree with that, but lots of people hold those views, and not just Dick Cheney.”

“That was important to do because the only imaginable justification is that [torture] is effective. That may not be a sufficient condition – we may reject it for moral reasons – but if it’s not effective it can’t possibly be justified.” Furthermore, game theory itself stood out as an appropriate technique since it was advanced by the philosophical, legal and operational proponents of torture themselves. “If those outcomes don’t match up to what the proponents then say, that’s a pretty devastating critique of their own model.”

Again, since Schiemann’s working from an actual public policy and the models it produced, he sees something that actually requires being questioned. “It seems to me that that’s not fundamentally flawed, but instead gets at something that’s fundamental.”

Still, there’s no joy in Teaneck now that his findings have been confirmed.

No one in authority has listened to Schiemann on this topic, then or now, even as he pushed out his torture-won’t-work research in blogs and editorials as well as academic venues. “Nobody’s interested in academics making predictions ahead of time, even though we get criticized for not doing it,” he said resignedly.

“As a human being and as an American, I’m disappointed at just how close the predictions of the model come. There was a part of me hoping that it wouldn’t be that bad, and I think all of us have seen that’s not the case. I take no joy in learning that someone was waterboarded to the point of being completely unresponsive and needed to be revived medically.

“I will say it’s frustrating to hear continued defenses of the program.”