Interrogating Ethnography – and Coming Up with the Wrong Answers?

Sociological research reports rarely attract high levels of public controversy. Alice Goffman’s 2014 book, On the Run: Fugitive Life in an American City, is an exception. It is based on her PhD, written out of six years participant observation in a black neighborhood of Philadelphia. The book has been widely acclaimed for its account of the survival strategies of young black men facing aggressive policing and other forms of institutional racism. It has, however, also has been challenged over its method, methodology and ethics.

One of the most persistent and hostile critics has been Steven Lubet, a legal academic and former criminal defence lawyer in Chicago. Lubet has now extended his attack into a general critique of US urban ethnography which, in his view, shares many of the same flaws as Goffman’s work. He reaches this conclusion by applying the standards of evidence and ethics that he teaches law students to a set of classic studies, from WEB DuBois’s 1899 The Philadelphia Negro to Matthew Desmond’s 2016 Evicted. To a greater or lesser degree most of these are found wanting.

In many ways, Lubet’s work is a form of legal imperialism that takes his own forms of knowledge and reasoning as a gold standard, with very little reflection on their contexts and construction. Three examples will have to suffice: his understanding of truth; his lack of scepticism about his own evidence; and his view on ethics – which I will deal with in a later post.

Like many black-letter lawyers, Lubet thinks that the trial process establishes truth in a quite fundamental sense. By assembling the admissible evidence and testing it in combative cross-examination, jurors and/or judges can determine “what really happened.” It should be said that this is a trial lawyer’s perspective. Law and society scholarship, including studies from Lubet’s own law school, takes a rather different view. Trial is where you end up when you cannot negotiate a settlement or bargain a plea. Law is a social institution for delivering the legitimate closure of arguments. Lawyers try to control the uncertainties of language through contracts, for example – but sometimes other events intervene and the courts decree what language is to be taken to mean in this unforeseen situation. Men and women argue about consent in cases of criminal sexual assault – the courts decide what language or actions will count as consent. The trial process is a way to determine “truth for all practical purposes.”

Lubet’s fundamentalist position leads him to demand that all ethnography should be fact-checked as far as possible against official documents or other public sources. There are three issues here.

1. Lubet is correct in saying that many qualitative researchers should be more precise about how they actually collected their data and what it consists of. ‘Ethnography’ is too loose a term – indeed Howard Becker has recently suggested that we should stop using it for that very reason. Interviews and observation do not produce data of the same order and we should not pretend that they do. However, this does not help Lubet with his other criticisms.
2. Qualitative researchers know a great deal about the processes through which official reports and statistics or media reports are created. This necessarily leads to some scepticism about Lubet’s claim that they are inherently more valid than observation or interviewing?
3. Lubet’s fact-checking is, in effect, a version of triangulation, whose limitations are familiar to qualitative researchers, ever since Michael Bloor’s critique more than 20 years ago. What do you do when your data sources do not agree? You cannot have an asymmetric proof procedure. If the sources agree, that’s nice. If they don’t, you are faced with invalidating the best method by reference to the second-best method, which is surely unacceptable. As David Silverman has pointed out, such differences are the starting point of a new inquiry.

We can illustrate this by reference to a section of Goffman’s account that Lubet makes great play with, namely the avoidance of emergency rooms by young black men with injuries or other medical conditions for fear of screening or arrest. Lubet’s response is that this would be unlawful: he has written to the hospital counsel for all the emergency facilities in the Philadelphia area who assure him that no such behavior goes on. Well, they would say that, wouldn’t they? This is not to suggest that they are lying – Diane Vaughan’s study of ‘internal censorship’ shows how senior managers and company lawyers in a pharmacy chain were buffered from knowledge of a major corporate billing fraud committed by lower-level managers in order to meet their performance targets. Lubet is simply uncritical about his own ‘evidence.’

There may be a straightforward explanation. Goffman could be reporting a genuine belief based on a mistaken inference – and possibly some opportunist police practices. I have never been in a US emergency room – but I have observed similar settings in the UK, and checked with serving police officers. In the UK, police are a constant presence in ER. They bring in victims of crime or accidents, accompany potential witnesses who need treatment, and may be summoned to supplement hospital security in dealing with violent patients. The presence is so taken for granted that there is folklore about the alleged tendency of police officers and nurses to marry each other! I have no problem in imagining Goffman’s informants seeing a similar police presence, cordial relationships with hospital staff, and possibly an exchange of favours like collecting coffee from a machine. If you are on edge anyway, you may well assume that this extends into unlawful exchanges of information. If you are a familiar face to patrol officers, then you may be greeted in a way that makes you feel uncomfortable. Perhaps there are even arrests made, which get recorded as taking place elsewhere because no-one wants that information on file. However, we do not need to imply police malpractice to produce a credible reconciliation of Lubet’s account and Goffman’s. We simply apply the basic qualitative research skill of asking how each informant is looking at a situation and what might explain the discrepancies.

Arguably, Alice Goffman should have explored this possibility – but the social organization of hospitals was not the central topic of her PhD. It may well be her misfortune to be a scion of US sociological royalty. A first book from a more anonymous scholar might have been less hyped by the publisher. It might also have attracted more considered responses.