Replication in Humanities Just as Desirable as in Sciences
Replication is not only possible and desirable in the sciences, but also in the humanities. At least, so we have argued in a recent comment in Nature and in a more detailed follow-up in Palgrave Communications. We know of only one other scholar, Katherine Rowe, to have made this claim. Failed replication attempts have received widespread attention in the biomedical and social sciences. The results are at times so troublesome that it is not uncommon to speak of a “replication crisis.”
This article by Rik Peels and Lex M. Bouter originally appeared on the LSE Impact of Social Sciences blog as “Replication is both possible and desirable in the humanities, just as it is in the sciences” and is reposted under the Creative Commons license (CC BY 3.0).
That we should also carry out replication studies in the humanities follows from the conjunction of two relatively simple facts. The first is the empirical nature of many disciplines within the humanities: humanistic scholarship often consists of gathering data with a view to answer one or more study questions, analyzing those data by way of a particular method, and drawing one or various conclusions in response to these study questions. The second is the nature of replication itself: to carry out a replication study is basically to do an independent repetition of an earlier study, answering the same study question by using the same or similar methods under the same or similar circumstances. It, thus, simply follows from what it is to replicate a study and from what the humanities do, that replication is possible in the humanities. To see that it is not only possible, but also desirable, is relatively straightforward as well, given the nature of replication: the original results become more trustworthy if an independent replication has been carried out and, therefore, more likely to be true.
This is not to deny that different kinds of replication can be carried out and that one kind of replication may be more suitable than another in certain instances. One can reanalyze existing datasets, one can collect new data with the same study protocol (a direct replication), or one can collect new data with a modified study protocol (a conceptual replication). This is also not to deny that further arguments can be made for the possibility and desirability of replication in the humanities. Elsewhere, we have given some examples of studies that are replicable or that have even been replicated. Historical research that employs a hermeneutical method has confirmed that Augustine was influenced by Gnosticism but also parted ways with Gnosticism at some point in his life. Deciphering Egyptian hieroglyphic was made easier by comparing the Demotic, hieroglyphic, and ancient Greek texts on the Rosetta stone found in 1799. And various methods (chemical analysis, historical scholarship, art theory) from art scholarship have confirmed that Sunset at Montmajour is a true Van Gogh.
In the social sciences and the biomedical sciences, not all scholars embrace the importance of replication. These critical views are also relevant for replication in the humanities. The typical concerns raised are summarised and refuted elegantly by Rolf Zwaan et al: the context is too variable, the theoretical value of replication is limited, direct replications are not feasible in certain domains, replications are a distraction, replications affect reputations, and there is no standard method to evaluate replication results. In the defense of replication some even argue that replication has more value than original discovery. The reason being that claimed original discoveries are often wrong and cannot be trusted before they have been replicated.
Image credit: 139904, via Pixabay (licensed under a CC0 1.0 license).
In the humanities concerns against replication have been voiced as well. In a rejoinder published in Nature, Sarah de Rijcke and Bart Penders argue that “quality criteria are crucially different in the humanities and the sciences”. As a general claim, that seems rather implausible. After all, any academic field seeks similar quality criteria, for example, methodological clarity, sound reasoning, clear description of datasets, rigorous definitions, and so on. What about the desideratum of replication, though? De Rijcke and Penders provide two reasons for thinking that replication is not to be pursued in the humanities.
First, the humanities seek meaning beyond truth. Thus, they seek not only to confirm that Sunset at Montmajour was indeed painted by Van Gogh (a truth), but also what its cultural meaning is and, according to them, there may be multiple valid answers to that. Even if the latter is true (there are worries here though: why would multiple valid answers not count as multiple truths?), that doesn’t disqualify the first point: establishing whether Sunset at Montmajour was indeed painted by Van Gogh was the purpose of an important study in the humanities and that study can be replicated. Just for the sake of clarity: we are not saying that every study in the humanities should be replicable. We only commit ourselves to the claim that replicability is possible and desirable for many studies in the humanities.
Second, De Rijcke and Penders stress that among the objects of the humanities are people and that people are interactive entities. This, they say, requires continued interaction with them, since humans might resist certain qualifications or disagree with preliminary results. That is, of course, true, but the same holds for the empirical social and biomedical sciences: their objects are usually also human beings and studying them often requires interaction with them. If replication is possible in the empirical sciences involving interaction with human beings, then so it is in the humanities. It is simply wrong to say that replication is a mark of quality only for “indifferent kinds” – phenomena that don’t interact with us in the sense mentioned above – such as atoms, stars, and economic trends. The fact that human beings interact and sometimes respond to research might make replication more complicated and more challenging, but there is no reason to think that that somehow renders it impossible.
Similar to the first concern of De Rijcke and Penders scholars in the humanities may argue that they often seek wisdom instead of knowledge. We disagree and are inclined to think that usually they seek objective knowledge and understanding. We agree that there is often some controversy about how to balance various features of different theories and that that may require a certain kind of wisdom (even though it may not be prudential in the ordinary sense of the word). However, that will be true for the sciences as well: one theory that explains some phenomenon or bit of evidence may do better on, say, explanatory scope and simplicity, whereas another does better on predictive power and coherence with background knowledge. Scholars both in the sciences and the humanities will have to balance these virtues of theories and attach a particular weight to each of them in comparison with the others.
We conclude that our point stands unscathed: it is a desirable feature for empirical studies in the humanities that they are replicable and it is equally desirable that the project of actually carrying out replication studies in the humanities gets off the ground.
