The bosses pretend to have theories, and we pretend to test them

Leo Tiokhin has hosted a new blog series on the use of formal models in metascience and, more generally, in psychology. The starting point for the series is the increasing recognition that psychology’s weaknesses don’t just lie in its recent replicatory embarassments. The underlying theories that all those (possibly non-replicable) experiments aim to test are also weak. That is: the theory as stated could give rise to multiple patterns in the data, and the data could be compatible with multiple theories, given how vaguely these are stated. In my contribution to Leo’s series, I invoked the old Soviet joke: the bosses pretend to have theories, and we pretend to test them.

Several contributors to the series point out the virtue, given this problem, of formalizing theories in mathematical or computational models. This undoubtedly has merit: if you convert a verbal psychological theory into a formal model, then you expose all your tacit assumptions; you are forced to make decisions where you had left these vague; you discover if your conclusions really must follow from your premises; and you are left with a much tighter statement of what your do-or-die predictions are. This is all good, and true.

However, my contribution, and also to some extent the one by Willem Frankenhuis, Karthik Panchanation and Paul Smaldino, provide a line of argumentation in the opposite direction. Theories in psychology are often weak exemplars of theories. One move is to make them stronger through formalization. The opposite move is to not claim that they are theories. I think for many areas of psychology, that makes a lot more sense. There are many important avenues of scientific enquiry that do not exactly have theories: descriptions of psychological phenomena; ontologies of psychological processes; uncovering of which things pattern together; working out which levers move which parts in which direction, and which levers move none. These enquiries can certainly feature, and meaningfully test, hypotheses, in local kind of way, but may not be underlain by anything as grandiose as a fully-identified theory.

Of course, there is always some kind of idea underlying research questions. Often in psychology this is better described as an interpretative framework, or a proto-theory. To try to press it into the mould of fully identified theory may be to subject it to the heartbreak of premature definition, which can take years to get over. The problem has been that psychologists have had to claim to be using a ‘theory’ to get their papers published (a little recognized form of publication bias). A psychology paper has needed to start with a ‘theory’ with a three-letter acronym like an opera has needed to start with an overture: terror management theory, error management theory, planned behaviour theory, reasoned action theory, social identity theory, social cognitive theory, social norms theory, regulatory focus theory, regulatory fit theory, life history theory, life course theory – I think you know the game. I even coined an acronym for the generation of these acronyms: CITEing, or Calling It Theory for Effect.

None of these frameworks is ready to be implemented as a computational model, which raises all kinds of interesting questions. What kinds of beasts are these? Would it be better if we did not need to invoke them or their ilk at all, and could just state what questions we want to answer, what parallels there are elsewhere, and what our hunches are? Although having theories in science is great, it might not be a prerequisite. Precisely stating your theory is especially excellent when you do actually have one. You should not feel pressured to state one as a rhetorical move, if in fact you are doing description or proto-theory.

People often misunderstand the pre-registration revolution as being the requirement to only do confirmatory analyses. But, as Willem and I have argued, it’s not this at all. It’s the freedom to do confirmatory analyses when these are appropriate, and exploratory ones when these are appropriate, and be clear and unashamed about which it really is: don’t muddle the one with the trappings of the other. Likewise with theories: be clear when you really have one, and be clear when you don’t. Just as having better theories can lead to a better psychology, so, possibly, invoking no theory, in some cases.

People sometimes link this point to the claim that ‘psychology is a young science’, not ready for its Newton or Darwin yet. That’s starting to look a little disreputable. I personally think there should be a one hundred year cut-off on the old ‘young science’ ploy, which means psychology has overstayed. The deeper problem for theories in psychology, as David Pietraszewski and Annie Wertz have recently argued, is not insufficient time, but a constant flip-flop about what its proper level of analysis is. Some frameworks work at the intentional level of analysis (the intuitive way we speak about people, with beliefs, desires, feelings, selves, things they can deliberately control and things they can’t), and others at the functional level of analysis (i.e. how do the information processing mechanisms actually work, which may or may not look isomorphic to intentional-level descriptions of the same processes). Add to the mix that evolutionary psychologists are sometimes also thinking at the level of ultimate causation (fitness consequences), and there is a heady recipe for total incoherence about what we are trying to do and what kind of thing would make us satisfied we had in fact done it. Hence the constant churn of what seem like adequate theories to some people, and seem entirely unlike adequate theories to other people. This is the big problem psychology needs to sort out: what level of analysis do we want in an explanation. The answer may be sometimes one, sometimes another, but ‘theories’, and authors, need clearly to say which they are trying to do.

Big thanks to Leo for hosting this interesting series.