The amount of management research is mind-boggling but I often wonder whether it has made much difference to the way we run organisations. Looking at the top-line figures for the UK, there isn’t much evidence of improvement at all. Our productivity has been behind that of our competitors for years and fell off a cliff after the recession.
The advocates of Evidence Based Management would argue that this is because we haven’t been paying enough attention to management research. Much of what we do in organisations has a spurious evidence base or none at all.
I wonder, though, whether Evidence Based Management, or EBMgt, which seems to be the approved acronym, is simply offering yet another holy grail. Or, to put it another way, can management ever really be evidence-based?
Certainly not in the scientific sense. Scientific research is governed by a rigorous set of requirements; for example, findings must be testable and experiments repeatable. This is almost impossible in management because the conditions in a workplace are constantly changing, so repeating a study even with the same people on a different day might yield different results because other organisational factors could be contaminating the environment. It is therefore almost impossible to repeat any experiment in organisational behaviour in the same way that an experiment could be repeated in a laboratory. As time moves on, so do people. They might be the same people but they will have been influenced by any number of events between one experiment and the next.
Most management research is based on social sciences, primarily psychology and economics with, to a lesser degree, sociology and anthropology. Professor Gary Gutting, of Notre Dame University, remarked:
Social sciences may be surrounded by the “paraphernalia” of the natural sciences, such as technical terminology, mathematical equations, empirical data and even carefully designed experiments.
While the physical sciences produce many detailed and precise predictions, the social sciences do not. The reason is that such predictions almost always require randomized controlled experiments, which are seldom possible when people are involved. For one thing, we are too complex: our behavior depends on an enormous number of tightly interconnected variables that are extraordinarily difficult to distinguish and study separately. Also, moral considerations forbid manipulating humans the way we do inanimate objects. As a result, most social science research falls far short of the natural sciences’ standard of controlled experiments.
We like to dress management up in the paraphernalia of the natural sciences. Lots of graphs, calculations and equations.
I’ve sat in assessment centres where people have gone to great lengths to add up all the carefully weighted scores from the various exercises only to discover that the numbers give them the ‘wrong’ answer. The candidate that everyone likes is not the winner. Now clearly, if the preferred candidate is way adrift of the winner you need to ask some questions about your collective judgement but if there are only a couple of points in it does it really matter?
What people often forget is that they have taken subjective judgements, in this case scores for exercises, and turned them into numbers. What is more, even the assessment instruments they have used, while being supported by considerable bodies of research, suffer from the limitations of social science described above. They too contain an element of subjectivity. It makes no sense, therefore, to use such data as if it were quantitative. It starts off being subjective and, even if we put lots of numbers and formulae around it, it is still subjective. This tendency among social scientists to mathematicize their findings is often termed Physics Envy!
The other problem with management research is that we rarely, if ever, study failure. As Stanford’s Jerker Denrell says, to understand success, we have to understand failure too. To be truly scientific, you would have to study the characteristics of people who fail as well as those who succeed. For example, your selection assessment centre scores might correlate strongly with people who do well in your organisation but you’ll never know whether those who did badly might have done equally well. Says Denrell
No managers should accept a theory about business unless they can be confident that the theory’s advocates are working off an unbiased data set.
The trouble is, organisations are massive biased datasets. We recruit and promote the people who do well in our assessment processes and assume that they do well because we have selected for the right things. However, if we were being truly scientific we would also take on those who didn’t do well, so that we could test our processes and our definitions of good performance.
Selection bias is, to an extent, inbuilt in most management research because to study managers and their behaviour is to study group of people already selected by previous definitions of merit. Focusing only on the most successful restricts the sample still further. Proper science demands that we set up control groups and test our theories to breaking point by trying to disprove them. That’s never going to happen in a corporate setting. It’s too much of a risk.
None of this is to say that there is no evidence that anything managers do works. The World Management Survey shows that good practices like setting clear goals and monitoring performance correlate strongly with organisational performance over time and across countries. This may be basic stuff but much more than this is quite difficult to prove with anything like the sort of certainty required in science.
As economist Chris Dillow observed recently,
Very many things in the social sciences are true but not very much so.
Trying to find strong evidence for much of what we call management is always going to be a challenge. That’s not to say we should ignore research, it’s just that it won’t give us the level of proof that Evidence Based Management advocates seem to crave. Even though research into management is becoming ever more sophisticated it will never be exact. Management will, as it always has, require a combination of clever analysis, good relationships, sound judgement and a lot of good luck. In other words, it’s much an art as a science.