Great reminders of the limitations of modelling by John Kay today (30.11.11) , especially in economic context
“The models share a common approach. They pose the question: “How would we make our decision if we had complete knowledge of the world?” With such information you might make a detailed assessment drawing together many different pieces of relevant information on matters such as costs, benefits, and consequences.
But little of this knowledge exists. So you make the missing data up. You assume the future will be like the past, or you extrapolate a trend. Whatever you do, no cell on the spreadsheet may be left unfilled. If necessary, you put a finger in the air.”
“We do great damage by claiming to know things that are not known, by asserting certainty in the face of uncertainty and ambiguity, and by attaching a veneer of rationality to decisions that have in fact been made on other, rarely articulated, grounds. The paradoxical result is all too obvious. The public sector and large bureaucratic organisations appear as paragons of good decision making process and exemplars of bad decisions.”
For me, as I’ve said before, it’s the process of enquiry (putting the model together if you will) that gives us the most insight, particularly when you are unlikely to be able to test it under closed conditions (in the real world). The output to a model is also often somewhat meaningless without the context under which it was derived, as well as assumptions and limitations.