UK: ORDER FROM CHAOS - CHAOS THEORY.

UK: ORDER FROM CHAOS - CHAOS THEORY. - Chaos theory suggests that management should place more emphasis on adaptability, initiative and entrepreneurial creativity to cope with a future that is inherently unknowable.

by Anita van de Vliet.
Last Updated: 31 Aug 2010

Chaos theory suggests that management should place more emphasis on adaptability, initiative and entrepreneurial creativity to cope with a future that is inherently unknowable.

'We're better at predicting events at the edge of the galaxy or inside the nucleus of an atom than whether it'll rain on auntie's garden party three Sundays from now ... We can't even predict the next drip from a dripping tap when it gets irregular. Each drip sets up the conditions for the next, the smallest variation blows prediction apart, and the weather is unpredictable the same way, will always be unpredictable. When you push the numbers through the computer you can see it on the screen. The future is disorder.' So Tom Stoppard, in his latest play, Arcadia, deftly unfolds one of the key principles of chaos theory. Acclaimed by its followers as the major intellectual revolution of recent times, 'chaos theory' (also known rather less catchily as 'complexity science' or 'non-linear dynamics') uses mathematical techniques boosted by computer power to explore aspects of nature which have hitherto proved resistant to analysis - in Stoppard's words, 'the ordinary-sized stuff which is our lives, the things people write poetry about - clouds, daffodils, waterfalls - and what happens in a cup of coffee when the cream goes in'. Its sphere is wide - from the behaviour of measles epidemics to earthquakes, from the rhythms of the brain to the evolution of economic systems. Its approach is not only multi-disciplinary but cross-disciplinary: biologists might draw on insights from engineering and physics, for example, while the study of stock-market prices might be related to weather records and other such apparently random phenomena. Economists have long been attracted by the new theories, which they in turn have helped to develop. Probably the best known is Brian Arthur, professor of population studies and economics at Stanford University and one of the founding fathers, intellectually speaking, of the Santa Fe Institute in New Mexico, which was specifically set up in the mid-1980s for the study of complex systems (the story of the Institute - its people and their ideas - is well told in Mitchell Waldrop's Complexity). More recently, aspects of chaos and complexity have also inspired a small but growing number of management and organisational development theorists. Foremost among these are Ralph Stacey, professor of management at the Business School of Hertfordshire University (co-author with David Parker of the recent Hobart Paper Chaos, Management and Economics, and author of a number of works on strategy and organisational dynamics in relation to chaos and complexity theory, including Managing Chaos, published in the US as Managing the Unknowable), and Jeffrey Goldstein (author of The Unshackled Organization) in the US. There is, as yet, no homogeneous body of writing which could be labelled 'chaos economic and management theory'. There is, however, a pervasive spirit. This puts the emphasis on adaptability, intuition, paradox and entrepreneurial creativity in the face of an unpredictable, indeed inherently unknowable, future. There is a shared belief that conventional economic theory, with its assumptions of equilibrium, diminishing returns and a single optimal outcome making the most efficient use of resources, is an 'agreeable picture', as Arthur puts it, but one that 'often does violence to reality'. There is an acknowledgment of the complexity of the world of human and economic affairs, and a rejection of the concept of 'Economic Man' as no more than a fiction. There is a sense of the flux of industrial history, as of all else. 'Economy is constantly on the edge of time,' wrote Arthur in his notes on 'Economics Old and New' in 1979. 'It rushes forward, structures constantly coalescing, decaying, changing.' There is also a belief, drawn from chaos and complexity theories in the natural sciences, in the creative role of disorder and irregularity.

At this point, a brief reminder of the basic tenets of chaos and complexity theories might be in order. As Stacey and Parker remark, the use of the shorthand term 'chaos' to cover the whole science of complexity or non-linear dynamics is unfortunate: in scientific terms, 'chaos' refers not to the word's popular meaning of utter muddle and confusion, but to the behaviour of a system - like the weather, for example - which is governed by simple physical laws but is so unpredictable as to appear random.

This unpredictability arises because of the system's extreme sensitivity to initial conditions: tiny variations are amplified with ultimately huge consequences. Whereas in a linear relationship, a given cause has one and only one effect, in non-linear relationships, a single action can have a host of different effects; and the interactions become so complex that the links between 'cause' and 'effect' disappear. In the example now known by every schoolchild (and probably taken more literally than its originator, Edward Lorenz, intended), a butterfly taking flight in the Amazonian jungle may trigger a hurricane in New York, but nobody will ever know for sure that it has.

Traditionally, natural and social scientists have assumed that it was acceptable to use linear approximations, modified by 'error' terms, to describe non-linear relationships, given that non-linear relationships are notoriously difficult to handle. Now, according to the chaos theorists, it is recognised that even the tiniest error or 'noise' in the system can balloon into huge consequences; and it has to be accepted that the future of these non-linear feedback systems is inherently unpredictable.

Despite the unpredictability of chaotic systems, there are recognisable patterns or categories of behaviour; and within these there is endless individual variety. Watch the clouds for a while, Stacey and Parker suggest, and you understand what scientists mean by chaotic behaviour. Watch the business environment, suggest other management chaos theorists, and seek out the pattern of potential.

Certainly, these patterns of behaviour are not confined to natural forms: the French mathematician Benoit Mandelbrot discovered, for example, when he fed cotton-price data covering 60 years into the computers that although each particular price change was random and unpredictable, the curves for daily and monthly price changes matched perfectly (see James Gleick, Chaos).

Students of complexity go a step further - beyond tracing the intricate patterns of, say, a snowflake, to trying to understand the spontaneous self-organisation and coherence of complex systems, such as life itself (in the case of the biologist Stuart Kauffman) or, more modestly, an ecosystem, a corporation or a marketplace. Whether snowflake or stock market, these non-linear systems are governed by the process of 'positive feedback', a term taken from engineering and neatly embodied in Stoppard's description of the dripping tap. Obviously, much of human behaviour is governed by this process: the actions we take (even those we don't), and our experiences, inform our understanding and so feed in to the choices we make. In a seminal article first published in Scientific American and reprinted in the first issue of The McKinsey Quarterly this year as 'Positive feedbacks in the economy', Arthur applies the concept to the dynamics of modern technology-intensive industries.

Conventional economic theory, he writes, is built on the assumption of diminishing returns or negative feedback (analogous to the thermostat on a heater drawing it back to the optimal temperature): 'Economic actions engender a negative feedback that leads to a predictable equilibrium for prices and market shares.' Thus the high oil prices of the 1970s encouraged energy conservation and increased oil exploration, and precipitated a predictable drop in prices by the early 1980s. Positive feedbacks, or increasing returns, in the economy, on the other hand, magnify the effects of small economic shifts; and instead of one single equilibrium point for the economy, the law of positive feedback makes for many possible equilibrium points. There is, moreover, no guarantee that the economic outcome that emerges will be the 'best' one.

This law of increasing returns - the idea that to them that hath shall be given - is well demonstrated by the history of the videocassette recorder. The VHS and Betamax formats were introduced at about the same time, and so began with roughly equal market shares. A combination of luck and corporate manoeuvring brought the VHS vendors a slightly bigger share early on - which meant that more video stores stocked VHS pre-recorded tapes, which meant that more customers chose VHS recorders, which meant that ultimately (and quite rapidly, in fact) VHS had taken command of virtually the entire market. This outcome emerged despite the fact that Betamax was technically a slightly superior product: the free market does not always and necessarily lead to the triumph of the best and most efficient technologies, as neoclassical economic theory would have one believe. An even more striking example of this is the QWERTY keyboard, which was actually designed (back in 1873) to slow typing down, since the early typewriters tended to jam when typists went too fast. But because one company started mass-production of a typewriter with the QWERTY layout, more typists began to learn the system, which meant that other typewriter companies began to offer the QWERTY keyboard. And so on.

According to Arthur, the law of diminishing returns or negative feedback will continue to apply to the resource-based parts of the economy (agriculture, bulk-goods production, mining). But for knowledge-based industries - computers, pharmaceuticals, missiles, aircraft, cars, software, telecommunications, fibre optics - it seems that the law of increasing returns and winner-take-all will prevail. These are industries which require large initial investments in research, development and tooling, but where incremental production is relatively cheap once sales begin. The process of increasing returns gathers pace further when items like computers or telecommunications that work in networks requiring compatibility are involved. And all pharmaceutical companies know the importance of being first in the market with a drug, since doctors and patients prefer to stick with what they know.

Self-reinforcing mechanisms also apply to hi-tech manufacturing and trade on an international basis, Arthur points out. He uses the example of the Japanese car makers and their inroads into the US market for small cars in the early 1970s. 'Countries that gain high volume and experience in a high-technology industry can reap advantages of lower cost and higher quality that may make it possible for them to shut out other countries,' he writes, adding that the process may also be observed in the world markets for television sets and integrated circuits.

So how should countries respond to a world economy where such rules apply? The appropriate policies for hi-tech parts of the economy, suggests Arthur, would encourage industries to be aggressive in seeking out product and process improvements - strengthening the national research base, encouraging firms in a single industry to pool resources in joint ventures sharing marketing networks, technical knowledge and standards.

On the corporate level, he suggests (in an interview with Fred Gluck, managing director of McKinsey) that the 'essence of surviving' in a positive-feedback environment is to be highly adaptive: 'If the flow is in your direction, go with it; if it isn't, don't resist - retreat.' Withdrawal isn't easy, he admits. 'The problems many previously successful companies are experiencing today stem from an inability to let go.' The role of the CEO is to act more like a venture capitalist: 'What counts is intuition, judgment, risk-taking and providing support and nourishment to a fledgling project.' Ralph Stacey speaks in similar vein, emphasising the role of intuition (derived, he stresses, from experience, rather than the notion of some angelic gift) and viewing problems in the round, holistically, in dealing with a 'chaotic' and unpredictable world. Managers, he suggests, must learn to reason through induction rather than deduction; and to argue by analogy, to think in metaphor and to accept paradox.

His own chaos-inspired management writing is wide-ranging and evolutionary in interest, covering such topics as the emergence of strategy, the dynamics of groups, and spontaneously self-organising systems. He draws on the scientific concept of the 'edge of chaos' as a metaphor for creativity in business: 'Tucked away between stability and instability, at the frontier, non-linear feedback systems generate forms of behaviour that are neither stable nor unstable. They are continuously new and creative. This property applies to non-linear feedback systems no matter where they are found.' All human organisations, including businesses, are precisely such non-linear feedback systems; and while it is not necessary or indeed desirable for all organisations to be chaotically creative all the time those that do should not think in terms of stability and adapting to their environment, but in terms of using 'amplifying feedback loops' or self-reinforcing mechanisms to shape customer needs.

On the question of planning and strategy, Stacey's starting point is that 'anything useful' about the long-term future is essentially unknowable, because of the very nature of the system. This undermines the conventional wisdom that success depends on a 'vision of where the company wants to be in 2020', the strategy to get to that point, and a shared culture. Instead, he believes, managers should recognise that those strategic planning meetings every Monday morning serve a ritual rather than a functional purpose, rather like the ceremonial laying of the foundation stone on a building. They should recognise too that those elaborate computer-modelled forecasts presented to the board to convince them of the wisdom of a proposed business venture are a fiction, and that their purpose is to allay anxiety rather than perform any genuinely predictive purpose. Real strategy, Stacey believes, is not derived from this sort of planning. No, real strategy emerges - from group dynamics, from the politicking and informal lobbying in the corridors, from the complicated patterns of relationships and interplay of personalities, from the pressure groups that spring up after the formal meeting is over.

And real success, he reminds you, lies not in total stability and 'sticking to your knitting', but in the tension between stability (in the day-to-day running of the business) and instability (in challenging the status quo). 'Instability is not just due to ignorance or incompetence, it is a fundamental property of successful business systems,' he believes. To this end, he says, creative organisations encourage counter-cultures and subversion: Honda, for example, hires large groups of managers in mid-career from other organisations with the express aim of introducing challenge and contention into the company. It encourages real-time learning and 'creative destruction', sloughing off old comfortable concepts and mental models and embracing new. Obviously, managers need to think ahead, all the time, he says. But the conventional view - that proposed ventures have to be validated by detailed predictions of the benefits in terms of discounted cashflow - traps companies into the unadventurous and the shoddy, and is based on an illusion. So what, one wonders, do managers make of it all? Stacey reports that his views usually provoke two very decided responses - anger and fear, on the one hand, and excitement or ready recognition on the other. The first camp resent not being given any off-the-shelf prescriptions and feel ill-at-ease with the ambiguities and uncertainties to which they are being exposed. The second, it seems, feel a sense of relief at finding out that they are not alone in being unable to forecast the future.

Comments Alan Wilson, chief executive of Anglian Water, where Stacey has conducted one of his workshops: 'A lot in life and business is totally unpredictable. Of course you have to plan ahead, research your markets, make sure you've got the right people and resources and strategy for new business ventures. But there are some things you can forecast and others you can't, and sometimes you have to go ahead because you believe instinctively that an action or investment is the right thing. We surround ourselves with forecasts and computer models for comfort, but everybody knows these are all speculative; and you can waste hours debating sets of financial projections, none of which can have any bearing on future reality.' If the influence of chaos theory on management practice is to bring greater honesty and realism, he thinks, then it can only be a good thing.

Find this article useful?

Get more great articles like this in your inbox every lunchtime

Subscribe

Get your essential reading delivered. Subscribe to Management Today