UK: THE DISASTER BUSINESS. - Computer failure is not the only company nightmare. Bombs have brought a new view of risk - and a windfall for some.

by Malcolm Brown.
Last Updated: 31 Aug 2010

Computer failure is not the only company nightmare. Bombs have brought a new view of risk - and a windfall for some.

Michael Warner won't forget 24 April in a hurry. It was his 46th birthday. It was also the day that he and 24 colleagues at chartered surveyors Richard Ellis narrowly escaped being blown up by the IRA. The firm had called a partners' meeting for that day, a Saturday. It was a toss-up whether it should be in an hotel or in the firm's boardroom at Old Broad Street in the City. The vote went in favour of a Surrey hotel. Had they chosen the boardroom the partners would have caught the full impact of the Bishopsgate bomb which went off at 10.24am. They would have been cut to ribbons, says Warner. 'The practice could have been decimated.'

Ironically, in the previous few weeks Warner had been putting the finishing touches to a disaster recovery manual for the firm's clients. Based on the experience of the previous year's bomb in St Mary Axe, the document is now being updated to incorporate lessons learned at Bishopsgate. Warner hopes it may help companies to devise their own disaster contingency plans. Many firms in the Square Mile now have such plans. The week after the Bishopsgate bomb, MORI and the Financial Times polled more than 100 organisations in the City and found that 74% of them had some form of contingency plan for dealing with crises caused by incidents such as terrorist bombs. That sounds impressive, but in fact most of these plans have been drawn up quite recently - 45% since the St Mary Axe bombing. A narrow escape, it seems, concentrates the mind wonderfully.

Disaster recovery is now a business in its own right. Over the past decade a cottage industry has sprung up which, estimates suggest, is worth about £150 million annually and growing at 25-30% a year. For a price, the disaster recovery specialists will provide companies with everything from back-up computers to bomb-, fire-and flood-proof dealing rooms where foreign exchange dealers, for example, can retreat if their own premises are made unusable.

The reserve computer room might be a 'hot site' - with computers in place, tested and ready for use - or a 'mobile', which can be wheeled to wherever the client wants it. Right of access to a hot site, complete with mainframe, would typically cost £50,000 or more a year, plus a daily rate for actual use if the company had a real emergency. 'That gets you a computer centre which becomes yours for the time you need it,' says Isobel Turner, marketing manager for one of the bigger disaster recovery companies, Comdisco Europe.

As part of the service, Comdisco will carry out an examination of all the client's communications, specify the site, and provide testing. 'The customer will come to the centre for three or four days and run a test, putting all his computer tapes on to our machines and getting all the bugs out. He can then ring us up and say, "We've got a problem, we want to come in."'

The focus of disaster recovery has, until recently, been almost entirely on information technology. That's to be expected, perhaps, given the central role that it plays in almost every aspect of business life. However, experts are now concerned that a sort of tunnel vision has been developing. Contingency plans have been too narrowly-based, believes Nick Simms, chairman of the financial industry special interest group of Survive, an organisation which brings together both the service providers and the users. Plans tend to be IT-orientated rather than business-orientated, often because the planning has been delegated to IT specialists who naturally think about disaster in computer terms. Yet, as Simms points out, incontrovertibly in view of recent events, by no means all disasters are computer-related.

'Take the example of being unable to get into your building - for whatever reason. There's no problem with your IT. The computers are working fine, you're just not allowed into the building to touch them. What you've got here is a business problem, and if your plan is a computer disaster recovery plan you're not covered... The trouble is that people are not doing the sort of things they need to do. They're not doing a risk analysis of the whole business. They're looking at the computer and saying "What happens if the computer fails, how do we recover from that?" rather than "What happens if something goes wrong and we can't do business?"'

The difference between an IT disaster and a company disaster is vividly illustrated by the Wallis Fashions fire, a spectacular conflagration which took place nearly a decade ago but which experts still regard as the archetypal catastrophe.

The fire destroyed Wallis's north London headquarters, along with its warehouse, stocks, designs, patterns, accounts - and computers. So complete was the disaster that the next day the company's staff had to be despatched to rush round its West End shops, pulling clothes from the rails so that they could be unstitched and new patterns drawn from them.

Although IT was affected of course, this was not simply, or even primarily, an IT disaster. If you have lost your warehouse and all its contents, observes John Brien, information systems controller at Sears Womenswear, which owns Wallis, the reinstatement of computer programmes which govern things like dispatch becomes almost academic. 'IT in isolation is nothing,' says Brien. 'The whole of a business should understand what it needs to do in the event of a disaster - how the business will operate, what its priorities will be. It's important to have a company disaster plan, not just an IT disaster plan.'

Companies are slowly taking this kind of thinking on board and the most forward-looking are adopting a more 'holistic' approach to disaster recovery. Warwick Woodhouse, director of training and consultancy services at travel group Thomas Cook, is one of the new generation. Business recovery planning, in his view, has three key objectives: to highlight preventative actions and methods, to minimise the impact of the disaster on the business and its customers, and to improve the company's ability to recover efficiently and effectively.

'Recovery is the planned process for getting back to where you were before the disaster,' says Woodhouse. 'Although the minimising process can keep the business running for a relatively short period, a return to normality is required as quickly as possible.'

Michael Warner's disaster manual extends far beyond IT. Not surprisingly, given the nature of Richard Ellis's business (plus the enormous physical damage caused by the St Mary Axe and Bishopsgate bombs), he places considerable emphasis on the property-related aspects of recovery. Warner sees contingency planning as a four-stage exercise: evaluation, evacuation, recovery and resumption - and, possibly, relocation. The initial stage, evaluation, is concerned with measuring the risks associated with the building that the company occupies - or which it is thinking of moving into - and considering how these might be minimised. 'Everyone should now be thinking of evaluating the risk of their premises and what could happen to them,' argues Warner. Bombing is not the only danger, nor necessarily the greatest. 'Are you on an aircraft flight path? Are you beside a dam which might burst? What sort of building are you in? If it's all glass should you be considering blast curtains, a different form of filing, a clear-desk policy?'

Stage two, evacuation, is mainly about preserving life when disaster strikes - or appears imminent. The third stage - recovery and resumption - is crucial in a business sense. Customers and clients will be sympathetic, but sympathy soon drains away if normal service is not resumed quickly. The first two or three days are likely to be critical. 'You need to have somewhere to go immediately,' advises Warner. 'You've got to keep your business going in the period following any disaster. You need to be operational the next day. You need to have access to someone who knows where there are premises, which are probably second-hand, which probably have telephone wiring in, and probably have some desks available. You've got to get in quickly.'

If the original premises are really badly damaged, the company may in the last resort have to consider relocation. The problem then is that it will have to take what are essentially long-term decisions - with major business implications - much more quickly than it would like.

Some of the detail in contingency plans will need to be approved at a very high level. The crisis team must have a thorough understanding of the dynamics of the business, so that it can judge the impact of the disaster and decide how best to keep going with minimum interruption. Nevertheless, a lot of the detail, particularly the bits relating to the period immediately following the disaster, will seem prosaic in the extreme. The team will need access to office layout plans, for example. It will need the home telephone numbers of key personnel; also lists of contractors (roofers, glaziers, plumbers, office equipment suppliers) and their out-of-hours telephone numbers. However basic it may appear, this sort of nuts-and-bolts information can be as vital to a successful recovery as the high-level strategic factors.

In the wake of the 1992 bomb, Bev Fitzgerald, a director of loss adjusters Thomas Howell, monitored progress at two office buildings, one on either side of one of the streets just off St Mary Axe. 'One was the subject of a contingency plan, and pre-organised glaziers had completed all necessary repairs by the Sunday, two days after the bombing. The other was still open to the elements a fortnight later, as that landlord had no plan and had been let down twice by casual glaziers. His tenants were, not unnaturally, up in arms.'

Insurance companies were among those hardest hit by the City bombings. But, of course, the insurance industry has its own special way of responding to disasters. As Fitzgerald points out, the lessons about contingency planning that were brought home by the bombings have not been lost on the big insurers. 'Increasingly insurers are going to look to policyholders to have worked up proper contingency plans which have been audited. As a corollary to that they must, in my view, offer a premium reduction for people who do.'

Six months after Bishopsgate, Warner is able to assess how well his own recovery plan stood up to a real-life disaster.

He thinks he got much of it right and intends to learn from the mistakes - incorporating the lessons into the updated version. On the plus side, most of Ellis's City operation was up and running by the Monday morning. A crisis management team, called together by Warner, had met in the firm's Berkeley Square offices at 4pm on Saturday afternoon, and it worked overnight with wiring contractors, telephone engineers and furniture suppliers so that 130 'exiles' from Old Broad Street could move into the West End building on Monday and Tuesday.

Business-as-usual letters were dispatched to clients on the Sunday night, a newsletter was written and on the desk of every member of staff on Monday morning, and by Tuesday one of the partners had negotiated a deal for new offices in the City. The interruption to normal business was minimal.

The minuses, says Warner, were things that seem obvious in retrospect but which were overlooked at the time. One of the biggest oversights was the absence of any priority lists for the retrieval of important material from the damaged building. It had simply been assumed that, after a disaster, it would be possible to obtain reasonable access, and that all material could be got out in an orderly fashion. 'In the event we were in for 10-minute smash-and-grab raids: "You've got 10 minutes and you can only take what you can carry". Having to select, from all departments, what they deemed most important was a very difficult issue.'

And over at National Westminster Bank, Mike Newens, director of group property and central services (and the bank's crisis chief), had a closely similar set of experiences - on a slightly grander scale. When the bomb went off Newens was actually in his office, although fortunately this was nearer to King's Cross station than to Bishopsgate.

He and his colleagues had been preparing a business plan which needed to be completed, printed and delivered by the following Wednesday. 'I heard a thud and took no notice of it,' recalls Newens. 'Then just about every phone started to ring, the pagers were going off and we quickly understood what was going on.'

At first Newens believed it was a great advantage to be so close to the scene. With the benefit of hindsight, he now thinks differently. The information coming through in the first hour and a half was hopelessly confused - and confusing - which was inevitable given the extent of the catastrophe. The bank's flagship building, NatWest Tower, was severely damaged, and 19 other buildings within a half-mile radius were affected. A total of 2,000 employees were displaced.

The efforts of the emergency team were remarkably successful. Just under half the 2,000 were back and working normally first thing on Monday morning, the rest within 48 hours. But much more important, says Newens, careful planning meant that all the business units were operational by start of business on the Monday.

The first 1,000 to return had been carefully chosen. 'If you have 10 businesses within the affected area, and the people you have in place represent only five of those businesses, then you've failed. What we did was to have core people to operate all the business units.'

Constructing a contingency plan, Newens suggests, is rather like constructing a business. This particular business has one clearly defined purpose - recovery. It is essential that it should have a structure that will operate for the period of the disaster and right through the subsequent recovery period.

'One of the key elements of a successful recovery is that the structure is known and understood,' maintains Newens. 'The authorities within the structure must be understood and communications within the structure must also be understood.' He makes a powerful point: 'The last thing you can spend time doing during an incident is debating how you're going to manage it.'

Like Warner, Newens acknowledges that some mistakes were made. The batteries for the bank's emergency mobile phones were all flat. Also, as the team found to its intense irritation, the 'call barring' system, which prevents access to directory enquiries and similar numbers during normal working, also prevents access during an emergency.

The City of London bombs highlighted - as nothing else could - the need for contingency planning and for disaster recovery planning. But will it all be a seven-day wonder? Will interest evaporate?

Bev Fitzgerald, of Thomas Howell, is one observer who is convinced that it won't; that past mistakes will not be allowed to recur. 'Even with a contingency plan it's difficult enough operating after, say, a catastrophic explosion. But the plan gives you a basis. Everyone knows what they're doing, you don't have people rushing around like headless chickens... You should spend the first few days saving your business - not working out how to save your business.'

In rare instances - where the effects of disaster are unusually widespread - effective planning may do much more than save the business. In 1991, during the Gulf War, disaster recovery planning not only allowed Thomas Cook to continue servicing the Middle East, it enabled the company to capture business from less well prepared competitors. Cook's Bahrain office had been through a contingency planning exercise some months earlier, when local managers asked themselves a string of 'what if?' questions: such as what if aggression takes place in the Middle East, or some manifestation of fundamentalist extremism?

The exercise produced three main conclusions. First, that in the event of an emergency Thomas Cook must maintain its presence in the region. Second, that it must maintain the flow of products, especially travellers' cheques. Finally, that the Bahrain operation should be organised in such a way that, if lines of communication to London were cut, it could continue to function more-or-less autonomously.

Bahrain staff predicted that the airport would be closed in an emergency, so set up a contingency distribution system for travellers' cheques. The plan went into operation. After the outbreak of war, Arab businessmen were queueing to transfer local currencies into travellers' cheques which were safe and negotiable. 'The company's commitment to keeping travellers' cheques in distribution impressed its agents,' claims Warwick Woodhouse. But the key element of the contingency plan was that major management functions should be devolved to Bahrain. 'A number of our competitors - who hadn't thought that through - could not function and make decisions down there because they did not have that authority... As a result, Thomas Cook Bahrain gained a major competitive advantage.'

In most circumstances, though, recovery is the name of the game. Disaster recovery experts often cite the '80% rule', which is based on surveys conducted both in America and in Europe. This states that 80% of companies which lack a well conceived and tested contingency plan go out of business within two years of suffering a major disaster.

How many British companies have effective contingency plans to enable them to recover from disaster? Research, by Keith Hearnden of Loughborough University, which was published earlier this year, suggests that the situation (even as far as computers are concerned) is far from satisfactory.

Hearnden examined a cross section of 421 organisations in commerce, manufacturing and the public sector. Although 239 of these (nearly 57%) had drawn up contingency plans designed to cope with a total breakdown in their computer facilities, only155 (37%) had ever tested their plans, and just 94 (22%) had tested them within the past six months.

Hearnden concludes that less than a quarter of UK organisations have a reliable plan. The notion that a contingency plan cannot be trusted until it has been tested is well founded. One survey of plans at large IBM mainframe sites showed that half of these failed, either completely or substantially, when first tested.

Find this article useful?

Get more great articles like this in your inbox every lunchtime

How COVID changes the world forever: A thought experiment

Silicon Valley ‘oracle’ Tim O’Reilly imagines how different sectors could emerge from the pandemic.

The CEO's guide to switching off

Too much hard work is counterproductive. Here four leaders share how they ease the pressure....

What Lego robots can teach us about motivating teams

People crave meaningful work, yet managers can so easily make it all seem futile.

What went wrong at Debenhams?

There are lessons in the high street store's sorry story.

How to find the right mentor or executive coach

One minute briefing: McDonald’s UK CEO Paul Pomroy.

What you don't want to copy from Silicon Valley

Workplace Evolution podcast: Twitter's former EMEA chief Bruce Daisley on Saturday emails, biased recruitment and...