Measuring the impact of IT on business appears to be a political rather than a technical necessity for most companies.
'Would you cost justify having a carpet in your office? No. Would you cost justify having a phone? Of course not. You just know it's an improvement and you spend what you have to.' For Roger Ellis, group IT controller at Blue Circle Industries, there has to be a strict policy on what's measurable and what's not when it comes to the return on IT investment. With 30 years' experience behind him, he has clear views about the benefits and pitfalls involved.
'Just this afternoon I was talking to America, trying to improve our supply chain management,' he says. 'Improve figures by 1% or 2% and the investment pays for itself. But how do you actually prove that improvement was down to our new computer system? We put a percentage on it, but the truth is, much of the decision-making process relies on gut feel.'
Those two little words, 'gut' and 'feel', fill experts in the measurement business with absolute horror. As Robert Pitt, head of technical and financial services at Andersen Consulting, puts it, the 'gut feel' IT director is a thing of the past. 'The IT director who spends on infrastructure or strategic investment without linking it to a defined programme of measurable business change is increasingly rare. It can't be tolerated, partly because the numbers are getting bigger.'
Of course, Ellis is being a tad disingenuous. Assessing the cost benefits of IT is straightforward if you're measuring whether automation - which directly reduces head-count - cuts costs. But he rightly points out that this sort of project is increasingly rare. 'Today's projects are more about improving marketability or market share,' Ellis says. 'And in our bathrooms business, so much is down to things like the weather and the strong pound' - how can you, he wonders, easily separate the effects of these from IT.
Even so, increasing numbers of British firms agree that measurable criteria must be established before making major IT investments. After all, IT has a budget reckoned, on average, at 2% of corporate turnover or a total spend of £41.3 billion in the UK. This fact alone is one reason, says Adrian Quayle, vice president of measurement services at the IT analyst Gartner Group, why the volume of his business increased 70% last year.
With so much at risk, he says, chief information officers and IT managers are more frequently required to provide justification for investments in hardware, software, services and staffing.Understanding and quantifying the value that IT provides is therefore vital.
Gartner's initial approach, Quayle says, is to classify clients broadly into three categories, 'which opens our minds to the sort of client we're dealing with and where they're coming from'. Category A is leading-edge, early adopters of new technology; category B is the very significant investors that don't dive into new technology straight away; category C is the slow adopter. This at least establishes the degree of guidance necessary.
Many chief information officer and IT managers remain ignorant of the real costs of providing or delivering services across functional areas of IT, says Quayle. And at this stage they need to establish barometers to determine areas that are thriving versus those that need attention. Individual cost components, such as the cost per call on the IT help desk, data centre hardware and software costs, staff experience, training levels, space allocation, must all be assessed and measured. However intangible a concept, this degree of measurability is possible, claims Quayle, so long as you can describe it and you have a goal.
Annie Brooking, managing director of Technology Broker, agrees, but is adamant that IT cannot be measured in isolation.The title of her book on the subject, Intellectual Capital, gives a clue as to what she means. 'There's been a shift in the make-up of the net value of a company,' claims Brooking. 'In 1977, 1% of the net value of a UK company was reckoned to be intangible assets.
In 1986 it was 44% and that trend is continuing in a big way.' She cites Nestle's acquisition of Rowntree for twice its share value as evidence.
Before you can consider measuring return on IT investment, says Brooking, a computer scientist by training, any valuation must look at the intangibles, which she breaks down into four areas: market assets, which is anything that gives market power like brands and franchises; intellectual property, such as copyright; human-centred assets like the knowledge of your staff; and finally infrastructure assets, which relates to a company's strength, corporate culture and relationship to the financial community.
'And this is where your IT systems come in. IT isn't the value of computers or software, it's the impact of using that IT on the business,' she says.
'Look at Barclays. If they did an inventory of all their computers and software, it could be worth £100 million. But if that disappeared, the bank wouldn't be able to open its doors, so the value of the IT infrastructure is greater than the value of the box. And when we're looking at an intellectual capital audit, we're looking at the strength of all assets - to what extent is IT fit for the purpose? It's very difficult to put a figure on it.'
Brooking's answer to such challenges is the Dream Ticket, 'the optimal set of assets a company can have in the perfect world'. This breaks down to between 28 and 32 elements, only one of which will always be IT infrastructure.
'One of the first things we do, is to ask, "What's the goal?"' says Brooking.
If the aim of the new IT system is to provide better customer service, you have to quantify what 'better' is. What does the customer service department do? Say a questionnaire establishes that customers currently give service an abysmal two out of 10 score, Brooking's team would then set a target, perhaps of four, then six and finally eight out of 10 over three years. 'So then I can go back to customers and ask what would make them happier,' she says. 'Of course, customer happiness isn't a measurable asset unless their happiness translates into increased sales. And that is obviously measurable.'
Brooking's emphasis on the need to see IT measurement in context is enthusiastically taken up by those on the front line. Disney's senior director of IT, Danny Carrao, argues that there's no such thing as an IT project, only business projects, so any assessment of return on IT, must be seen in that context. 'When I first started here seven years ago, everything was driven by financial targets, but we have really changed and now it is goal-driven, every project is measured, based on how well it will help the organisation achieve its objectives from an investment perspective,' says Carrao.
'The measurability of its success lies in the answer to a couple of questions such as "does it help us strategically?"' Carrao adds. 'Say we want better communications between our European offices so we can reduce the amount of time it takes to get product to market. That's measurable. Three to six months later we'll do a post-implementation review and ask: "Have we achieved what we set out in our expenditure justifications? Have we achieved the aim of never missing key product release dates? Did we meet product design deadlines all over the world?"' The IT is of course a vital part of the process but in terms of specific measurement of IT, he says, 'we don't get too hung up on it'.
Carrao believes there's no such thing as an IT project. It is part of the business, so requires a business person to manage it, not an IT director.
'In the past, we had IT solutions come riding in on a white horse, saying, "We'll save this from happening again". The project makes sense and it gets financial backing. It's implemented, and maybe it does what that person claimed. But it doesn't necessarily grow the business.'
Other executives like to speak the growth-oriented language of those like Carrao - but tend to act differently. Some attempt at measurement at least arms the IT director with the statistics that may reassure less knowledgeable members of the board, who may otherwise be reluctant to give the green light on investment (see survey box). Jonathan Steel, chief executive of the business, economic and political research company Bathwick Group, claims this is one of the most common reasons why IT directors attempt to measure return on investment. 'There's a general issue with senior executives' understanding of IT,' he says.
'They put over-reliance on return on investment calculation. We need commitment from business leaders to educate themselves about IT, so they can make a sensible judgment.' They make too many IT decisions on the basis of subsequent cost savings, rather than innovation gains, he says.
Andrew Bottomley, director of research at the financial analyst and investment house, Durlacher, is as baffled as Steel by the unrealistic expectations many senior directors have of returns on technology. 'This isn't a sterile black box delivering a return,' says Bottomley. 'It needs to be mapped onto the corporate and IT culture. Unfortunately, senior executives are often only happy if they're being sold a particular percentage return.
And that disregards not only the running costs of IT, like renewing licences and additional consultancy, but also how well the IT will fit your corporate culture and how your people work with the IT.'
Few companies show much regard for their culture, he moans: 'If you're an IT director fire-fighting on EMU and year 2000, higher level IT measurement goes by the by. But that's so short-termist.'
In the US, things are rather different, and though the UK is certainly the leading country in Europe in terms of measurement, with the Dutch and Scandinavians not far behind, 'Americans are the measurement kings', says Quayle. On the Continent, says Quayle, you find there's more dependence on state industries in which people initially tend to be more suspicious of measuring the impact of IT.' But in the more hi-tech industries, like banking, insurance, IT, and retail, aided by the increasing globalisation of business, more reliance on measurement is inevitable, he says. The government sector is notably absent from Quayle's list of clients. 'We once hoped out-sourcing would change that but there's reluctance to benchmark those big deals fully.' He adds: 'Government is a type C organisation.'
However suspect the motives or potentially misleading the measurement results, it is worth attempting for one very good reason, says Catherine Griffiths, research fellow at Imperial College, London, and co-author of Regaining Control of IT Investment.
'The very process of staff setting targets, analysing needs and picking up on good practice across the company helps build a reliable picture of the strategy links between IT and business.'
'IF IT GROWS OUR BUSINESS KNOWLEDGE, IT WORKS'
'We spend half a per cent of annual sales - £5 billion to £6 billion - on IT per year,' says Brian Keating, director of business IT development at Safeway, 'but the ABC card, our loyalty card, was a multi-million pound IT project designed to give us leverage and growth.
We approached this from a business perspective first. We didn't worry about IT measurement until we'd got the aims and achievements out of the way, things we knew we could measure, so we'd have a benchmark in the months and years after implementation.'
First, Safeway had customers wanting to get back to old corner-shop attitudes, so the company wanted IT to provide that personal relationship. Second, the aim wasn't to buy the best or cheapest IT, but whatever would fit in with an overall strategy. Finally, the company looked at the purpose of its IT. ABC was futuristic and the main thing Safeway was trying to do was see if the company could find out as much as possible about its shoppers.
'We used a marketing agency to do a questionnaire and focus group discussions,' says Keating, 'then followed up with a very detailed questionnaire in a trial store; we looked at things like changing behaviour, how much they spent,set against measured financial costs in creating a system that would provide us with that information.
'We've ended up,' he adds, 'with a system that can hold all transactional information and then we can use it in store. It changes our thinking.We knew the IT was there, it was a question of setting up our measurement criteria: if it grows the business and our business knowledge, then it works.'
MANAGERS DO IT BECAUSE THEY HAVE TO
Paul McNabb runs Cambridge Information Network (CIN), an internet community of 1,400 chief information officers. Just under half are European, the rest are in the US - and all represent companies with revenues of $50 million to $10 billion. In March of this year, CIN surveyed its members on return on investment. Asked about the effectiveness of IT investment measurement, a third of IT directors said it could not be measured effectively.
Nevertheless, it was still the method of choice for making key investment decisions; 33% said this was because it was required by the company; 61% because they actually believed it was a good way of analysing investment; 64% said it helped them decide IT priorities; 75% said it helped justify their decision.
'They go through ROI because it's good business discipline and builds links between the IT and business units,' says McNabb. 'It's not a rational prioritisation tool, it's the "doing it" that's the value.' IT and business operations consultancy Compass found broadly similar results to CIN. Two-thirds of a survey of more than 500 CEOs of the biggest companies in the US and Europe revealed that while there was scepticism about the efficacy of measurement, it was still carried out for largely political purposes.