UK: Cut the chaff, keep the wheat.

UK: Cut the chaff, keep the wheat. - Too much information can be as dangerous as a little knowledge, but new technology could provide the cure for information overload.

by Jane Bird.
Last Updated: 31 Aug 2010

Too much information can be as dangerous as a little knowledge, but new technology could provide the cure for information overload.

Never before has so much information been available - according to a report released by BT last month, British office workers consume an average 125,000 words every month at work. That's the equivalent of reading 15 Jeffrey Archer novels a year.

More information has been produced in the last 30 years than in the whole of the previous 5,000. The Internet and e-mail have added a vast new dimension to the problem and many business people now feel they are drowning in data. A survey by Reuters last year found that one-third of managers suffer from ill health as a direct consequence of stress associated with information overload. Almost half thought the Internet would be a prime cause of information overload in the next two years.

And that can be bad news for business. In the UK 1.7 million days are lost to stress-related illness every year at a cost to business of some £3.5 billion. Productivity is also adversely affected - 43% of the managers surveyed by Reuters believed that their ability to make decisions was adversely affected by having too much information.

Many businesses rely on third parties - in the form of on-line information services such as MAID, Reuters and FT Profile - to sift information for them. However, users still have to go out and search the service provider's database for what they want, just as Internet users surf the Web - a method of information access described in computer jargon as 'pull'. The problem with this approach is that it takes time and you need to know what you're looking for. Anyone who has used the Internet will be familiar with the experience of typing in a couple of key words and scoring tens of thousands of hits. It's easy to summon a vast quantity of irrelevance, and it's also easy to miss a crucial piece of information.

The software industry has been aware of the problem for some time and much effort has gone into areas such as decision support, knowledge engineering and data mining. These products attempt to make sense of large databases by providing better searching and sifting tools. 'Push' technology tackles the problem from a different angle, anticipating what people will find useful and sending it directly to their desktop. Push technology comes in four forms: e-mail; broadcast (using space in TV broadcasting frequencies); off-line browsing/Webcasting (you periodically poll your favourite Web sites and pull down any changes since the last download); and personal broadcast networks which send programs in addition to data so that users can exploit it better. No longer need customers remember to take the initiative and call suppliers: details of new products, price discounts and service schemes can be sent directly to them.

Industry experts believe that push technology will play a key role in commercialising the Internet. 'Push will drive the use of the Internet well beyond today's manual fetch-it-from-the-Web-site,' says a report from Forrester Research of Cambridge, Massachusetts. 'Consumers will gain the convenience of automatic delivery,' it continues.

The business information services have been among the first aboard the push bandwagon. MAID, the London-based business information provider, is deploying push technology, through the 'alert' function of its Profound service (annual subscription £5,950). Users set up personalised alerts to track information on companies, markets and areas of activity such as mergers and acquisitions. Fresh information is then sent to users as required, on a daily, weekly or monthly basis.

Reuters, meanwhile, has identified push as a key direction for future development via its California-based subsidiary Tibco, which specialises in Webcasting technology. The intention is to apply this to Reuters Business Briefing, the news and business information database (currently priced from nothing to £500 a month depending on the level of service).

On-line business information providers are also keen to offer their subscribers rapid access to news stories - a service already offered by US-based PointCast on the World Wide Web. Personalised news services allow users to specify which stories they want to receive, specific to an individual company, sector or topic.

FT Profile, part of the Financial Times group, launches its first push service, News Alert, this month. News Alert will inform people within large organisations of up-to-the-minute news stories relevant to their business. A general category of stories is defined as relevant to the corporate subscriber's business - pharmaceuticals or telecommunications, for example. A PC within that organisation maintains a permanent link with the News Alert service so that it can receive stories as they break.

The PC acts as a server on the organisation's intranet so that the stories can be accessed by hundreds of employees who have to simply click on an icon on their screens. Particular stories can also be packaged for individuals in client companies with special interests.

MAID's offering, Profound LiveWire will be available later this year.

Prices for such services will vary according to speed, access and quality of service. Profound LiveWire, for example, will start at £150 a month for up to 50 users. PointCast is free to users and paid for by advertising.

However, push technology alone will not solve the information overload problem. Indeed, e-mail, another form of push, is fast becoming a prime cause of the trouble. A survey by the US software company Novell found that 94% of employees waste one hour each day dealing with unnecessary e-mails, while half receive flame-mails or abusive e-mails from managers and colleagues.

On top of this, a new plague has hit e-mail recipients in the form of unsolicited e-mails or adverts offering anything from adult videos to slimming pills. Spamming, as the practice is known, has become so widespread that its volume has at times disabled parts of the Internet.

Many e-mail problems can be solved simply by better electronic etiquette.

In Novell, for example, staff are told never to send e-mail in haste or to too many people. The company has also incorporated a number of features into its networking software to help filter e-mails. For example, if messages are prefixed with special identifiers - 'junk' is suggested for e-mails offering cars for sale - they can automatically be parked in folders.

E-mails from particular individuals or on certain subjects can be similarly treated. They can also be set to expire automatically after a given period of time.

The idea of better labelling of the content of documents has also been implemented at US-based Sequent Computer Systems, a systems integration company, which is building a database of corporate knowledge on its intranet.

The aim is to create a single place for people to turn to for whatever corporate information they require. To help create useful indices and contents tables, anyone who puts information on the system must assign characteristics to it such as subject and type of data. 'The data can then be accessed in two ways,' says Roger Swanson, Sequent's chief knowledge officer. 'You can either browse through the logical structure, clicking on links like a table of contents, or you can search on key words.' If people want to provide feedback to the person who provided the information, they can send comments such as 'not so' or 'tell me more'.

Intranet databases such as Sequent's are gradually reducing information load because individuals no longer need to be sent copies of documents they can access remotely at will. Intranets also make push technology more viable by acting as a funnel through which companies can channel employee access to the Internet. They reduce congestion which can at times slow the Internet to near standstill, while allowing companies to circumscribe what individuals are allowed to view. 'Intranets inside corporations have created a fundamental shift in the market and are the key driver behind the next generation of information services,' says Donal Smith, managing director of the electronic business information division of FT Information. 'They make it so much easier to publish data for hundreds of thousands of people.'

Microsoft's model for solving the information overload problem is the equivalent of an electronic secretary or PA that knows your preferences and learns by observing your work patterns. 'A human assistant doesn't have everything about you written down, nor do you tell your assistant everything explicitly,' says Andrew Lees, director of product marketing at Microsoft UK. 'He or she just learns about you as you go along.' A lot of this could be done by software in the future, Lees says.

Microsoft is investing heavily in research on linguistics and the interpretation of natural language so that ultimately its software will enable PCs to listen, read and learn. 'As they become more intelligent, they will be able to help far more effectively in sifting relevant information,' says Lees. However, at present facilities are fairly crude. So far, Microsoft's main offerings are document summarisation and mapping, available under the latest version of Word, its word processing software. Document summarisation attempts to understand the key parts of a document by looking for clues such as headlines, underlined sections and recurrent words or phrases. It uses these to make a precis of any length.

Document mapping aims to provide the equivalent of leafing through a sheaf of papers to spot a particular page or paragraph. It uses a technique similar to hyperlink on the Internet - click on a key word and the text skips to its next recurrence or to a related word. 'These techniques help navigate long documents and will improve as we get better at linguistics,' says Lees.

The first incarnation of Microsoft's personal assistant model will appear this autumn in the form of a proactive agent that can go out on the Web and find information of particular relevance to an individual. The agent facility will be incorporated in the next version of its browser, Explorer 4, due for launch in October. Your agent knows which Web sites interest you and even which pages within those sites you tend to visit. It can be set to dial sites as frequently as required, find out whether there have been any changes, and if so, immediately notify you.

Employees with PCs on corporate networks permanently linked to the Internet will see a message at the bottom of their screens informing them when one of their favourite pages has changed, when the change occurred, and perhaps even a summary of the change. 'People with laptop computers could set their machines to dial the Net before taking the train to work and then read the updates during the journey,' says Lees.

Microsoft is also enabling information providers to personalise their Web sites for individual visitors. Using its SiteServer software, providers can observe callers' viewing patterns, building up databases about topics that interest them. Details such as name, address, age, work and hobbies can also be requested so that Web sites can be customised for each caller.

'An individual version of a Web site can be created for each electronic visitor,' says Lees.

As with corporate databases and information services, better labelling of contents will improve the performance of agents in future and help them more effectively identify what is of interest. It is, however, questionable whether they could ever completely replace the task of searching databases for what you want.

Ultimately the distinction between push and pull technology is likely to become blurred. In future, people will probably summon information to their desktops by pressing on-screen buttons in much the same way as we currently select channels on our TV screens. PointCast, for example, will have a button on the Microsoft Windows desktop later this year. Other information providers are expected to follow suit. Over time, it should become easier to get what you want when you want it, without fear of missing something crucial and without all the extraneous rubbish. Whether this will make us all less stressed remains to be seen.

Find this article useful?

Get more great articles like this in your inbox every lunchtime

Books for the weekend: Daniel Goleman, Jack Welch, Nelson Mandela

Beaverbrooks CEO Anna Blackburn shares her reading list.

What happens next: COVID-19 lessons from Italian CEOs

Part I: Marco Alvera, chief executive of €15bn Lombardy-based energy firm Snam, on living with...

Coronavirus communications: Dos and don'ts

Uncertainty and isolation make it more important than ever to be seen, to be heard...

Leadership lessons: Mervyn Davies, former CEO of Standard Chartered and trade minister

"People talk about pressure – I worked 24 hours a day. There is more pressure...

How to reinvent your career through motherhood and midlife

Pay it Forward podcast: Former Marie Claire editor-in-chief Trish Halpin and BITE managing editor Nicky...