Just as Moliere's bourgeois gentilhomme was thrilled to learn that he was talking in prose, I was delighted when Gary Klein's book convinced me that I'd once had a tiny flash of insight myself.
Many years ago, a girlfriend and I were the only guests in a bleak hotel on Dartmoor, with bedrooms fashioned from a ruined old stable block. Around 3am, the girlfriend woke me and told me she had seen a ghost.
We were keen to investigate further, but the trouble was, the only way to do this would be to interrogate the owners, and they would deny any knowledge for fear it wouldn't look pretty on Trip Advisor. In the morning, we strolled over to the restaurant and the proprietress came in to take our order.
'Good morning,' she said. 'Tea or coffee?' We ordered and she left. 'It wasn't a dream,' I whispered, 'she knows.' The girlfriend looked at me as if I was an idiot - I hadn't asked anything.
But I had sensed that it was a classic case of the dog that didn't bark. When you stay in country hotels, the first thing they always ask is 'How did you sleep?'. This woman hadn't - probably because she didn't want to hear the answer. Emboldened by this realisation, I put the thumbscrews on her until the very grisly truth tumbled out.
Klein starts off with something similar: a patrol car is stuck in a traffic jam behind a brand-new BMW whose driver is smoking. The cops watch as the driver flicks his ash all over the upholstery. 'What kind of guy would do such a thing to his new car?' one cop muses. Then the other has a flash of insight - no one would. So they pull him over and, yes, he has nicked it.
Such examples are the simplest forms of insight that Klein describes: a one-off event, requiring no preparation or knowledge (beyond awareness of social norms) and the insight comes from a combination of subconscious detection of an anomaly and the mental leap in identifying the most likely cause of it.
In his research for this book, Klein worked through 120 cases, many of them far more complex and important. In several, it was a case of scrutinising apparent coincidences, investigating whether a wider pattern could be discerned and worrying away at the issue until the underlying reason was uncovered. This led directly to the discovery of Aids, pulsars and the cause of stomach ulcers.
In the last of these, when Barry Marshall suggested stomach ulcers might be caused by bacteria, no one took him seriously. Even when he took samples from ulcer sufferers, the first 30 cases showed no evidence of bacteria. But the 31st did, as, by pure chance, that sample hadn't been examined until five days after the others.
It turned out that this type of bacteria needed longer to grow a culture. This led Marshall to two conclusions. First, don't necessarily trust all evidence, and, second, the greatest obstacle to knowledge isn't ignorance; it's the illusion of knowledge.
Klein then explains why many systems and organisations - even those that claim to encourage innovation - unintentionally obstruct insight. They target the reduction of errors and the perfecting of an existing process, rather than producing an environment in which will-o'-the-wisp insight and true creativity can flourish. This may explain why the greatest breakthroughs rarely happen in big companies or groups.
He also argues that bouts of insight are less likely to happen in relaxed conditions - pressure helps. The ideal circumstances are what Klein terms 'creative desperation': his favourite example is how Daniel Boone brilliantly improvised a rescue of his daughter from a Shawnee raiding party.
Much of the rest of the book is devoted to explaining the mechanics of insight and tips as to how to achieve it. True insight almost never occurs when we are concentrating on a task. We are more likely to have an epiphany when hill-walking than when writing a strategy document.
Will this book make you more insightful? Probably not, just as men who read earnest books about attracting women usually find that in the real world their well-rehearsed witticisms and killer displays of emotional intelligence fail to trump the instant impact of their receding hairlines and expanding waists.
So don't think of it as 'insight for dummies', but rather as a brilliant discourse on a fascinating subject. It's written in a crisp, fluent, Gladwellish way and the pages flit by.
- John McLaren is chairman of the Barchester Group, a non-executive director of several companies in the UK and US, and a novelist.
Seeing What Others Don't: The remarkable ways we gain insights
Nicholas Brealey Publishing, £12.99