To Combine or Not to Combine - Selecting Among Forecasts and Their Combinations

In many situations, even small improvements in forecasting accuracy can have a major influence on a company's bottom line, impacting business areas ranging from cost reduction to providing better customer service. INSEAD Senior Research Fellow and Lecturer Michèle Hibon, and Assistant Professor of Information Systems Theodoros Evgeniou, utilize information from the M3-competition to compare the performance of combined and individual forecasting methods with respect to method selection, inherent risk, and whether results differ between best and worst-case scenarios.

by Theodoros Evgeniou, Michèle Hibon
Last Updated: 23 Jul 2013

In many situations, even small improvements in forecasting accuracy can have a major influence on a company's bottom line, impacting business areas ranging from cost reduction to providing better customer service. However, technical knowledge of forecasting required for accurate planning is often lacking, resulting in material losses and significant opportunity costs. INSEAD Senior Research Fellow and Lecturer Michèle Hibon, and Assistant Professor of Information Systems Theodoros Evgeniou, utilize macroeconomic, industry and company time-series data from the M-3Competition to compare performance between combined and individual forecasting methods.

Stressing simplicity of modeling, practical benefit and minimizing risk of forecast inaccuracy, this paper aims to increase understanding of a powerful tool for planning and prediction of future events.

The method of combining forecasts was first introduced in 1969 as a potential improvement over using any one individual method, from the range of alternatives. Over the years, much research has followed, comparing methods to determine which is most effective.

In 1979, Makridakis and Hibon led an ambitious study to compare a large amount of major time series methods across multiple series. While this study offered some groundbreaking conclusions, a more important legacy may be its role as a catalyst for developing a series of competitions to serve as a forum for structuring future research.

In 1982, the M-Competition was launched (see Makridakis et al. 1982 for further information) to encourage a common ground for independent empirical research and to advance collective understanding of factors exerting influence upon forecasting accuracy. A key benefit of this competition was that it enabled conclusions from one study to be replicated using the same data, providing greater validity of results.

This proved to be a milestone in forecasting, with the paper responsible for establishing this event, <em>M-Competition: The Accuracy of Extrapolation (Time Series Methods): Results of a Forecasting Competition</em> (Makridakis, Hibon et al 1982), being recognized as so important, it was awarded the top prize as 'Favourite Paper Over the Last 25 Years' by the International Symposium on Forecasting. This success led to additional forums including the M-2 and M-3 competitions, in 1993 and 2000 respectively.

Major conclusions included the fact that statistically complex methods did not necessarily provide more accurate forecasts than simple methods, and that, on average, combinations of methods outperformed use of individual methods. However, since then other studies, notably Larrick and Soll in 2002, have challenged these conclusions, suggesting there are some conditions where it is better not to combine forecasts of experts.

To respond to this, data from the M-3 Competition was used to compare individual and combined approaches, to determine whether one method had an advantage over the other, on average, as well as whether practical considerations of risk could be significant in this study.

The findings revealed that best performance, for both individual and combinations of methods, was similar. This result also held for cases where the same methods were used for each time series studied, as well as when different methods were used. A clear advantage, however, was found in the lower range, as the worst performance of the individual methods was far more inaccurate in comparison to the worst performance of combined methods.

The most important finding may regard the practical applications of this research. While some conditions have been discovered to favor individual forecasting methods, this also introduces the very real problem of how to choose which method to use. This selection factor and the considerable difference observed in downside volatility, combine to demonstrate the presence of much greater risk when using individual methods.

Future study aimed at improving forecasting accuracy will likely focus on selection of individual methods, building on foundations from the M-3 Competition to assess factors including spread, variance and bracketing rate. While this may ultimately result in further gains in accuracy, combining methods seems to provide the best practical solution available now. These results should provide benefits for many lines of business including economic forecasting, supply chain management, drawing insight from consumer behavior and even evaluating trends in capital market prices.

International Journal of Forecasting, January-March 2005

Theodoros Evgeniou, Michèle Hibon recommends

Click here to read the full article on INSEAD Knowledge

Read more

Find this article useful?

Get more great articles like this in your inbox every lunchtime

When should you step down as CEO?

Bob Iger's departure poses an unpopular question for bosses.

The death and resurrection of the premium customer

Top-end service is no longer at the discretion of the management.

What HS2 can teach you about project failure

And how you can prevent projects going astray.

35 Women Under 35 2020: Nominations open

Management Today's 35 Women Under 35 showcases the country's rising stars in business. Here's how...

Practical steps for breaking silos

Briefing: Adam Williams, former CEO of influencer marketing agency Takumi, shares what he has learned...

The Power 50: Proof that you can be a part-time CEO

Just a few years ago, executives were reluctant to admit they worked part-time for fear...