How do you forecast hierarchical time series?
- Advanced Analytics, Data Science
- data analysis, forecast, prognosis, time series
- 4 min reading time
Dr. Dominik Bertsche
When it comes to forecasting time series, the end always justifies the means, i.e. the most accurate model is the model of choice. However, the model should always be kept as simple as possible to minimize the risk of overfitting. Data-driven algorithms usually forecast time series univariately, i.e. each observation series is modeled separately.
This means that a time series, such as the profit of a company, is first examined for central characteristics and modelled on the basis of these:
- Trend
- Seasonal effects
- Other dynamics such as autocorrelation
On this basis, the corresponding time series is predicted over the desired horizon. The Smart Predict functionality in the SAP Analytics Cloud offers a good introduction to the prediction of time series. For the implementation of more complex and individualised methods, statistical software solutions such as R or Python are suitable.
For more on forecasting time series in general (including definition, function, and forecasting models), see this wiki article.
What are hierarchical time series?
Hierarchical time series are omnipresent in every company, for example in the form of a profit and loss account (P&L) or balance sheet hierarchy.
For the sake of simplicity, in this article we will consider a simple P&L hierarchy with three levels. The top level of the hierarchy consists of only one line item, the Corporate profit. This result is the sum of the two time series Expense and Incomewhich are on the middle level. At the lowest level are the so-called basic time series Personnel costs, Material costs, Sales revenue and extraordinary income are located here. The expenses the sum of personnel- and material coststhe Yield the sum of revenues and extraordinary income. Thus, we consider a total of seven time series, with the observations of the upper two levels, i.e. Corporate earnings, expense and Yieldare merely aggregations of the basic time series.
In practice, hierarchies in corporations are much more complex, so that ten or more levels are not uncommon. However, the three levels in the previous example are sufficient to explain all the essential concepts related to forecasting hierarchical time series.
Methods for forecasting hierarchical time series
The simplest approach would be to choose the most accurate model for each of the seven time series in the described hierarchy and let it produce a forecast. However, this approach violates the structure of the hierarchy. For example, the forecast of the Effort is usually not equal to the sum of the predictions for the personnel costs and the cost of materials. To fulfil this necessary condition of the hierarchy, three methods exist, which are briefly explained below:
Probably the most intuitive approach forecasts only the basic time series - in this case Personnel costs, Material costs, sales revenue and extraordinary income, using common algorithms. Then the forecasts are aggregated according to the hierarchy described above, i.e. the forecast of the expense is the sum of the forecasts for personnel costs and the Cost of materials.
Although this method is the most obvious variant at first glance, it has its weaknesses, especially in the case of complex and multi-layered hierarchies. This is due to the fact that the time series at the lowest levels often have little structure and are therefore difficult to forecast. Thus, significant forecasting errors can occur here. If the forecasts of these basic time series are now aggregated over numerous hierarchical levels, the danger of aggregating forecast errors is also real. Under certain circumstances, this can lead to strong deviations in the most important business indicators, which are usually located at the higher hierarchy levels.
Alternatively, one can only use the time series on the highest hierarchical level, in the scenario described the company resultusing a suitable algorithm. This has the advantage that this indicator usually has a solid structure and can therefore be forecast accurately. Unfortunately, it is not intuitively clear how this forecast is distributed to the lower levels. There are different approaches here, for example, the forecast of the company's earnings based on the historical weights on the forecasts for the expense and the earnings and so on. The disadvantage of this approach is that any trends and seasonalities in the base time series may be ignored in the forecasts.
This approach is a hybrid of "bottom-up" and "top-down". Statistical methods are used to directly forecast the positions of a level that corresponds to neither the lowest nor the highest level. In our example, this would be the time series of the middle hierarchical level, the Effort and the Yield. The predictions of all items on the levels above, here the Company's earnings, would be determined using bottom-up. On the other hand, the forecasts of the time series on the levels below, here Personnel costs, Material costs, sales revenue and extraordinary income, according to a top-down approach.
Conclusion
The approaches presented are only the basic outlines of the range of methods available. There are numerous parameters that can be adjusted, such as the disaggregation method within the top-down approach. Users should first answer for themselves what they primarily want to achieve with the forecast. Do they deal with multi-layered hierarchical structures and the focus is mainly on a preview of the decisive business key figures? In this case, a top-down or middle-out approach, which starts at an upper level, is advisable. If, on the other hand, all current developments are to be mapped in order to be able to react to them as quickly and flexibly as possible? Then a bottom-up or middle-out approach is more advisable, which starts at a lower level. Overall, unfortunately, it is not possible to make a general statement about which method delivers the best results. It is necessary to first answer the above questions, then test the different options, evaluate them objectively and select the best alternative for the scenario at hand.
Would you like to know more?
Would you like to delve deeper into this topic? Then we would be happy to present the forecasting of hierarchical time series to you in person - very much in combination with details on other functions of the SAP Analytics Cloud. Simply get in touch with us!
