Luận văn Thạc sĩ: Dự báo chuỗi thời gian phân cấp kinh doanh bằng mô hình LSTM

The University of Southampton 2019/20 Faculty of Social Sciences Southampton Business School MSc Dissertation Forecast for business hierarchical time series with LSTM model ERGO number: 58984 Student number: …. Presented for MSc. Business Analytics and Management Sciences This project is entirely the original work of student registration number …. I declare that this dissertation is my own work, and that where material is obtained from published or unpublished works, this has been fully acknowledged in the references.

This dissertation may include material of my own work from a research proposal that has been previously submitted for assessment for this programme. Word Count: 13613 words ABSTRACT Forecasting is a popular business analysis tool. Hierarchical forecasting in business is a well-known advanced forecasting method. It has been researched to improve the reconciliation approach.

However, seldom do practitioners and academicians consider improving methods for the base forecast. The application of neural network models, generating the base forecast for hierarchical forecasting, are not of practitioners’ concern. However, with what this method has done in improving the forecasting accuracy for univariate business time series; neural network method is expected to improve the hierarchical forecasting accuracy. The specific neural network methods used in this dissertation are Long-Short Term Memory (LSTM) and its extension – LSTM Encoder-Decoder.

LSTM is an enhancement of recurrent neural network, which can identify, update and carry series pattern for long sequence. It has proved in capability in forecasting, especially in energy demand and production. When being applied to the Australian tourism demand dataset under this dissertation, LSTM models do not bring satisfactory accuracy. Although models’ accuracy is not as good as expected, in comparison with classical methods, including Holt-Winters’ and SARIMA, the deviations of value among models are not significant.

LSTM models still reflect correct trends in series and helpful to some extent of business analysis and business decisions, especially under the current business situation of COVID-19 pandemic. i TABLE OF CONTENTS Chapter 1 Introduction.3 Objective and Aim.5 Chapter 2 Literature review .1 Business forecasting for univariate time series .2 The hierarchical structure of business time series .2 Hierarchical forecasting for business time series .1 Hierarchical forecasting approach .1 Bottom-up approach .2 Top-down approach .3 Middle-out approach .4 Optimal reconciliation approach .2 LSTM forecasting method for business hierarchical time series…. 16 Chapter 3 Data and Methodology .2 Time plot for identifying series patterns .1 The bottom-up approach for forecasting hierarchical time series… .2 Hierarchy forecasting with base forecasting at the bottom level by Holt-Winters’ and SARIMA model .1 Hierarchy forecasting with base forecasting at the bottom level by Holt-Winters’ model .2 Hierarchy forecasting with base forecasting at the bottom level by the SARIMA model .3 Hierarchy forecasting with base forecasting at the bottom level by LSTM models .2 LSTM model for generating base forecasting in hierarchical forecasting. 42 Chapter 4 Results and analysis.1 Forecasting results of base forecast at bottom-level series (univariate series) .2 Forecasting result of higher level in the hierarchical structure.

64 iii Appendix 1: Holt-Winters’ model results by Python’s hts-package. 70 Appendix 2: Time plot of original values and predicted values of all series in the hierarchical structure of Australian tourism. 73 Appendix 3: Classical models’ code. 82 Appendix 4: LSTM models’ code.

84 iv TABLE OF FIGURES Figure 1.Hierarchical time series. Grouped time series. Aggregate retail sale. Artificial neural network.

Recurrent neural network. LSTM ED network. The hierarchical structure of Australian quarterly visitor nights dataset. Time plot of zones in NSW.

Time plot of zones in QLD. Time plot of zones in SAU. Time plot of zones in VIC. Time plot of zones in WAU.

Time plot of zones in OTH. Time plot of original value and predicted values of NSW_SthCo. Time plot of original value and predicted values of SAU_Coast. Time plot of original value and predicted values of SAU_Inner.

Time plot of original value and predicted values of WAU_Metro. Time plot of original value and predicted values of WAU_Inner. Time plot of original value and predicted values of NSW_NthIn. Time plot of original value and predicted values at the top level.

Time plot of original value and predicted values of NSW. Time plot of original value and predicted values of WAU. Australian states and zones. Forecasting error of 4 methods in bottom level series.

Forecasting error of 4 methods for higher-level series. 50 vi Introduction 31340067 CHAPTER 1 INTRODUCTION 1.1 Background Forecasting hierarchical time series of hierarchical forecasting is an advanced forecasting method for a set of time series which can be modelled in a hierarchical structure (Karmy and Maldonado, 2019). There are two types of hierarchical structures that can be conducted hierarchical forecasting on, which are hierarchical and grouped structure. The difference between them is that hierarchical structure is disaggregated by one attribute, such as by geographical factor.

Meanwhile, the grouped structure can be disaggregated by two more attributes. These attributes are nested and crossed. For example, the total sale of sweaters by company A is disaggregated by sale in the North and the South of the country or by two colours, black and white in the first hierarchical level. In the second level, two attributes are crossed and nested by the sale of white in the North, sale of white in the South, sale of black in the North and sale of black in the South.Hierarchical time series Page 1 Introduction 31340067 Total X Y AX AY BX BY Total A B AX AY BX BY Figure 1.

Grouped time series The hierarchical structure of time series appears in many fields, especially in business. In business, time series can be disaggregated by various attributes of concern (Hyndman and Athanasopoulos, 2018), such as location, category, the purpose of uses or distributor. As “a centralised forecast approach” with ability to satisfy many kinds of information requirements, hierarchical forecasting can provide support for users at different levels of business management, enhance forecast accuracy and reduce the magnitude of forecast problem (Fliedner, 2001). Hierarchical forecasting has four approaches to generate coherent forecasting for all time series in the hierarchical structure, which are bottom-up, top-down, middle-out and optimal reconciliation approach.

Page 2 Introduction 31340067 Those approaches will be explained in detail in Chapter 2: Literature Review. However, regardless of which approach is applied, the base forecast is always required. The base forecast can be conducted on top- level, bottom-level or even a middle-level series following the chosen approach. Practitioners, who desire to enhance the accuracy of hierarchical forecasting can pursuit either advanced reconciliation method or base forecast methods.

Recent papers in hierarchical business forecasting show that practitioners focus more on developing reconciliation method. They still apply traditional methods, such as exponential smoothing (ETS) and Autoregressive Integrated Moving Average (ARIMA) as the base forecast method (Spiliotis et al., 2020a; Mancuso, Piccialli and Sudoso, 2020; Karmy and Maldonado, 2019; Huber, Gossmann and Stuckenschmidt, 2017). These methods and its extensions like Holt model, Holt-Winters’ model, seasonal ARIMA (SARIMA) and Seasonal ARIMA with eXogenous factor (ARIMA) are highly developed and straightforward to implement in the programming software. They are even developed for auto-fit in programming package for hierarchical forecasting called hts- package in both R and Python.

Advanced base forecast methods are not yet of interest and concern of practitioner in business when researching hierarchical forecasting. Besides aforementioned traditional methods, other base forecast methods, including regression and neural network, mostly artificial neural network (ANN), are available but they are mainly conducted on business univariate time series, not on time series in the hierarchical structure. Many previous papers have a conclusion that a complex model which can identify and model the nonlinear relationship between business forecasting and other business factors as well as identify particular patterns in the series brings Page 3 Introduction 31340067 higher forecasting accuracy for univariate business time series (Alon et al., 2001, Chu and Zhang, 2003; Zhang and Qi, 2005; Ahmed, Gayar & Shishiny, 2010; Lu, Lee and Lian, 2012; Ma, Fildes and Huang, 2016). One recent paper that focused on a nonlinear method for the base forecast in hierarchical forecasting of Karmy and Maldonado (2019) introduced a method called “Support Vector Regression” for hierarchical time series of European Travel Retail industry and used traditional methods as the benchmark to compare with their newly developed model.

Although both regression and neural network method have proved to be useful in forecasting for univariate business time series, seldom did practitioner come up with the idea of improving the base forecast in hierarchical forecasting by neural network methods. In many cases, NN is proved to be better and more flexible because it is a data-driven method, which requires little assumption about the models but still “simulates a wide variety of underlying nonlinear behaviours” (Chu and Zhang, 2003). Therefore, it requires little assumptions of model and relationship to be made before applying (Zhang and Qi, 2005). A simple “feedforward neural network, with only one hidden layer”, is developed and implemented easily by function nnetar in R programming and Tensorflow platform in Python for more complex neural networks are highly developed and updated to facilitate and leverage the accuracy of forecasting using neural network.2 Proposed Methodology Realising the potential of the neural network model in business forecasting, I would like to introduce another kind of NN, which is called Long-short term memory (LSTM) instead of continuing to employ familiar ANN models for business hierarchical time series in this dissertation.

LSTM is deep NN, which uses and learns from observations in time series to forecast single Page 4 Introduction 31340067 or multi-step ahead. It can identify, update and carry unique patterns in the time series for a long sequence. It is better than ANN in terms of identifying data characteristics while still maintains the forecasting accuracy and solution complexity (Siami-Namini, Tavakoli and Siami Namin, 2018).3 Objective and Aim The objective of this dissertation is to build LSTM model and one of its direct extension, LSTM Encoder-Decoder model, on the Tensorflow platform in Python programming and generate the base forecast for hierarchical forecasting for a business time series to see whether LSTM model can improve the accuracy of forecasting business hierarchical time series. If the final results, in comparison with traditional methods, including Holt-Winters’ and ARIMA, is better, LSTM model is a significant development in hierarchical business forecasting.4 Dissertation Structure This dissertation has six chapters.

Chapter 1: Introduction delivers an overview of the topic, rationale, objectives and outline of this dissertation. Chapter 2: Literature Review provides principals and theory of business forecasting, hierarchical business forecasting to clarify why hierarchical forecasting is appropriate and helpful for business. LSTM forecasting method for hierarchical time series as well as limitation of current methods and previous papers are provided to highlight the novelty of the dissertation. This chapter is the baseline for models developing in Chapter 3: Data and Methodology.

In Chapter 3, all necessary inputs for establishing LSTM model for hierarchical forecasting will be prepared and explained in detail, including data source and data preliminary analysis, reconciliation approach, LSTM models establishing process based on what were required and clarified in Chapter 2. The error measurement method Page 5 Introduction 31340067 is also defined in this chapter. After being established, LSTM model is able to generate predicted value for all time series in the hierarchical structure. The result of Hierarchical forecasting with LSTM model is presented and analysed in Chap 4: Results and Analysis.

Further comparison among results of hierarchical forecasting with LSTM model and results of hierarchical forecasting with traditional models (Holt-Winters’ and ARIMA) are provided in this chapter to support in-depth analysis. The last chapter is Chapter 5: Discussion, results of hierarchical forecasting with LSTM models will be discussed and compared with previous research as well as practical implications of LSTM models.