Forecasting Energy Consumption using Sequence to Sequence Attention models
Abstract
To combat negative environmental conditions, reduce operating costs, and identify energy savings opportunities, it is essential to efficiently manage energy consumption. Internet of Things (IoT) devices, including widely-used smart meters, have created possibilities for sensor based energy forecasting. Machine learning algorithms commonly used for energy forecasting, such as FeedForward Neural Networks, are not well-suited for interpreting the time dimensionality of a signal. Consequently, this thesis applies Sequence-to-Sequence (S2S) Recurrent Neural Networks (RNNs) with attention for electrical load forecasting. The S2S and S2S attention architectures commonly used for neural machine translation are adapted for energy forecasting. An RNN enables capturing time dependencies present in the load data, while the S2S RNN model strengthens consecutive sequence prediction by combining two RNNs: encoder and decoder. Adding the attention mechanism to these S2S RNNs alleviates the burden of connecting the encoder and decoder. Presented experiments compare a regular S2S model and four S2S attention models with two baseline models, the conventional Non-S2S RNN and a Deep Neural Network (DNN). Furthermore, each RNN model was evaluated with three different RNN-cells: Vanilla RNN, Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) cell. All models were trained and tested on one building-level electrical load dataset, with five-minute incremental data. Results showed that the S2S Bahdanau et al. attention model was the dominant model as it outperformed all other models for nearly all forecasting lengths.