Evolving Deep Neural Network Architectures for Time Series Data
Abstract
Deep neural networks (DNN) have achieved exceptional success in real-world applications. DNN architectures play a crucial role in their performance and are usually manually designed by experts in the field. Nevertheless, such a design process is labour-intensive and highly dependent on the researcher’s expertise. Neural architecture search (NAS) is an approach developed in the last few years for automating DNN architecture engineering. Evolutionary algorithms have shown high success and have received much attention among different strategies to perform NAS. Evolutionary algorithms are a class of population-based, stochastic search techniques inspired by biological evolution. These algorithms share the same types of evolution mechanics as genetics: mutation, fitness assignment, selection, and offspring creation.
This research proposes an algorithm for developing evolutionary neural architecture search (ENAS) for time series tasks. The algorithm focusses on neural architecture search during the evolution process by reducing the importance of the weights using a shared weight method. By applying shared weights, the ENAS retains the time series patterns for a designated task, such as load forecasting or anomaly detection. Later, in a separate phase, the DNN weights are adjusted individually, improving the model performance.
To show the algorithm’s performance, three frameworks were developed. The first and second frameworks were related to load forecasting tasks; alternatively, the third framework was developed for anomaly detection in IoT time series. The first framework, DNN centred on architecture evolution (DNN-CAE), presents ENAS for load forecasting for energy consumption in a single house. The second framework, similarity centred architecture evolution search (SCAES), combines ENAS with a technique similar to transfer learning to perform short load forecasting in a neighbourhood with multiple household load consumption. Finally, the third framework, ENAS for evolved centred cluster detector (ENAS-ECCD), presents an approach for supervised multi-label time-series anomaly detection. Through the experiments performed on each of the three frameworks, we demonstrated the flexibility of the ENAS algorithm to develop DNN for time series. The results suggested that the proposed algorithm can evolve a neural architecture for different time series tasks such as load forecasting and anomaly detection.