Thesis Format
Monograph
Degree
Doctor of Philosophy
Program
Electrical and Computer Engineering
Supervisor
Capretz, Miriam
Abstract
Deep neural networks (DNN) have achieved exceptional success in real-world applications. DNN architectures play a crucial role in their performance and are usually manually designed by experts in the field. Nevertheless, such a design process is labour-intensive and highly dependent on the researcher’s expertise. Neural architecture search (NAS) is an approach developed in the last few years for automating DNN architecture engineering. Evolutionary algorithms have shown high success and have received much attention among different strategies to perform NAS. Evolutionary algorithms are a class of population-based, stochastic search techniques inspired by biological evolution. These algorithms share the same types of evolution mechanics as genetics: mutation, fitness assignment, selection, and offspring creation.
This research proposes an algorithm for developing evolutionary neural architecture search (ENAS) for time series tasks. The algorithm focusses on neural architecture search during the evolution process by reducing the importance of the weights using a shared weight method. By applying shared weights, the ENAS retains the time series patterns for a designated task, such as load forecasting or anomaly detection. Later, in a separate phase, the DNN weights are adjusted individually, improving the model performance.
To show the algorithm’s performance, three frameworks were developed. The first and second frameworks were related to load forecasting tasks; alternatively, the third framework was developed for anomaly detection in IoT time series. The first framework, DNN centred on architecture evolution (DNN-CAE), presents ENAS for load forecasting for energy consumption in a single house. The second framework, similarity centred architecture evolution search (SCAES), combines ENAS with a technique similar to transfer learning to perform short load forecasting in a neighbourhood with multiple household load consumption. Finally, the third framework, ENAS for evolved centred cluster detector (ENAS-ECCD), presents an approach for supervised multi-label time-series anomaly detection. Through the experiments performed on each of the three frameworks, we demonstrated the flexibility of the ENAS algorithm to develop DNN for time series. The results suggested that the proposed algorithm can evolve a neural architecture for different time series tasks such as load forecasting and anomaly detection.
Summary for Lay Audience
In computer science and mathematics, algorithms are procedures used for solving a problem or performing a computation. Algorithms act as a precise list of instructions to solve a problem by conducting a step-by-step procedure. Neural networks are algorithms in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. In the last years, the area of neural networks has attracted the interest of new enthusiasts. This may be because of its flexibility to solve problems similar to humans. Among the challenges a neural network enthusiast has to face, there is one of higher importance: the neural network's design. It can take several days of trial and error for a neural network enthusiast to design a good model from scratch. The work presented in this thesis develops an algorithm to automatically design neural networks. The proposed algorithm uses evolutionary algorithms, which are inspired by nature and solve problems through processes that emulate the behaviour of living organisms. Thus, the proposed algorithm called evolutionary neural architecture search with shared weights was used to solve real-world problems using three different methods. The experimental results have shown that the proposed algorithm can automatically design neural networks for real-world applications.
Recommended Citation
Gomez Rosero, Santiago Vladimir, "Evolving Deep Neural Network Architectures for Time Series Data" (2022). Electronic Thesis and Dissertation Repository. 8555.
https://ir.lib.uwo.ca/etd/8555
Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 License.