Intelligence is an extremely complicated concept, and understanding it with a view to creating artificial intelligence is one of the most important and interesting problems in science and technology today.
We take up the challenge of moving empirical state-of-the-art Deep Learning beyond the massive use of labelled data, without compromising the learning performance. The research is an interplay between research in Deep Learning from the point of view of application performance (engineering) and research from a view point of deduction and foundation (mathematics).
We focus specifically on four areas:
- Deep Reinforcement Learning (DRL), the learner iteratively chooses an action and in this way moves from one state to a new state. There is an associated reward to each action, and the overall goal for the learner is to maximise the cumulative reward without the need for target labels. Reinforcement learning was in 2016 combined with Deep Networks resulting in DRL with super-human performance on very complex problems. DRL will be used on problems such as automated driving and autonomous robotics, and probably many others.
- Generative Adversarial Networks (GANs) are systems consisting of one network generating new (fake) data, and another that tries to discriminate between fake and real data. The two networks are trained together without labelled target data. When trained, the discriminator is a classifier trained in an unsupervised manner. GAN is used to gain a deeper understanding of the DL’s hierarchical features and to problems such as 3D structure from a 2D image and generalising from small labelled datasets, and probably many others.
- Deep Learning provides hierarchical data representations, embedding data points in a high-dimensional space. It is conjectured that real-world data actually clustered around a submanifold (subset) of much lower dimension than the ambient high-dimensional space. The methods of estimating geometric and topological property of this submanifold is the method of Manifold Learning
- With access to a data source, more examples of similar data can be constructed to use as input for further analysis. This method is called Data Augmentation. Mathematically, this can be regarded as interpolation. Especially in mathematics in general and numerical analysis in particular, interpolation has been an important tool for many years and there is a large body of knowledge often relating to the “geometry” of the data. We investigate how it may be put to new use for the purpose of data augmentation to move Deep Learning beyond the massive use of labelled data.
We aim to develop new techniques to reduce the amount of labelled data traditionally needed in industrial supervised learning. We also aim to develop methods for learning from experience (DRL). This will enable reduction of domain knowledge and automate existing industrial processes.
