Network Architecture and Optimization: Methods of Dynamical Systems in Machine Learning(Xiao Wang)


The recent development of machine learning has provided new and challenging research problems in the areas of non-convex optimization and dynamical systems. We focus on two fundamental aspects of machine learning: training the network parameters and selecting the architecture of a neural network, where the techniques of differentiable and topological dynamical systems play an essential role. In the first part of the talk, we present a result that the meta first-order methods (gradient descent, mirror descent, manifold gradient descent, etc) avoid saddle points by introducing the global stability  theory of dynamical systems. In the second part, we present a new result on the expressivity power of deep neural networks: the depth-width trade-offs for ReLU networks via Sharkovskii’s Theorem. The related research has a variety of applications in deep learning, approximation theory, partial differential equations, etc, and has connections to Hilbert’s 13th problem. As an open ended area in representation learning, there are a lot more open problems to be answered in the future.


2020-09-08   10:00 ~ 11:00   


Xiao Wang, Singapore University of Technology and Design


Zoom ID: 91389575421; PW:123456