Abstract: Deep neural networks have achieved significant empirical success in many fields, including the fields of computer vision and natural language processing. Along with its empirical success, deep learning has been theoretically shown to be attractive in terms of its expressive power. However, the theory of expressive power does not ensure that we can efficiently find an optimal solution in terms of optimization and generalization, during the optimization process. In this talk, I will discuss some mathematical properties of optimization and generalization for deep neural networks.