Deep learning (the training of deep neural nets) is a very simple idea. Yet it has led to many striking applications throughout science and industry over the last decade. In mathematics the impact has so-far been modest. I will discuss a few instances where it has proved useful, and led to interesting (pure) mathematics. I will also reflect on my experience as a pure mathematician interacting with deep learning. I will also discuss what can be learned from the successful examples that I understand, and try to guess an answer to the question in the title.

(Deep learning also raises interesting mathematical questions, but this talk won't be about this.)