Once described as alchemy, a quantitative science of machine learning is emerging. This talk will seek to unify the scientific approaches taken in machine learning, neuroscience, and physics. We will show how conceptual and mathematical tools in physics, such as symmetry, may illuminate the universal mechanisms behind learning and computation in biological and artificial neural networks. We plan to (i) generalize Noether’s theorem in physics to reveal how scale symmetry of the normalization makes learning more stable and efficient, (ii) dissect deep learning models of the retina by symmetry to explore how the eye predicts future events, and (iii) identify phase transitions in the loss landscapes of self-supervised learning as a potential cause of a failure mode called the dimensional collapse.