The emergence of machines that seem to offer the same or better capabilities than humans raised interests in many sectors who are eager to utilize them. Many may dream about building better and bigger machines that can do more amazing things. My hopes and dreams take different angles: communicating with these machines to learn from and with them. How are they solving complex problems? Can we learn something from them, and expand what we know? This talk shares a few first steps towards this goal: 1) we will share theoretical and empirical results that proves how the existing set of tools to learn about these machines require further examination. 2) We will share a few ways to study these machines: surgical, observational and controlled studies in reinforcement learning settings to discover emergent behaviors in multi-agent environments.