From Hypothesis to Reality: Designing a Superhuman Racing AI Agent Using a Deep Reinforcement Learning

This work started out as a grand challenge to create an AI agent that could beat the world’s best Gran Turismo (GT) drivers. In order to develop an agent capable of competing against the world’s best drivers, GT Sophy was trained to master the following driving skills: race car control, racing tactics, and racing etiquette. This talk will tell you a story about the technical evolution of GT Sophy from a research outcome to an in-game feature that could be introduced as part of the Gran Turismo 7 (GT7) PlayStation racing game. Sony AI, in collaboration with Polyphony Digital and Sony Interactive Entertainment, designed novel deep reinforcement learning approaches with unique training and evaluation methods on a modern cloud computing platform to accommodate this project. In less than two years since GT Sophy appeared on the cover of Nature, the breakthrough AI agent has now become a permanent in-game feature of GT7.

Biography

Alisa Devlic is a Senior Research Scientist at Sony AI, working on applying and advancing the state-of-the-art in deep reinforcement learning in the gaming domain. She obtained her Ph.D. in Communication Systems from KTH, Stockholm, Sweden and did her postdoctoral work at IMT Atlantique, Rennes, France. Before joining Sony AI, she worked both in industry labs (Ericsson Research and Huawei Technologies) and academia (KTH, IMT Atlantique, and University of Zagreb). Alisa (co)authored more than 30 papers and obtained best paper awards at IEEE WoWMoM 2015, ICC 2017 and MMSP 2018. She was part of the Sony AI team who developed a GT Sophy, the first racing agent that can outperform the world’s best e-sports drivers and published results in the Nature journal. She started working in the field of Reinforcement Learning in 2020, when she joined Sony AI.