MINDS CDT doctoral researcher | University of Southampton
High-performance reinforcement learning on a desktop PC
Formal project title: Sample-efficient exploration in visually complex reinforcement learning tasks
Tyler Clark is demonstrating that high-performance deep reinforcement learning can run on standard desktop computing. By reducing reliance on expensive large-scale infrastructure, his research could make this area of AI more accessible to researchers and developers with limited computing resources.
Profiles: LinkedIn | Google Scholar | GitHub | AI Tango on YouTube
Research at a glance
| Research focus | Deep reinforcement learning on standard desktop computing |
| MINDS theme | Embedded Artificial Intelligence |
| Methods | Deep reinforcement learning, value-based agents, game-based test environments |
| Wider applications | Robotics, game-playing and other learning-by-doing tasks |
| Featured output | ICML 2025 conference paper |
| MINDS cohort | 2022 |
| Current status | Doctoral researcher |
Reinforcement learning without a data centre
Deep reinforcement learning is a branch of AI in which a computer program, known as an agent, learns through trial and error. It tries an action, receives feedback on how well that action worked and gradually improves. The approach is used in applications ranging from robotics to game-playing.
Achieving the strongest results has typically required large amounts of computing power, often more than an individual researcher or small team can access. That has made leading-edge work in the field difficult to reproduce or build on without specialist data-centre infrastructure.
Tyler’s research asks how much performance can be achieved within the limits of a standard desktop PC. In his ICML 2025 paper, Tyler and his co-authors present Beyond The Rainbow, an agent that brings together several established improvements from across reinforcement learning research in a single, efficient design. The paper reports state-of-the-art results for reinforcement learning on a desktop PC across a widely used benchmark of Atari games, with training on that benchmark taking around 12 hours on a high-end desktop.
The same approach was also applied, with minimal changes, to more complex 3D games, including Super Mario Galaxy, Mario Kart and Mortal Kombat.
Why the research matters
Lowering barriers to experimentation
Reducing dependence on large-scale computing infrastructure could make reinforcement learning research more accessible to researchers, developers and smaller organisations.
Faster research cycles
Shorter training times on accessible hardware could make it quicker to test ideas, compare methods and build on published work.
Cost and energy implications
Training on a single desktop rather than large computing clusters has potential cost and energy implications, although the effect in any setting depends on the task and the hardware involved.
Research approach
Tyler’s work builds on Rainbow DQN, a well-established reinforcement learning method that itself combined several earlier advances. Beyond The Rainbow adds six further improvements drawn from across the field, with the aim of achieving strong performance while keeping computational demands within the reach of a desktop PC. Each component was tested through ablation studies, in which parts of the method are removed in turn to show how much each one contributes.
Video games provide a useful test environment because they offer complex, varied challenges with clear measures of success. Tyler’s wider doctoral project explores sample-efficient reinforcement learning: agents that learn from fewer interactions with their environment. This matters wherever those interactions are slow, costly or impractical, such as training in the real world.
Research and outputs
Featured output: ICML 2025 conference paper
Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC, by Tyler Clark, Mark Towers, Christine Evers and Jonathon Hare, was published in the Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), a leading international machine learning conference.
Read the Beyond The Rainbow paper in the ICML 2025 proceedings
Explore the Beyond The Rainbow code on GitHub
Research profiles
Sharing research through AI Tango
Alongside his doctoral research, Tyler shares research ideas and technical work with a wider audience through AI Tango, his YouTube channel. AI Tango is Tyler’s own channel and is not an official MINDS or University of Southampton channel.
Related story
Tyler presented his desktop reinforcement learning research at the 2026 MINDS and SustAI Showcase. Read the Showcase story and watch the accompanying research film.
Connect with Tyler
Interested in efficient reinforcement learning, accessible AI computing or game-based AI research? Connect with Tyler through LinkedIn, explore his AI Tango channel or contact the MINDS team about a wider research or collaboration enquiry.
Not sure whether this is the right research area? The MINDS team can help direct your enquiry.