The world of robotics and artificial intelligence is constantly pushing the boundaries of what machines can achieve, and the latest project from Hudson Nock and their team is a testament to that. They've taken on the challenging task of teaching a robot to play air hockey, and the results are impressive. What sets this project apart is the meticulous attention to detail and the comprehensive documentation provided by the team.
The project's technical reports and README file offer a rare glimpse into the inner workings of a sim-to-real project. The team's approach involves training their agent entirely inside a simulator, which is then loaded onto physical hardware. This zero-shot transfer is a significant achievement, especially in a fast-paced and chaotic game like air hockey.
One of the key challenges in this project is the gap between a tidy model and a messy world. The team had to account for various factors, such as the wooden table's uneven surface, the varying bounce of the rails, and the impact of hard accelerations on the power supply voltage. They measured and modeled these factors as much as possible, randomizing the rest in the simulator to ensure the agent doesn't rely on specific conditions.
The modeling process is fascinating. The mallet's reaction to motor voltage is described by a third-order transfer function, controlled by a feedforward and PID controller that follows a target path with remarkable precision. The puck's gliding motion is governed by a simple nonlinear differential equation. However, collisions proved more complex. The team had to train a tiny neural network with just 112 parameters to predict both the outcome of collisions and the associated uncertainty, ensuring the agent experiences realistic and unpredictable bounces.
Sensing is achieved through a single camera. The puck is equipped with retroreflective tape and illuminated by a bright LED array, ensuring it appears as a crisp dot even at short exposure times. A custom calibration process addresses the table's warping and pins position error to around a millimeter across the entire surface. When the puck is partially blocked by the gantry, a contour-based tracker keeps it in view, and the camera also locates the opponent's mallet at 120 frames per second using a hollow retroreflective marker.
The simulator is a crucial component of this project. The team developed their own physics engine from scratch, utilizing analytical solutions for both the mallet and puck's equations of motion. This approach eliminates slow numerical integration and employs an adaptive timestep for collision detection, allowing for big steps during quiet moments without skipping hits. The vectorized code enables the simulation to run thousands of matches simultaneously, achieving a throughput of approximately 230 times real-time on an Intel i5 laptop, making agent training feasible.
The learning process utilizes Soft Actor-Critic with a network of around 200,000 parameters. To prevent the agent from specializing in a single opponent type, it trains against a diverse range of opponents, including a defensive agent, a hand-coded blocker, and earlier versions of itself. This approach, combined with domain randomization tuned to measured hardware noise, ensures the policy remains unpredictable and adaptable.
The project's GitHub page (https://github.com/HudsonNock/Air-Hockey-Sim) provides access to all the code, reports, and system diagrams. The video embedded in the article showcases the robot's impressive skills in playing air hockey against a human opponent. This project is a testament to the team's expertise and dedication, offering valuable insights into the field of robotics and machine learning.
