Only about 6% of players actually win a prize from arcade claw games, where users maneuver a grasping metal claw to attempt to pick up a prize from a tightly packed pile. The game is purposely imprecise, with a weak claw, to encourage token spends while restricting prize rewards. It’s a low-stake situation, with only a few cents wasted if the player loses, but what about autonomous robotic pickers with more responsibilities, like organizing food stuffs or sorting through recycling? With higher stakes, do they have a higher success rate? 

Thanks to a new machine learning approach that enables robotic arm to separate out piles of objects, recognize the individual objects, and grasp the desired one, yes — they can correctly select and grasp the right object with 97% accuracy, on average. 

Paper Information

Yuxiang Yang, Zhihao Ni, Mingyu Gao, Jing Zhang and Dacheng Tao, "Collaborative Pushing and Grasping of Tightly Stacked Objects via Deep Reinforcement Learning," IEEE/CAA J. Autom. Sinica, vol. 9, no. 1, pp. 135-145, Jan. 2022. doi: 10.1109/JAS.2021.1004255

https://www.ieee-jas.net/en/article/doi/10.1109/JAS.2021.1004255

“By creating a unique collaborative pushing and grasping reinforcement learning network and reward functions, the robot can effectively grab and sort closely packed objects in real-world circumstances,” said paper author Jing Zhang, research fellow at the School of Computer Science , University of Sydney. “Industrial parts sorting and residential waste sorting are two examples of how the proposed technology could be used.” 

The research team taught a robotic arm to push their piles apart, separating the objects, before attempting to grasp any single one. Once the arm separated out the objects enough to recognize individual items, the implemented algorithm could reference a common household items database to identify which item should be grasped. 

The pushing is critical, according to Zhang. In simulations, just grasping led to a high accuracy rate of roughly 35%, while pushing and grasping had a high accuracy rate of 100%. That perfect score dipped slightly to slightly above 97%, depending on the number of objects in the pile. In comparison, the grasping-only approach dropped 10 percentage points when the pile size changed. 

In a real-world experiment, the robotic arm successfully separated out and grasped desired objects about 97% of the time. The 100% success rate was likely hampered by a lack of a pushing boundary, so the robotic arm could accidentally lose objects by pushing them too far away. 

The researchers plan to rectify this issue in future work and continue to refine their approach.