, Available online , doi: 10.1109/JAS.2026.126437
Abstract:
Autonomous parking in diverse and constrained environments remains a key challenge for intelligent vehicles. Existing learning-based approaches often suffer from poor generalizability and rely heavily on handcrafted, distance-driven rewards, leading to inefficient and unnatural behaviors in complex spaces. This work proposes a Reeds-Shepp (RS) cost map-guided dual-critic reinforcement learning (RL) framework to address these issues. First, an attention-based learnable RS cost map is constructed to encode environment-dependent pose feasibility for motion planning. Second, to improve the effectiveness of the RL method, a dual-critic architecture is introduced, decoupling Euclidean distance-based rewards from heuristic-guided feasibility-aware evaluations. The distance-based reward facilitates efficient task completion, while the heuristic-guided reward incorporates global maneuverability cost, guiding the agent to select feasible yet potentially non-intuitive maneuvers. Furthermore, a curriculum learning strategy is employed to accelerate convergence and enhance parking precision, particularly in confined parallel slots. Extensive experiments across multiple parking slot types, including parallel, vertical, and slant, demonstrate that the proposed method achieves a 97.3% average success rate, outperforming the baselines by up to 24.5%. Notably, for the most challenging parallel parking scenario, our method achieves a 92.9% success rate, which still represents an 8.3% improvement over the best baseline. The results in both simulation and real-vehicle experiments highlight the superior generalizability, safety, and convergence efficiency of the proposed method.
S. Yang, M. Ruan, Y. Yang, T. Zhang, and M. Fu, “Learnable reeds-shepp cost map-guided dual-critic reinforcement learning for generalizable autonomous parking,” IEEE/CAA J. Autom. Sinica, early access, 2026. doi: 10.1109/JAS.2026.126437.