I am currently a postdoctoral scholar at the UCLA Mobility Lab, working under the guidance of Prof. Jiaqi Ma. Previously, I was a research intern at the NVIDIA Research Autonomous Vehicle Research Group and a visiting student researcher at UC Berkeley in the Mechanical Systems Control (MSC) Lab. I earned my Ph.D. from Nanyang Technological University (NTU), where I conducted research in the Automated Driving and Human-Machine System (AutoMan) Lab under the supervision of Prof. Chen Lyu.
My research focuses on the intersection of robotics, mobility, and artificial intelligence (AI). I aim to develop algorithms and techniques that enable machines/robots to interact with humans naturally and make intelligent decisions. My research interests include deep learning, reinforcement learning, and generative AI, applied to areas such as perception, prediction, decision-making, simulation in autonomous driving, and human-machine interaction. My work has led to the publication of over 30 papers in top AI, ITS, and robotics journals and conferences.
π₯ News
- 2024.07: Β ππ Our ITSC invited session on Learning-powered and Knowledge-driven Autonomous Driving has received 11 paper submissions, all of which were accepted. Congratulations to all the authors! Looking forward to seeing you in Edmonton, Canada!
- 2024.06: Β ππ Our team secured first place in the Waymo Open Dataset Occupancy Flow Challenge and second place in the Sim Agents Challenge! Check out our technical reports on the Waymo challenge website and CVPR 2024 Workshop on Autonomous Driving.
- 2024.05: Β ππ Our paper on online belief prediction and POMDP planning has been accepted by RAL!
- 2024.01: Β ππ Our paper on joint prediction and planning for tree policy has been accepted by ICRA! See you in Yokohama, Japan!
- 2023.11: Β I was invited by zdjszx.com to give a public lecture on βScalable, Learnable, and Interactive Decision-making for Autonomous Drivingβ. The recorded version of the lecture (in Chinese) is available for viewing on bilibili.
- 2023.10: Β ππ Our paper on brain-inspired reinforcement learning for safe autonomous driving has been accepted by TPAMI!
- 2023.09: Β ππ We won the best paper runner-up award in ITSC 2023!
- 2023.09: Β ππ Our paper on human-guided reinforcement learning for robot navigation has been accepted by TPAMI!
- 2023.08: Β ππ Our GameFormer paper has been accepted by ICCV as Oral presentation!
- 2023.06: Β ππ Our team won the innovation award in the nuPlan Planning Challenge! Check out our report and presentation on our GameFormer Planner.
π Publications
Highlights
Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Driving
Zhiyu Huang, Chen Tang, Chen Lv, Masayoshi Tomizuka, Wei Zhan
IEEE Robotics and Automation Letters, 2024
- we propose an online belief-update-based behavior prediction model and an efficient planner for POMDPs. We develop a Transformer-based prediction model, enhanced with a recurrent neural memory model, to dynamically update latent belief state and infer the intentions of other agents.
DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving
Zhiyu Huang, Peter Karkus, Boris Ivanovic, Yuxiao Chen, Marco Pavone, Chen Lv
IEEE International Conference on Robotics and Automation (ICRA), 2024
Paper |
- We employ a tree-structured policy planner and propose a differentiable joint training framework for both ego-conditioned prediction and cost evaluation models, resulting in a direct improvement of the final planning performance.
Learning-enabled Decision-making for Autonomous Driving: Framework and Methodology
PhD Thesis, 2024
- This thesis presents a comprehensive framework and a series of learning-based methodologies for decision-making in AVs, with the objective of improving the scalability, adaptability, and alignment of their decision-making systems.
GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving
Zhiyu Huang, Haochen Liu, Chen Lv
IEEE/CVF International Conference on Computer Vision (ICCV), 2023
Oral presentation (top 3%)
Paper | Project | | GameFormer Planner
- We address the interaction prediction problem by formulating it with hierarchical game theory and implementing it with TransFormer networks.
Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving
Zhiyu Huang, Haochen Liu, Jingda Wu, Wenhui Huang, Chen Lv
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2023
Paper |
- We propose an interaction-aware motion prediction model that is able to predict other agentsβ future trajectories according to the ego agentβs future plans, i.e., their reactions to the egoβs actions.
Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving
Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv
IEEE Transactions on Intelligent Transportation Systems, 2023
- Distinguished from existing learning-based methods that directly output decisions, we introduce a predictive behavior planning framework that learns to predict and evaluate from human driving data.
Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving
Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv
IEEE Transactions on Neural Networks and Learning Systems, 2023
- We propose an end-to-end differentiable framework that integrates prediction and planning modules and is able to learn the cost function from data.
Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Driving
Zhiyu Huang, Xiaoyu Mo, Chen Lv
IEEE International Conference on Robotics and Automation (ICRA), 2022
- We propose a neural prediction framework based on the Transformer structure to model the relationship among the interacting agents and extract the attention of the target agent on the map waypoints.
Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
Zhiyu Huang, Jingda Wu, Chen Lv
IEEE Transactions on Neural Networks and Learning Systems, 2022
- We propose a novel framework to incorporate human prior knowledge in DRL, in order to improve the sample efficiency and save the effort of designing sophisticated reward functions.
Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning
Zhiyu Huang, Jingda Wu, Chen Lv
IEEE Transactions on Intelligent Transportation Systems, 2021
Paper |
- We propose a structural assumption about internal reward function-based human driving behavior and employ sampling-based maximum entropy inverse reinforcement learning (IRL) algorithm to infer the reward function parameters from naturalistic human driving data.
Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding
Zhiyu Huang, Chen Lv, Yang Xing, Jingda Wu
IEEE Sensors Journal, 2020
- We propose a novel deep neural network-based system for end-to-end autonomous driving, consisting of multimodal sensor fusion, scene understanding, and conditional driving policy modules.
All Publications
Journal
-
Safety-Aware Human-in-the-Loop Reinforcement Learning With Shared Control for Autonomous Driving, Wenhui Huang, Haochen Liu, Zhiyu Huang, Chen Lv, IEEE Transactions on Intelligent Transportation Systems
-
Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving, Haochen Liu, Zhiyu Huang, Xiaoyu Mo, Chen Lv, IEEE Transactions on Intelligent Vehicles, 2024
-
Transformer-Based Traffic-Aware Predictive Energy Management of a Fuel Cell Electric Vehicle, Jingda Wu, Zhiyu Huang, Chen Lv, IEEE Transactions on Vehicular Technology, 2024
-
Map-Adaptive Multimodal Trajectory Prediction via Intention-Aware Unimodal Trajectory Predictors, Xiaoyu Mo, Haochen Liu, Zhiyu Huang, Xiuxian Li, Chen Lv, IEEE Transactions on Intelligent Transportation Systems, 2023
-
Fear-Neuro-Inspired Reinforcement Learning for Safe Autonomous Driving, Xiangkun He, Jingda Wu, Zhiyu Huang, Zhongxu Hu, Jun Wang, Alberto Sangiovanni-Vincentelli, Chen Lv, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
-
Human-Guided Reinforcement Learning with Sim-to-Real Transfer for Autonomous Navigation, Jingda Wu, Yanxin Zhou, Haohan Yang, Zhiyu Huang, Chen Lv, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
-
Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving, Jingda Wu, Zhiyu Huang, Chen Lv, IEEE Transactions on Intelligent Vehicles, 2022
-
Prioritized Experience-based Reinforcement Learning With Human Guidance for Autonomous Driving, Jingda Wu, Zhiyu Huang, Wenhui Huang, Chen Lv, IEEE Transactions on Neural Networks and Learning Systems, 2022
-
Towards Human-in-the-loop AI: Enhancing Deep Reinforcement Learning via Real-time Human Guidance for Autonomous Driving, Jingda Wu, Zhiyu Huang, Zhongxu Hu, Chen Lv, Engineering, 2022
-
Multi-Agent Trajectory Prediction With Heterogeneous Edge-Enhanced Graph Attention Network, Xiaoyu Mo, Zhiyu Huang, Yang Xing, Chen Lv, IEEE Transactions on Intelligent Transportation Systems, 2022
-
Personalized Trajectory Planning and Control of Lane-Change Maneuvers for Autonomous Driving, Chao Huang, Hailong Huang, Peng Hang, Hongbo Gao, Jingda Wu, Zhiyu Huang, Chen Lv, IEEE Transactions on Vehicular Technology, 2021
Conference
-
Occupancy Prediction-Guided Neural Planner for Autonomous Driving, Haochen Liu, Zhiyu Huang, Chen Lv, IEEE International Conference on Intelligent Transportation Systems (ITSC), 2023
-
Multi-modal Hierarchical Transformer for Occupancy Flow Field Prediction in Autonomous Driving, Haochen Liu, Zhiyu Huang, Chen Lv, IEEE International Conference on Robotics and Automation (ICRA), 2023
-
Stochastic Multimodal Interaction Prediction for Urban Driving, Xiaoyu Mo, Zhiyu Huang, Chen Lv, IEEE International Conference on Intelligent Transportation Systems (ITSC), 2022
-
ReCoAt: A Deep Learning-based Framework for Multi-Modal Motion Prediction in Autonomous Driving Application, Zhiyu Huang, Xiaoyu Mo, Chen Lv, IEEE International Conference on Intelligent Transportation Systems (ITSC), 2022
-
Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving, Haochen Liu, Zhiyu Huang, Jingda Wu, Chen Lv, IEEE Intelligent Vehicles Symposium (IV), 2022
-
Digital Twin-enabled Reinforcement Learning for End-to-end Autonomous Driving, Jingda Wu, Zhiyu Huang, Peng Hang, Chao Huang, Niels De Boer, Chen Lv, IEEE International Conference on Digital Twins and Parallel Intelligence (DTPI), 2021
-
Multi-scale driver behaviors reasoning system for intelligent vehicles based on a joint deep learning framework, Yang Xing, Zhongxu Hu, Zhiyu Huang, Chen Lv, Dongpu Cao, Efstathios Velenis, IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020
Preprint
-
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking, Daniel Dauner, Marcel Hallgarten, Tianyu Li, Xinshuo Weng, Zhiyu Huang, Zetong Yang, Hongyang Li, Igor Gilitschenski, Boris Ivanovic, Marco Pavone, Andreas Geiger, Kashyap Chitta, arXiv, 2024
-
Versatile Scene-Consistent Traffic Scenario Generation as Optimization with Diffusion, Zhiyu Huang, Zixu Zhang, Ameya Vaidya, Yuxiao Chen, Chen Lv, Jaime FernΓ‘ndez Fisac, arXiv, 2024
-
Hybrid-Prediction Integrated Planning for Autonomous Driving, Haochen Liu, Zhiyu Huang, Wenhui Huang, Haohan Yang, Xiaoyu Mo, Chen Lv, arXiv, 2024
π Honors and Awards
- 2024.06 1st Place Winner, Waymo Open Dataset Occupancy Flow Challenge, CVPR Workshop on Autonomous Driving
- 2024.06 2rd Place Winner, Waymo Open Dataset Sim Agents Challenge, CVPR Workshop on Autonomous Driving
- 2023.09 Best Paper Runner-up Award, ITSC 2023
- 2023.06 Innovation Award, nuPlan Planning Challenge, CVPR Workshop on End-to-End Autonomous Driving | [video]
- 2023.06 3rd Place Winner, Waymo Open Dataset Motion Prediction Challenge, CVPR Workshop on Autonomous Driving
- 2022.12 3rd Place Winner, Most Innovative Award, Driving SMARTS Competition, NeurIPS Competition Track | [slides]
- 2022.06 2nd Place Winner, Waymo Open Dataset Occupancy and Flow Prediction Challenge, CVPR Workshop on Autonomous Driving
- 2022.03 2nd Place Winner, IEEE VTS Motor Vehicles Challenge, VPPC
- 2021.06 1st Place Winner, Waymo Open Dataset Interaction Prediction Challenge, CVPR Workshop on Autonomous Driving | [video]
- 2021.06 2nd Place Winner, Waymo Open Dataset Motion Prediction Challenge, CVPR Workshop on Autonomous Driving
- 2019.06 Outstanding Graduate (Top 1%)
- 2018.10 National Scholarship (Top 1%)
- 2017.10 National Scholarship (Top 1%)
π Education
- 2019 - 2024, Doctor of Philosophy, Robotics and Intelligent Systems, Nanyang Technological University, Singapore
- 2015 - 2019, Bachelor of Engineering, Vehicle Engineering, Chongqing University, Chongqing, China
π Academic Services
Program Committee
- Lead organizer of Special Session on learning-powered prediction and decision-making at ITSC, 2023
- Lead organizer of Invited Session on learning-powered and knowledge-driven autonomous driving at ITSC, 2024
Journal Reviewer
- IEEE Transactions on Intelligent Transportation Systems
- IEEE Transactions on Neural Networks and Learning Systems
- IEEE Transactions on Intelligent Vehicles
- IEEE Transactions on Cybernetics
- IEEE Robotics and Automation Letters
- Transportation Research Part C: Emerging Technologies
- Engineering Applications of Artificial Intelligence
- Artificial Intelligence Review
Conference Reviewer
- IEEE International Conference on Robotics and Automation (ICRA) 2022 β 2024
- IEEE Intelligent Vehicles Symposium (IV) 2022 β 2024
- IEEE Intelligent Transportation Systems Conference (ITSC) 2022 β 2024
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2023
- European Conference on Computer Vision (ECCV) 2024