Zhiyu Huang

Zhiyu Huang

University of California, Los Angeles

I am currently a postdoctoral scholar at the UCLA Mobility Lab. Starting in Fall 2026, I will join North Carolina State University (NCSU) as an Assistant Professor, where I will launch the PARIS Lab (Physical AI, Robotics, and Intelligent Systems Lab). Previously, I was a research intern at the NVIDIA Research Autonomous Vehicle Group and a visiting student researcher at UC Berkeley in the Mechanical Systems Control (MSC) Lab. I earned my Ph.D. from Nanyang Technological University (NTU), where I conducted research in the Automated Driving and Human-Machine System (AutoMan) Lab.

My research lies at the intersection of intelligent systems, robotics, and physical AI. I develop generalizable algorithms and systems that enable physical intelligent agents to perceive and understand the world, interact seamlessly with humans and complex environments, reason and make decisions, and execute dexterous actions in the physical world. My work broadly spans machine learning, including deep learning, reinforcement learning, and generative AI, with applications in perception, world modeling, decision-making, control, and simulation for robotics, embodied intelligence, and autonomous systems. To date, I have authored over 40 papers published in leading journals and conferences across AI, robotics, and intelligent systems.

🔥 News

  • 2026.06:  🎉🎉 We will organize a workshop on Foundation Models for V2X-Based Cooperative Autonomous Driving at CVPR 2026. See you in Denver, Colorado!
  • 2026.05:  🎉🎉 Two papers on efficient VLA and 3D scene reconstruction have been accepted to ICML 2026!
  • 2026.03:  🎉🎉 Our paper on masked generation for traffic scenarios has been accepted to CVPR 2026 (Findings)!
  • 2025.11:  🎉🎉 Our paper on regulation-aware decision-making for autonomous driving with LLM has been accepted by AAAI 2026!
  • 2025.10:  🎉🎉 We organized a tutorial on Beyond Self-Driving: Exploring Three Levels of Driving Automation at ICCV 2025.
  • 2025.09:  🎉🎉 I was listed on Stanford/Elsevier Top 2% Scientists list (Artificial Intelligence & Image Processing, Single-year impact)
  • 2025.09:  🎉🎉 Our paper on VLA model for end-to-end autonomus driving has been accepted by NeurIPS 2025!
  • 2025.06:  🎉🎉 Two papers on multi-agent cooperative perception and model training have been accepted for publication at ICCV 2025!
  • 2025.05:  🎉🎉 Honored to receive the NTU MAE Best PhD Thesis Award. Grateful for the recognition and support!
  • 2025.01:  🎉🎉 Our paper on generative driving policy and reinforcement learning fine-tuning has been accepted by ICRA 2025!
  • 2024.12:  🎉🎉 Our paper on hybrid prediction integrated planning for autonomous driving has been accepted by TPAMI!
  • 2024.09:  🎉🎉 Our paper on end-to-end driving benchmark has been accepted at NeurIPS 2024!

📝 Publications

Highlights

sym

TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments

Zhiyu Huang, Yun Zhang, Johnson Liu, Rui Song, Chen Tang, Jiaqi Ma

International Conference on Machine Learning (ICML), 2026

Paper | Project |

  • We introduce Think-in-Control (TIC)-VLA, a latency-aware framework that explicitly models delayed semantic reasoning during action generation.
sym

MDG: Masked Denoising Generation for Multi-Agent Behavior Modeling in Traffic Environments

Zhiyu Huang, Zewei Zhou, Tianhui Cai, Yun Zhang, Jiaqi Ma

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, 2026

Paper

  • We propose Masked Denoising Generation (MDG), a unified generative framework that reformulates multi-agent behavior modeling as the reconstruction of independently noised spatiotemporal tensors.
sym

AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang*, Bolei Zhou, Jiaqi Ma

Neural Information Processing Systems (NeurIPS), 2025

Paper | Project |

  • We propose AutoVLA, a novel VLA model that unifies reasoning and action generation within a single autoregressive generation model for end-to-end autonomous driving.
sym

Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning

Zhiyu Huang, Xinshuo Weng, Maximilian Igl, Yuxiao Chen, Yulong Cao, Boris Ivanovic, Marco Pavone, Chen Lv

IEEE International Conference on Robotics and Automation (ICRA), 2025

Paper | Project

  • We introduce the Gen-Drive framework, which shifts from the traditional prediction and deterministic planning framework to a generation-then-evaluation planning paradigm.
sym

Versatile Behavior Diffusion for Generalized Traffic Simulation

Zhiyu Huang, Zixu Zhang, Ameya Vaidya, Yuxiao Chen, Chen Lv, Jaime Fernández Fisac

IEEE Transactions on Intelligent Transportation Systems, 2025

Paper | Project |

  • We propose VBD, a novel traffic scenario generation framework that utilizes diffusion generative models to predict scene-consistent and controllable multi-agent interactions in closed-loop settings.
sym

Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Driving

Zhiyu Huang, Chen Tang, Chen Lv, Masayoshi Tomizuka, Wei Zhan

IEEE Robotics and Automation Letters, 2024

Paper

  • We propose an online belief-update-based behavior prediction model and an efficient planner for POMDPs. We develop a Transformer-based prediction model, enhanced with a recurrent neural memory model, to dynamically update latent belief state and infer the intentions of other agents.
sym

DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving

Zhiyu Huang, Peter Karkus, Boris Ivanovic, Yuxiao Chen, Marco Pavone, Chen Lv

IEEE International Conference on Robotics and Automation (ICRA), 2024

Paper |

  • We employ a tree-structured policy planner and propose a differentiable joint training framework for both ego-conditioned prediction and cost evaluation models, resulting in a direct improvement of the final planning performance.
sym

Learning-enabled Decision-making for Autonomous Driving: Framework and Methodology

PhD Thesis, 2024

Thesis

  • This thesis presents a comprehensive framework and a series of learning-based methodologies for decision-making in AVs, with the objective of improving the scalability, adaptability, and alignment of their decision-making systems.
sym

GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving

Zhiyu Huang, Haochen Liu, Chen Lv

IEEE/CVF International Conference on Computer Vision (ICCV), 2023

Oral presentation (top 3%)

Paper | Project | | GameFormer Planner

  • We address the interaction prediction problem by formulating it with hierarchical game theory and implementing it with TransFormer networks.
sym

Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving

Zhiyu Huang, Haochen Liu, Jingda Wu, Wenhui Huang, Chen Lv

IEEE International Conference on Intelligent Transportation Systems (ITSC), 2023

Best Paper Runner-up Award

Paper |

  • We propose an interaction-aware motion prediction model that is able to predict other agents’ future trajectories according to the ego agent’s future plans, i.e., their reactions to the ego’s actions.
sym

Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv

IEEE Transactions on Intelligent Transportation Systems, 2023

Paper

  • Distinguished from existing learning-based methods that directly output decisions, we introduce a predictive behavior planning framework that learns to predict and evaluate from human driving data.
sym

Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving

Zhiyu Huang, Haochen Liu, Jingda Wu, Chen Lv

IEEE Transactions on Neural Networks and Learning Systems, 2023

Paper | Project |

  • We propose an end-to-end differentiable framework that integrates prediction and planning modules and is able to learn the cost function from data.
sym

Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Driving

Zhiyu Huang, Xiaoyu Mo, Chen Lv

IEEE International Conference on Robotics and Automation (ICRA), 2022

Paper

  • We propose a neural prediction framework based on the Transformer structure to model the relationship among the interacting agents and extract the attention of the target agent on the map waypoints.
sym

Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving

Zhiyu Huang, Jingda Wu, Chen Lv

IEEE Transactions on Neural Networks and Learning Systems, 2022

Paper | Project |

  • We propose a novel framework to incorporate human prior knowledge in DRL, in order to improve the sample efficiency and save the effort of designing sophisticated reward functions.
sym

Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning

Zhiyu Huang, Jingda Wu, Chen Lv

IEEE Transactions on Intelligent Transportation Systems, 2021

Paper |

  • We propose a structural assumption about internal reward function-based human driving behavior and employ sampling-based maximum entropy inverse reinforcement learning (IRL) algorithm to infer the reward function parameters from naturalistic human driving data.
sym

Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding

Zhiyu Huang, Chen Lv, Yang Xing, Jingda Wu

IEEE Sensors Journal, 2020

Paper

  • We propose a novel deep neural network-based system for end-to-end autonomous driving, consisting of multimodal sensor fusion, scene understanding, and conditional driving policy modules.

All Publications

Journal

Conference

Preprint

🎖 Honors and Awards

📖 Education

  • 2019 - 2024, Doctor of Philosophy, Robotics and Intelligent Systems, Nanyang Technological University, Singapore
  • 2023.07 – 2024.01, Visiting Student Researcher, University of California, Berkeley, United States
  • 2015 - 2019, Bachelor of Engineering, Vehicle Engineering, Chongqing University, Chongqing, China

📚 Academic Services

Program Committee

Associate Editor

Journal Reviewer

  • IEEE Transactions on Intelligent Transportation Systems
  • IEEE Transactions on Neural Networks and Learning Systems
  • IEEE Transactions on Intelligent Vehicles
  • IEEE Transactions on Cybernetics
  • IEEE Robotics and Automation Letters
  • Engineering Applications of Artificial Intelligence
  • Artificial Intelligence Review

Conference Reviewer

  • IEEE International Conference on Robotics and Automation (ICRA)
  • IEEE Intelligent Transportation Systems Conference (ITSC)
  • IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • European Conference on Computer Vision (ECCV)
  • Conference on Robot Learning (CoRL)
  • IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • IEEE/CVF International Conference on Computer Vision (ICCV)
  • Conference on Neural Information Processing Systems (NeurIPS)
  • Annual AAAI Conference on Artificial Intelligence
  • International Conference on Machine Learning (ICML)