Shanghai Jiao Tong University
Shanghai, China

Renming Huang

I am a researcher at Shanghai Jiao Tong University (SJTU), advised by Prof. Panpan Cai.

I received my master's degree from the University of Electronic Science and Technology of China (UESTC), where I was advised by Prof. Yang Yang and Prof. Guoqing Wang. I also work closely with Prof. Peng Wang.

My research lies at the intersection of robotics and reinforcement learning. I study how robots can learn from human and robot demonstrations and generalize to long-horizon tasks in open-world environments. In particular, I investigate action representations and tokenization for robot learning. I am interested in enabling robots to infer intent from human behavior, pushing the boundaries of generalization, and building efficient robot-learning methods with compositional generalization.

News

Jun 2026 arXiv PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty PO-PDDL Preprint available.
Apr 2026 RSS 2026 Mimic Intent, Not Just Trajectories MINT Accepted.
Feb 2026 arXiv Any House Any Task: Scalable Long-Horizon Planning for Abstract Human Tasks AHAT Preprint available.
Oct 2025 UIST 2025 AttentionAR: AR Adaptation and Warning for Real-World Safety via Attention Modeling and MLLM Reasoning AttentionAR Accepted.
Nov 2024 CoRL 2024 Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidance EGR-PO Accepted.
Sep 2024 ECCV 2024 Diffusion Models as Optimizers for Efficient Planning in Offline RL Diffusion Planner Accepted.
Mar 2023 VR 2023 Multimodal Apology: Using WebXR to Repair Trust with Virtual Companion Multimodal Apology Accepted.

Selected Publications

PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty

Wenjing Tang, Xuanjin Jin, Yuan Liu, Renming Huang, Cewu Lu, Panpan Cai

arXiv preprint arXiv arXiv Project — A demonstration-driven framework that learns symbolic POMDP models from real-robot videos for robust belief-space planning under perception and execution uncertainty.

Mimic Intent, Not Just Trajectories

Renming Huang, Chengyang Zeng, Weichao Tang, Junyi Cai, Cewu Lu, Panpan Cai

Robotics: Science and Systems RSS 2026 arXiv Code — We propose learning to mimic the underlying intent of demonstrations rather than directly copying trajectories, enabling more robust and generalizable robot behavior.

Any House Any Task: Scalable Long-Horizon Planning for Abstract Human Tasks

Zhihong Liu, Yang Li, Renming Huang, Cewu Lu, Panpan Cai

arXiv preprint arXiv arXiv — A scalable framework for long-horizon task planning that generalizes across diverse house layouts and abstract human task specifications.

Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidance

Renming Huang, Shenyu Liu, Yuying Pei, Peng Wang, Guansong Wang, Yang Yang, Heng Shen

Conference on Robot Learning CoRL 2024 arXiv — We propose a subgoal-guided imitation learning framework that enables goal-reaching policy learning from non-expert, suboptimal observations without requiring expert demonstrations.

Diffusion Models as Optimizers for Efficient Planning in Offline RL

Renming Huang, Yuying Pei, Guansong Wang, Yanjiang Guo, Yang Yang, Peng Wang, Heng Shen

European Conference on Computer Vision ECCV 2024 arXiv — We leverage diffusion models as trajectory optimizers for offline reinforcement learning, achieving efficient planning by treating diffusion sampling as an optimization process.

See the full list on the Publications page or Google Scholar.