About
I am a Ph.D. working with Prof. Chongjie Zhang and Prof. Yang Gao at Institute for Interdisciplinary Information Sciences, Tsinghua University, headed by Prof. Andrew Yao. I received my B.Sc. degree in Theoretical and Applied Mechanics from Peking University in 2019, with a double major in Computer Science. During my Ph.D., I was fortunate to visit Northwestern University and work with Prof. Zhaoran Wang .
My primary research goal is to build intelligent and autonomous agents and deepen the understanding of our own intelligence. Toward this goal, my research interest has been focused on designing algorithms for decision-making in a principled and data-driven approach, which are essential for agents to act and learn effectively in complex environments. Current directions I am working on include offline reinforcement learning, reinforcement learning with foundation models, and reinforcement learning theory.
Please feel free to contact me if you are interested in collaborating with me.
Publications and Preprints
-
Bayesian Design Principles for Offline-to-Online Reinforcement Learning
Hao Hu*, Yiqin Yang*, Jianing Ye, Chengjie Wu, Ziqing Mai, Yujing Hu, Tangjie Lv, Changjie Fan, Qianchuan Zhao, Chongjie Zhang
Forty-first International Conference on Machine Learning (ICML), 2024
PDF | Code -
Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents
Zhihan Liu*, Hao Hu*, Shenao Zhang*, Hongyi Guo, Shuqi Ke, Boyi Liu, Zhaoran Wang
Forty-first International Conference on Machine Learning (ICML), 2024
NeurIPS Workshop on Foundation Models for Decision Making, 2023
PDF | Code | Project Page -
Planning, Fast and Slow: Online Reinforcement Learning with Action-Free Offline Data via Multiscale Planners
Chengjie Wu*, Hao Hu*, Yiqin Yang, Ning Zhang, Chongjie Zhang
Forty-first International Conference on Machine Learning (ICML), 2024
PDF | Code -
Stylized Offline Reinforcement Learning: Extracting Diverse High-Quality Behaviors from Heterogeneous Datasets
Yihuan Mao, Chengjie Wu, Xi Chen, Hao Hu, Ji Jiang, Tianze Zhou, Tangjie Lv, Changjie Fan, Zhipeng Hu, Yi Wu, Yujing Hu, Chongjie Zhang
The Twelfth International Conference on Learning Representations (ICLR), 2024
PDF -
One Objective to Rule Them All: A Maximization Objective Fusing Estimation and Planning for Exploration
Zhihan Liu* Miao Lu* Wei Xiong* Han Zhong, Hao Hu, Shenao Zhang, Sirui Zheng, Zhuoran Yang, Zhaoran Wang
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS Spotlight), 2023
PDF | Code -
Unsupervised Behavior Extraction via Random Intent Priors
Hao Hu*, Yiqin Yang* Jianing Ye, Ziqing Mai, Chongjie Zhang
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023
PDF | Code -
What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
Rui Yang, Yong Lin, Xiaoteng Ma, Hao Hu , Chongjie Zhang, Tong Zhang
Eleventh International Conference on Learning Representations (ICLR), 2023
PDF | Code -
The Provable Benefit of Unsupervised Data Sharing for Offline Reinforcement Learning
Hao Hu*, Yiqin Yang*, Qianchuan Zhao, Chongjie Zhang
Eleventh International Conference on Learning Representations (ICLR), 2023
PDF | Code -
Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery
Yiqin Yang*, Hao Hu*, Wenzhe Li*, Siyuan Li, Jun Yang, Qianchuan Zhao, Chongjie Zhang
Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2023
PDF | Code -
On the Role of Discount Factor in Offline Reinforcement Learning
Hao Hu*, Yiqin Yang*, Qianchuan Zhao, Chongjie Zhang
International Conference on Machine Learning (ICML), 2022
PDF | Code -
Offline Reinforcement Learning with Value-based Episodic Memory
Xiaoteng Ma*, Yiqin Yang*, Hao Hu*, Qihan Liu, Jun Yang, Chongjie Zhang, Qianchuan Zhao, Bin Liang
Tenth International Conference on Learning Representations (ICLR), 2022
PDF | Code -
On the Estimation Bias in Double Q-Learning
Zhizhou Ren, Guangxiang Zhu, Hao Hu, Beining Han, Jianglun Chen, Chongjie Zhang
Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS), 2021
PDF | Code -
Generalizable Episodic Memory for Deep Reinforcement Learning
Hao Hu, Jianing Ye, Zhizhou Ren, Guangxiang Zhu, and Chongjie Zhang
International Conference on Machine Learning (ICML), 2021
PDF | Code -
MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration
Jin Zhang*, Jianhao Wang*, Hao Hu, Tong Chen, Yingfeng Chen, Changjie Fan, Chongjie Zhang
International Conference on Machine Learning (ICML), 2021
PDF | Code -
Query-Efficient Offline Preference-Based Reinforcement Learning via In-Dataset Exploration
Hao Hu*, Yiqin Yang*, Shuai Wang, Bo Liu, Yang Gao, Chongjie Zhang
Under Review
PDF
Experience
- Visiting Scholar (Mar. 2023 -- Sept. 2023)
NURL group, IEMS, Northwestern University
Supervisor: Zhaoran Wang
- Engineering Intern (Jun. 2018 -- Sept. 2018)
News & Relevance Team, STCA, Microsoft
- Research Intern (Nov. 2017 -- Jun. 2018)
VMC group, Peking University
Supervisor: Shiliang Zhang
Services
- Teaching Assistant
Artificial Intelligence: Principles and Techniques, Fall, 2020
Artificial Intelligence: Principles and Techniques, Fall, 2021
Deep Reinforcement Learning, Spring, 2022 - Student Instructor
Theoretical Mechanics, Spring, 2018
- Reviewer
NeurIPS (2023,2024), ICLR (2024,2025), ICML (2024), AAAI (2024), AAMAS (2023), TMLR
Selected Talks
- Data-driven Reinforcement Learning
Bytedance AI Lab, 2023.11 | PDF
- Unsupervised Behavior Extraction via Random Intent Priors
RL China, 2023.10 | PDF
- On the Role of Discount Factor in Offline
Reinforcement Learning
RL China, 2022.06 | PDF | Video
- Bayesian Design Principles for Offline-to-Online Reinforcement Learning
RL China, 2022.06 | PDF | Video
- Generalizable Episodic Memory for Deep Reinforcement Learning
Nanjing University, 2021.07 | PDF
Education
-
Ph.D. in Computer Science
IIIS, Tsinghua University, 2019 -- 2024
-
B.Sc. in Theoretical and Applied Mechanics
Peking University 2015 -- 2019
Double major: Computer Science and Technology