About Me
I am an assistant professor of computer science at Brown University, where I direct the PALM🌴 research lab, studying computer vision, machine learning, and artificial intelligence. I work part-time as a staff research scientist at Google Research.
Our research focuses on multimodal concept learning and reasoning, temporal dynamics modeling of humans and objects, and transfer learning of representation and skills to embodied agents. I believe multimodal learning is a pathway for computer vision to help language understanding, robotics, and social science.
Previously, I received my Ph.D. from the University of Southern California in 2016, advised by Prof. Ram Nevatia. I completed my bachelor degree in Computer Science at Tsinghua University in 2011.
I have received Brown University's Richard B. Salomon Faculty Research Award and Samsung AIT's Global Research Outreach Award for multimodal concept learning from videos. My research on behavior prediction appeared in the CVPR 2019 best paper finalist. I am serving as area chairs for CVPR, NeurIPS, and ACL conferences. I am also a junior faculty teaching fellow at Brown.
Teaching
- CSCI 2470, Deep Learning, Fall.
- CSCI 2952-N, Advanced Topics in Deep Learning, Spring.
- Short course on Multimodal Transformers, ICASSP 2022 and AAAI 2023.
- CSCI 2950-K, Special Topics in Computational Linguistics (co-taught with Eugene Charniak), Fall 2020.
Group
PhD students
- Apoorv Khandelwal (Presidential Fellow, co-advised with Ellie Pavlick)
- Calvin Luo
- Megan Wei
- Nate Gillman
- Shijie Wang
- Tian Yun (Co-advised with Ellie Pavlick)
- Yuan Zang
- Zilai Zeng (Incoming)
- Zitian Tang
Alumni
- Kevin Zhao (master of 2024 at Brown)
- Mandy He (class of 2024 at Brown, now software engineer at Duolingo)
- Minh Quan Do (class of 2024 at Brown)
- David Heffren (class of 2024 at Brown, now PhD student at JHU Applied Math)
- John Ryan Byers (class of 2024 at Brown, new master student at Cornell Tech)
- Ce Zhang (master of 2023 at Brown, now PhD student at UNC CS)
- Changcheng Fu (master of 2023 at Brown, now PhD student at USC CS)
- Kunal Handa (class of 2023 at Brown, now master student at Oxford)
- Jessica Li (class of 2023 at Brown, now software engineer at Headway)
- Tian Yun (master of 2022 at Brown, now PhD student at Brown CS)
- Usha Bhalla (class of 2022 at Brown, now PhD student at Harvard CS)
- Emily Byun (class of 2021 at Brown, now PhD student at CMU MLD)
Mentorship
Services
- Area Chair, CVPR 2020 to 2024.
- Area Chair, ICCV 2023.
- Area Chair, ECCV 2022 and 2024.
- Area Chair, ACL 2023.
- Area Chair, NeurIPS 2023.
- Senior PC, AAAI 2021 and 2022.
- Area Chair, WACV 2017 and 2018.