Liwei Wang

I am an Assistant Professor in Computer Science and Engineering department at The Chinese University of Hong Kong (CUHK). Before coming to HK, I have worked for more than two years as a Senior Researcher in Tencent AI Lab at Bellevue, US.

I got my PhD from Computer Science Department, University of Illinois at Urbana-Champaign, advised by Prof. Svetlana Lazebnik. Here is my Short Bio.

The Language and Vision ("LaVi") Lab I lead in CSE@CUHK do research in Computer Vision, NLP, with the emphasis on the intersection of vision and language.

The goal of LaVi Lab is to build interactive AI systems that can not only understand and recreate the visual world but also communicate like human beings using natural language
, which covers topics of language+vision; embodied AI; multi-modal dialogue; learning from interaction; efficient AI; machine learning methods for Vision and NLP.

LaVi is hiring Phd students, Research Interns and Fulltime Research Engineers in Language +Vision. Candidates with strong research/engineering background are encouraged to apply!
Please drop me an email directly to
, if you want to work with me.

Email / Google Scholar / To Join LaVi Lab / Publications /

  • 2022/07:
    Our LaVi team won the annual NLP challenge LIC 2022 (Multi-modal Video Understanding track) hosted by CCF and CIPS, check department news here .
  • 2022/07:
    One ECCV paper is accepted
  • 2022/03:
    Three papers got accepted to CVPR 2022 including our new work on Language + Vision.
  • 2022/02:
    One ACL 2022 long paper from our group on "Probing Pre-trained Models" has bee accepted.
  • 2022/02:
    Serving as an Area Chair of ECCV 2022 .
  • 2021/12:
    I was invited to give an online talk at Microsoft Vision-and-Language Research Talk Series.
  • 2021/12:
  • 2021/08:
    I am teaching CSCI 5640 NLP at CUHK.
  • 2021/08:
    Our long paper on Dialog Rewriting was accepted to EMNLP 2021.
  • 2021/08:
    I will teach a graduate level new course in CUHK - CSCI 5640 NLP in the coming semester.
  • 2021/08:
    Three papers got accepted to ICCV 2021, including one Oral on Language + Vision.
  • 2021/07:
    Our collaborated work had won the referit3d CVPR 2021 challenge !
  • 2021/05:
    I will teach a new NLP course in CSE at CUHK next semester.
  • 2021/05:
    New work on text generation accepted to ACL 2021.
  • 2021/04:
    I am joining the Editorial Board of IJCV - top computer vision journal in the world (CCF A journal in AI).
  • 2021/04:
    I will serve as the program committee of EMNLP 2021.
  • 2021/04:
    CUHK CSE 2022 Fall Early Admission starts! For Phd/Master applicants, please click .
  • 2021/03:
    Our work on Logical Reasoning has been accepted by NAACL 2021. Congrats to my intern.
  • 2021/03:
    Four CVPR 2021 papers are accepted including one oral paper.
  • 2020/12:
    I joined the CSE@CUHK as an assistant professor and started my team working on Language and Vision ("LaVi" Lab).
Research Highlights

My students or interns are indicated by '*'. Click full publication list

FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows

Jianqiao Zhao*, Yanyang Li*, Wanyu Du*, Yangfeng Ji, Dong Yu, Michael R. Lyu, Liwei Wang
Arxiv, 2022

Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency

Yanyang Li*, Fuli Luo, Runxin Xu, Songfang Huang, Fei Huang, Liwei Wang
ACL, 2022, long paper    

Multi-View Transformer for 3D Visual Grounding

Shijia Huang*, Yilun Chen, Jiaya Jia, Liwei Wang
CVPR, 2022     Code

SAT: 2D Semantics Assisted Training for 3D Visual Grounding

Zhengyuan Yang, Songyang Zhang, Liwei Wang, Jiebo Luo
ICCV, 2021, Oral Presentation     Code

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation

Liwei Wang, Jing Huang, Yin Li, Kun Xu, Zhengyuan Yang, Dong Yu
CVPR, 2021     Code

Robust Dialogue Utterance Rewriting as Sequence Tagging

Jie Hao, Linfeng Song, Liwei Wang, Kun Xu, Zhaopeng Tu, Dong Yu
EMNLP, 2021,     Code

Comprehensive Image Captioning via Scene Graph Decomposition

Yiwu Zhong*, Liwei Wang, Jianshu Chen, Dong Yu, Yin Li
ECCV, 2020     Code

Improving One-stage Visual Grounding by Recursive Sub-query Construction

Zhengyuan Yang, Tianlang Chen, Liwei Wang, Jiebo Luo
ECCV, 2020     Code

MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning

Jie Lei*, Liwei Wang, Yelong Shen, Dong Yu, Tamara Berg, Mohit Bansal
ACL, 2020     Code

A Fast and Accurate One-Stage Approach to Visual Grounding​

Zhengyuan Yang*, Boqing Gong, Liwei Wang, Wenbing Huang, Dong Yu, Jiebo Luo
ICCV, 2019, Oral Presentation     Code

Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech

Aditya Deshpande, Jyoti Aneja, Liwei Wang, Alexander Schwing, D. A. Forsyth
CVPR, 2019, Oral Presentation

Learning Two-Branch Neural Networks for Image-Text Matching Tasks

Liwei Wang, Yin Li, Jing Huang, Svetlana Lazebnik
TPAMI, 2018     Code

Learning structural motif representations for efficient protein structure search

Yang Liu, Qing Ye, Liwei Wang, Jian Peng
Bioinformatics, 2018     Code

Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space

Liwei Wang, Alex Schwing, Svetlana Lazebnik
NeurIPS, 2017

Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models

Bryan Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, Svetlana Lazebnik
IJCV, 2016     Project

Learning Deep Structure-Preserving Image-Text Embeddings

Liwei Wang, Yin Li, Svetlana Lazebnik
CVPR, 2016     Code

Improving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections

Yunchao Gong, Liwei Wang, Micah Hodosh, Julia Hockenmaier2 Svetlana Lazebnik
ECCV, 2014     Code

Multi-Scale Orderless Pooling of Deep Convolutional Activation Features

Yunchao Gong, Liwei Wang, Ruiqi Guo, Svetlana Lazebnik
ECCV, 2014     Code

Learning Sparse Covariance Patterns for Natural Scenes

Liwei Wang, Yin Li, Jiaya Jia, Jian Sun, David Wipf, James M. Rehg
CVPR, 2012

Bayesian Face Revisited: A Joint Formulation

Dong Chen, Xudong Cao, Liwei Wang, Fang Wen, Jian Sun
ECCV, 2012

Former Interns and Mentored Students

I'm extremely thankful for working with these excellent interns and mentored students: Zhengyuan Yang (intern, phd@University of Rochester), Jie Lei (intern, phd@UNC Chapel Hill), Jeff Zhang (intern, student@UIUC), Aishan Liu (intern, now Assistant Professor at Beihang U), Wanyu Du (intern, phd@University of Virginia), Darryl Hanan(intern, phd@UNC Chapel Hill), Yiwu Zhong (intern, phd@University of Wisconsin-Madison), Yifan (intern, phd@UCSD), Yinya (intern, phd@SYSU), Jing Huang (master@UIUC, now@google research)

(Templates adapted from Arun's webpage.)