On Research

I am marching towards Synergetic & Holistic Intelligence.

My long term goal is to advance AI research and technologies in interrelated fields such as computer vision, machine learning, language understanding and robotics, to build intelligent systems, either virtual or embodied, to facilitate understanding multiple sensory inputs, to gain actionable insights from perception to cognition, to solve important real-world problems and to better serve our human race.

In the medium term, I am putting more emphasis on computer vision, machine learning and their applications, with a strong focus on accurate and efficient understanding of various types of objects and activities from sensory inputs such as images and videos. Over the past few years, I have explored a wide range of topics towards accurate and efficient visual understanding: from image-level classification, to instance-level object detection, to video-level detection and tracking, and more recently to spatio-temporal activity recognition and pixel-level segmentation etc. My team and I have been lucky to have won some international AI competitions and set new state-of-the-arts on major computer vision benchmarks. I am also fortunate to have been working on a broad spectrum of applied research projects with more than $10 million support, from research assistant, to team leader, and PI/Co-PI, with collaborators and support from industry, academic units and government agencies. This enables me to understand the true depth of challenges arose from real-world data and problems, or even in collaboration, management and technology transfer.

To emphasize, my current research focuses on accurate & efficient visual understanding for AI systems & application, in particular I have recently worked in:

    • Computer Vision: classification, object detection, segmentation, activity recognition, etc.
    • Machine Learning: deep learning, weakly-supervised learning, transfer learning, efficient learning, etc.
    • AI Systems & Applications for Science, Education, Agriculture, Medcine, Finance, Transportation, etc.

My research activities include multiple aspects to solve such problems and to advance AI research: projects, papers, competitions, organizing workshops, training students etc.

Research Highlights

Please find more publication and technical reports on Google Scholar. Some of our codes are available at GitHub.

Abbreviations: [C]: Conference; [J] Journal; [W] Workshop; [TR]: Technical Report; [B]: Book; [P]: Patent; [Comp]: Competition; [Proj]: Project; [Org] Program Organization; [SOTA]: State-of-the-art (at the time of publication)

Classification & Learning:

Object Detection:


Activity Recognition:

    • [Proj] Deep Intermodal Video Analytics (DIVA), sponsored by IARPA, 2017.10 - 2021.09
    • [W] Object-Centric Spatio-Temporal Activity Detection and Recognition, Mandis Beigi, Lisa M Brown, Quanfu Fan, John Henning, Chung-Ching Lin, Honghui Shi, Chiao-fe Shu, Rogerio Feris, NIST TRECVID Workshop, 2018
    • [Comp] NIST/IARPA TRECVID Activity Recognition Challenge 1st Place (2018)

Visual Relationship, Reasoning, Grounding; Vision + Language:

    • [Proj] Collaborative research on multimedia with Blender Lab at UIUC, 2020
    • [Proj] Collaborative research on visual reasoning with IBM Research, 2020
    • [Comp] Visual Relationship Detection - Google AI Open Images Challenge, Silver Medal (2018)

Human-Centered Vision:

Low-Level Vision:

AI Systems:

AI Applications - Science & Engineering:

    • [J] Deep Learning-based Automated Image Segmentation for Concrete Petrographic Analysis, Yu Song, Zilong Huang, Chuanyue Shen, Humphrey Shi, David A Lange, Cement and Concrete Research, 2020 (top journal for materials in civil engineering)
    • [Proj] Cement Phase Segmentation, collaborated with UIUC Civil Engineering
    • [Proj] Galaxy Classification, collaborated with UIUC Astronomy
    • [Proj] Gravitational Lens Detection, collaborated with UIUC Astronomy

AI Applications - Education:

    • [Proj] Intelligent Learning Advisor, sponsored by IBM Research
    • [Proj] AI for Education, sponsored by New Oriental Education Technology

AI Applications - Agriculture:

AI Applications - Medicine:

    • [C] FOAL: Fast Online Adaptive Learning for Cardiac Motion Estimation, Hanchao Yu, Shanhui Sun, Haichao Yu, Xiao Chen, Honghui Shi, Thomas Huang, Terrence Chen, CVPR, 2020 (fast & online adaptation method, acceptance rate 22.0 %)
    • [Proj] Multi-modal Medical Image Understanding, sponsored by Jump ARCHES
    • [Proj] Multiphoton Image Analysis for Cancer Diagnosis, sponsored by Mayo Clinic & UIUC

AI Applications - Finance:

    • [Proj] Deep Learning in Financial Modeling and Strategy, sponsored by Jump Trading