Robotic Visual Instruction

Yanbang Li, Ziyang Gong, Haoyang Li, Xiaoqi Huang, Haolan Kang, Guangping Bai, Xianzheng Ma

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

We propose a novel paradigm for Human-Robot Interaction (HRI): using hand-drawn sketches for robotic instruction, as an alternative to native language instructions.

Papersenses

A novel approach of creating paper-based human-computer interfaces by drawing and origami, with computer vision and deep learning methods. (1)A game engine creating games and controllers by drawing on paper. (2)A series of paper-based sensors in physical computing system. (2021)

Tornado-Potato Robot

Tornado Potato Climbing Robot is an novel solution for smoothy steady stair-climbing inspired by Tornado Potato. The robot utilizes Archimedean spiral as its locomotion mechanism, providing more stability and smoothness than both legged robots and tracked robots for stair climbing. (2020)

Infinite 360° FOV Tracking AR Projection Mirror

A glasses-free Augmented Reality (AR) installation where the displayed content within a room space dynamically adapts to the user's viewpoint. By precisely manipulating mirror reflections, the system re-routes the projector's light path, significantly extending the AR projection's display range and facilitating dynamic interaction with the user's position and field of view. (2023)

Sky Chair

A chair that if you do not sit on it, it will fly away. Indoors, the chair will stay on the ceiling; when you want to sit, you need to pull the chair down. Once the user leaves the chair, it will float back to the ceiling by itself. It redefines the spatial layout of furniture by freeing up floor space completely. (2019)

Mimosa

Mimosa is a street installation design for london using real mimosa plant as a rain sensor. (This project uses capacitive technology to integrate mimosa plant leaves into the circuit, transforming them into tactile/rainwater capacitive sensors.) When the wheather get rainy, the water drop fall from the sky and touch the leaves of the real mimosa plant. Then the natural mimosa close their leaves while the robotic mimosa will open its leaves as a shelter for passengers. (2019)