Deep Learning

Aligning Step-by-Step Instructional Diagrams to Video Demonstrations
We introduce a new framework and dataset (IAW) for aligning assembly diagrams from instruction manuals with real-world video segments, enabling cross-modal retrieval and step-level alignment between illustrated instructions and assembly actions in videos.
Jiahao Zhang, Anoop Cherian, Yanbin Liu, Yizhak Ben-Shabat, Cristian Rodriguez, Stephen Gould
PDF Cite Code Dataset Poster Slides Video DOI ArXiv Supplementary
Aligning Step-by-Step Instructional Diagrams to Video Demonstrations
GoferBot: A Visual Guided Human-Robot Collaborative Assembly System
GoferBot is a novel assembly system that seamlessly integrates all sub-modules by utilising implicit semantic information purely from visual perception.
Zheyu Zhuang, Yizhak Ben-Shabat, Jiahao Zhang, Stephen Gould, Robert Mahony
PDF Cite Video DOI ArXiv
GoferBot: A Visual Guided Human-Robot Collaborative Assembly System
Image Caption Generator
An encoder(Resnet152)-decoder(LSTM) implementation of image caption model.
Code
Image Caption Generator