Transporter: Unsupervised Learning of Object Keypoints for Perception and Control

This directory contains a Sonnet implementation of the Transporter architecture and a notebook explaining how the model can be used for keypoint inference. To launch the notebook in Google colab, click here.

The Transporter is a neural network architecture for discovering concise geometric object representations in terms of keypoints or image-space coordinates. Our method learns from raw video frames in a fully unsupervised manner, by transporting learnt image features between video frames using a keypoint bottleneck. The discovered keypoints track objects and object parts across long time-horizons more accurately than recent similar methods.

For details, see our paper Unsupervised Learning of Object Keypoints for Perception and Control.

If you use the code here please cite this paper.

OpenSource

19 Jun 2019