OpenJaws: A Versatile Framework for Accurate Manipulation Demonstrations

Paper: OpenJaws: An Open Hardware UMI Framework with Stereo Localization for Generalist Manipulation

Authors: Adithyavairavan Murali, Yash Oza, Karanbir Chahal, Neel Jawale, Silas De Vries, Hesam Rabeti, Erwin Coumans, Balakumar Sundaralingam, Stan Birchfield

Executive Summary

The research introduces OpenJaws, a versatile platform designed for gathering manipulation demonstrations using handheld data-collection devices. This platform addresses key limitations found in current systems, particularly regarding the accuracy of localization, which is essential for effective robotic manipulation tasks.

Traditional approaches rely on monocular visual-inertial odometry to track the position of robotic end-effectors, but they are hindered by issues such as scale ambiguity and drift. These limitations can compromise the reliability of the data collected. Alternatively, some systems utilize proprietary software and hardware that add complexity and cost, making them less accessible for widespread use.

OpenJaws consists of an open hardware and software framework that features a modular, 3D-printed gripper compatible with a standardized ISO adapter. This design is paired with an affordable stereo camera, which enhances localization capabilities by providing more accurate camera trajectories compared to monocular systems.

To support the development and evaluation of this technology, the researchers created UMI-Loc-Bench-v1, a benchmark tool specifically designed to measure the localization accuracy of handheld data-collection devices in real-world scenarios. The findings indicate that stereo localization significantly outperforms the current monocular methods, suggesting that businesses could benefit from more accurate data collection for robotic applications.

Furthermore, the research demonstrates that policies trained using data collected with OpenJaws can be implemented on actual robots to perform various tasks. This indicates potential applications in diverse fields such as manufacturing, logistics, and healthcare, where precise manipulation is crucial.

The entire OpenJaws framework, including its hardware designs, software code, and collected data, is available under a commercially permissive license, promoting accessibility and encouraging further innovation in robotics. The resources can be accessed at the provided website, allowing companies to explore integration into their own robotic systems without significant upfront investment in proprietary technologies.

Academic Abstract

Handheld data-collection interfaces such as the Universal Manipulation Interface (UMI) allow manipulation demonstrations to be gathered in the wild without a robot in the loop, but the quality of the resulting action labels is bounded by the accuracy of vision-based localization. Existing systems recover end-effector pose from monocular visual-inertial odometry, which is subject to scale ambiguity and drift, or delegate tracking to closed-source modules and external infrastructure that reintroduce cost and setup burden. We present OpenJaws, a fully open hardware and software framework for sensorized UMI that improves localization and simplifies deployment across robot platforms. OpenJaws pairs a modular, 3D-printed gripper exposing a standardized ISO adapter with a low-cost stereo camera to recover metric camera trajectories. We further introduce UMI-Loc-Bench-v1, a benchmark quantifying real-world localization accuracy for handheld data-collection devices, and show that stereo localization substantially outperforms monocular alternatives. Finally, we demonstrate that policies trained on data collected with OpenJaws can be deployed on a real robot across multiple tasks. Hardware, software, and data are released under a commercially permissive license. Code, data, and videos are available at https://open-jaws.github.io/