Dexterous Manipulation via Vision-Intent-Action Models

Autonomous robots will require human-like visual manipulation capabilities to perform experiments in situ, build space habitats, and maintain living-spaces and research equipment. However, while current object-centric approaches to learning visual affordances have led to significant advancements, robots still fall short of truly dexterous manipulation. Object-centric representations do not provide adequate information for physical reasoning about robot actions, as they only encode abstract concepts of what a robot can do to objects in the environment. In contrast, dexterous manipulation necessitates models that encode information on how robots can influence their environments in terms of grounded actions. Building upon insights from sampling-based planning, this work initiates the transition to action-centric frameworks through the introduction of Vision-Intent-Action (VIA) models. By encoding the actions robots can take to fulfill their intents, these models will improve robots' physical reasoning capabilities and support lifelong learning, so robots can consistently accomplish intricate manipulation tasks and acquire new skills as they work.

This is a mature technology (TRL 7+) — the realistic path in is usually NASA's Technology Transfer Program: licensing an existing NASA patent, or a Space Act Agreement to use NASA facilities/expertise directly. NASA also runs a startup licensing program with no upfront fee for companies formed to commercialize a specific NASA technology.

None of these are guaranteed paths for this specific project — TechPort itself doesn't have an "apply" button. Reaching out to the contact(s) above with a specific question is usually the fastest way to find out what's actually open.