Facebook AI has announced a new collaboration with Matterport that will advance Habitat, a simulation platform for research in Embodied AI, to help robots better interact with the physical world.

Through the partnership, spatial data company Matterport will make the largest ever dataset of 3D indoor spaces available for academic, non-commercial uses.

The Habitat-Matterport 3D Research Dataset (HM3D) is a collection of 1,000 high-resolution Matterport digital twins made up of residential, commercial, and civic spaces generated precisely from real-world environments.

A digital twin is a virtual, real-time representation of a physical object or process.

HM3D aims to advance embodied AI research, which seeks to teach robots and virtual AI assistants to understand and interact with the complexities of the physical world.

“Until now, this rich spatial data has been glaringly absent in the field, so HM3D has the potential to change the landscape of embodied AI and 3D computer vision,” said Dhruv Batra, research scientist at Facebook AI Research. “Our hope is that the 3D dataset brings researchers closer to building intelligent machines, to do for embodied AI what pioneers before us did for 2D computer vision and other areas of AI.”

HM3D is a step towards helping robots navigate real-world environments, better understanding the variations of spaces such as bedrooms, bathrooms, kitchens and hallways, as well as the different configurations of those rooms within every structure.

It can also assist robots in recognising how objects within rooms are typically arranged so that instructions are correctly understood.

The research could one day be used in production applications like robots that can retrieve medicine from a bedroom nightstand or AR glasses that can help people remember where they left their keys.

“We are excited to collaborate with Facebook as we provide the academic and research communities access to this unique spatial dataset that is sure to impact how we work and live,” said Conway Chen, vice president of business development and alliances at Matterport. “With more than five million spaces captured with the Matterport platform, we are the only company that can offer a diverse library of high-resolution, data-rich digital twins of various styles, sizes, and complexities from across the world."

Chen added: "HM3D can also be used more broadly by academia, and we can’t wait to see what innovations emerge.”

Yasutaka Furukawa, associate professor of computing science at Simon Fraser University, said that the market has been challenged with a lack of spatial data to advance innovation in real-estate, construction, robotics, and augmented reality.

“But with the HM3D dataset from the collaboration between Matterport and Facebook AI, we're excited about the significant progress we'll make in advancing research in indoor scene reconstruction, generation, and analysis at a house-scale for the first time," added Furukawa.

Share Story: