The Intel RealSense T265 Tracking Camera solves a fundamental problem in interfacing with the real world by helpfully answering “Where am I?” Looky here:
One of the most important tasks in interfacing with the real world from a computer is to calculate your position in relationship to a map of the surrounding environment. When you do this dynamically, this is known as Simultaneous Localization And Mapping, or SLAM.
If you’ve been around the mobile robotics world at all (rovers, drones, cars), you probably have heard of this term. There are other applications too, such as Augmented Reality (AR) where a computing system must place the user precisely in the surrounding environment. Suffice it to say, it’s a foundational problem.
SLAM is a computational problem. How does a device construct or update a map of an unknown environment while simultaneously keeping track of its own location within that environment? People do this naturally in small places such as a house. At a larger scale, people have been clever enough to use visual navigational aids, such as the stars, to help build their maps.
This V-SLAM solution does something very similar. Two fisheye cameras combine with the information from an Inertial Measurement Unit (IMU) to navigate using visual features to track its way around even unknown environments with accuracy.
Let’s just say that this is a non-trivial problem. If you have tried to implement this yourself, you know that it can be expensive and time consuming. The Intel RealSense T265 Tracking Camera provides precise and robust tracking that has been extensively tested in a variety of conditions and environments.
The T265 is a self-contained tracking system that plugs into a USB port. Install the librealsense SDK, and you can start streaming pose data right away.
Here’s some tech specs:
- Global Shutter, Fisheye Field of View = 163 degrees
- Fixed Focus, Infrared Cut Filter
- 848 x 800 resolution
- 30 frames per second
Inertial Measurement Unit (IMU)
- 6 Degrees of Freedom (6 DoF)
Visual Processing Unit (VPU)
- Movidius MA215x ASIC (Application Specific Integrated Circuit)
The Power Requirement is 300 mA at 5V (!!!). The package is 108mm Wide x 24.5mm High x 12.50mm Deep. The camera weighs 60 grams.
To interface with the camera, Intel provides the open source library librealsense. On the JetsonHacksNano account on Github, there is a repository named installLibrealsense. The repository contains convenience scripts to install librealsense.
Note: Starting with L4T 32.2.1/JetPack 4.2.2 a swap file is now part of the default install. You do not need to create a swap file if you are using this release or later. Skip the following step if using 32.2.1 or above.
In order to use the install script, you will either need to create a swapfile to ease an out of memory issue, or modify the install script to run less jobs during the make process. In the video, we chose the swapfile route. To install the swapfile:
$ git clone https://github.com/jetsonhacksnano/installSwapfile
$ cd installSwapfile
$ cd ..
You’re now ready to install librealsense.
$ git clone https://github.com/jetsonhacksnano/installLibrealsense
$ cd installLibrealsense
While the installLibrealsense.sh script has the option to compile the librealsense with CUDA support, we do not select that option. If you are using the T265 alone, there is no advantage in using CUDA, as the librealsense CUDA routines only convert images from the RealSense Depth cameras (D415, D435 and so on).
The location of librealsense SDK products:
- The library is installed in /usr/local/lib
- The header files are in /usr/local/include
- The demos and tools are located in /usr/local/bin
Go to the demos and tools directory, and checkout the realsense-viewer application and all of the different demonstrations!
The Intel RealSense T265 is a powerful tool for use in robotics and augmented/virtual reality. Well worth checking out!
- Tested on Jetson Nano L4T 32.1.0
- If you have a mobile robot, you can send wheel odometry to the RealSense T265 through the librealsense SDK for better accuracy. The details are still being worked out.