Halaman

    Social Items

Showing posts with label Machine learning. Show all posts
Showing posts with label Machine learning. Show all posts

 For robotics applications, many consider Robot Operating System (ROS) as the default go-to solution. The version of ROS that runs on the NVIDIA Jetson Nano Developer Kit is ROS Melodic. Installing ROS on the Jetson Nano is simple. Looky here:

Background

ROS was originally developed at Stanford University as a platform to integrate methods drawn from all areas of artificial intelligence, including machine learning, vision, navigation, planning, reasoning, and speech/natural language processing.

From 2008 until 2013, development on ROS was performed primarily at the robotics research company Willow Garage who open-sourced the code. During that time, researchers at over 20 different institutions collaborated with Willow Garage and contributed to the codebase. In 2013, ROS stewardship transitioned to the Open Source Robotics Foundation.

From the ROS website:

The Robot Operating System (ROS) is a flexible framework for writing robot software. It is a collection of tools, libraries, and conventions that aim to simplify the task of creating complex and robust robot behavior across a wide variety of robotic platforms.

Why? Because creating truly robust, general-purpose robot software is hard. From the robot’s perspective, problems that seem trivial to humans often vary wildly between instances of tasks and environments. Dealing with these variations is so hard that no single individual, laboratory, or institution can hope to do it on their own.

Core Components

At the lowest level, ROS offers a message passing interface that provides inter-process communication. Like most message-passing systems, ROS has a publish/subscribe mechanism along with request/response procedure calls. An important thing to remember about ROS, and one of the reasons that it is so powerful, is that you can run the system on a heterogeneous group of computers. This allows you to distribute tasks across different systems easily.

For example, you may want to have the Jetson running as the main node, and controlling other processors as control subsystems. A concrete example is to have the Jetson doing a high-level task like path planning, and instructing microcontrollers to perform lower-level tasks like controlling motors to drive the robot to a goal.

At a higher level, ROS provides facilities and tools for a Robot Description Language, diagnostics, pose estimation, localization, navigation, and visualization. 

You can read more about the Core Components here.

Installation

The installROS repository on the JetsonHacksNano account on Github contains convenience scripts to help us install ROS. The main script, installROS.sh, is a straightforward implementation of the install instructions taken from the ROS Wiki. The instructions install ROS Melodic on the Jetson Nano.

You can clone the repository on to the Jetson:

$ git clone https://github.com/JetsonHacksNano/installROS.git
$ cd installROS

installROS.sh

The install script installROS.sh will install the prerequisites and ROS packages you specify. Usage:

Usage: ./installROS.sh  [[-p package] | [-h]]
 -p | --package <packagename>  ROS package to install
                               Multiple Usage allowed
                               The first package should be a base package. One of the following:
                                 ros-melodic-ros-base
                                 ros-melodic-desktop
                                 ros-melodic-desktop-full
 

Default is ros-melodic-ros-base if do not specify any packages. Typically people will install ros-base if they are not running any desktop applications on the robot.

Example Usage:

$ ./installROS.sh -p ros-melodic-desktop -p ros-melodic-rgbd-launch

This script installs a baseline ROS environment. There are several tasks:

  • Enable repositories universe, multiverse, and restricted
  • Adds the ROS sources list
  • Sets the needed keys
  • Loads specified ROS packages (defaults to ros-melodic-base-ros if none specified)
  • Initializes rosdep

You can edit this file to add the ROS packages for your application.

setupCatkinWorkspace.sh

setupCatkinWorkspace.sh builds a Catkin Workspace.

Usage:

$ ./setupCatkinWorkspace.sh [optionalWorkspaceName]

where optionalWorkspaceName is the name and path of the workspace to be used. The default workspace name is catkin_ws. If a path is not specified, the default path is the current home directory. This script also sets up some ROS environment variables.

The script sets placeholders for some ROS environment variables in the file ~/.bashrc

The script .bashrc is located in the home directory. The preceding period indicates that the file is “hidden”. The names of the ROS variables that the script adds are (they should be towards the bottom of the .bashrc file):

  • ROS_MASTER_URI
  • ROS_IP

The script sets ROS_MASTER_URI to the local host, and basically lists the network interfaces after the ROS_IP entry. You will need to configure these variables for your robots network configuration and how you desire your network topology.

Notes

  • In the video, the Jetson Nano is freshly prepared with L4T 32.2.1 / JetPack 4.2.2
  • In the video, the Jetson Nano is running from a micro-SD card.
Cheers

Install ROS on Jetson Nano

 

Model


Requirements:

  • Python3
  • TensorFlow
  • pip install -r requirements.txt

  Usage

$ python train.py
Hyperparameters
$ python train.py -h
usage: train.py [-h] [--embedding_size EMBEDDING_SIZE]
                [--num_layers NUM_LAYERS] [--num_hidden NUM_HIDDEN]
                [--keep_prob KEEP_PROB] [--learning_rate LEARNING_RATE]
                [--batch_size BATCH_SIZE] [--num_epochs NUM_EPOCHS]
                [--max_document_len MAX_DOCUMENT_LEN]

optional arguments:
  -h, --help            show this help message and exit
  --embedding_size EMBEDDING_SIZE
                        embedding size.
  --num_layers NUM_LAYERS
                        RNN network depth.
  --num_hidden NUM_HIDDEN
                        RNN network size.
  --keep_prob KEEP_PROB
                        dropout keep prob.
  --learning_rate LEARNING_RATE
                        learning rate.
  --batch_size BATCH_SIZE
                        batch size.
  --num_epochs NUM_EPOCHS
                        number of epochs.
  --max_document_len MAX_DOCUMENT_LEN
                        max document length.

Experimental Results

Language Model Training Loss



Text Classification Training Loss

Thanks & Cheers
Stack Exchanges

Multi-task Learning with TensorFlow

 The Intel RealSense T265 Tracking Camera solves a fundamental problem in interfacing with the real world by helpfully answering “Where am I?” Looky here:



Background

One of the most important tasks in interfacing with the real world from a computer is to calculate your position in relationship to a map of the surrounding environment. When you do this dynamically, this is known as Simultaneous Localization And Mapping, or SLAM.

If you’ve been around the mobile robotics world at all (rovers, drones, cars), you probably have heard of this term. There are other applications too, such as Augmented Reality (AR) where a computing system must place the user precisely in the surrounding environment. Suffice it to say, it’s a foundational problem.

SLAM is a computational problem. How does a device construct or update a map of an unknown environment while simultaneously keeping track of its own location within that environment? People do this naturally in small places such as a house. At a larger scale, people have been clever enough to use visual navigational aids, such as the stars, to help build their maps.

This  V-SLAM solution does something very similar. Two fisheye cameras combine with the information from an  Inertial  Measurement  Unit (IMU) to navigate using visual features to track its way around even unknown environments with accuracy. 

Let’s just say that this is a non-trivial problem. If you have tried to implement this yourself, you know that it can be expensive and time consuming. The Intel RealSense T265 Tracking Camera provides precise and robust tracking that has been extensively tested in a variety of conditions and environments.

The T265 is a self-contained tracking system that plugs into a USB port. Install the librealsense SDK, and you can start streaming pose data right away.

Tech Stuffs

Here’s some tech specs:

Cameras

  • OV9282
  • Global Shutter, Fisheye Field of View = 163 degrees
  • Fixed Focus, Infrared Cut Filter
  • 848 x 800 resolution
  • 30 frames per second

Inertial Measurement Unit (IMU)

  • 6 Degrees of Freedom (6 DoF)
  • Accelerometer 
  • Gyroscope

Visual Processing Unit (VPU)

  • Movidius MA215x ASIC (Application Specific Integrated Circuit)

The Power Requirement is 300 mA at 5V (!!!). The package is 108mm Wide x 24.5mm High x 12.50mm Deep. The camera weighs 60 grams.

Installation

To interface with the camera,  Intel provides the open source library librealsense. On the JetsonHacksNano account on Github, there is a repository named installLibrealsense. The repository contains convenience scripts to install librealsense.

Note: Starting with L4T 32.2.1/JetPack 4.2.2 a swap file is now part of the default install. You do not need to create a swap file if you are using this release or later. Skip the following step if using 32.2.1 or above.

In order to use the install script, you will either need to create a swapfile to ease an out of memory issue, or modify the install script to run less jobs during the make process. In the video, we chose the swapfile route. To install the swapfile:

$ git clone https://github.com/jetsonhacksnano/installSwapfile
$ cd installSwapfile
$ ./installSwapfile.sh
$ cd ..

You’re now ready to install librealsense.

$ git clone https://github.com/jetsonhacksnano/installLibrealsense
$ cd installLibrealsense
$ ./installLibrealsense.sh

While the installLibrealsense.sh script has the option to compile the librealsense with CUDA support, we do not select that option. If you are using the T265 alone, there is no advantage in using CUDA, as the librealsense CUDA routines only convert images from the RealSense Depth cameras (D415, D435 and so on).

The location of librealsense SDK products:

  • The library is installed in /usr/local/lib
  • The header files are in /usr/local/include
  • The demos and tools are located in /usr/local/bin

Go to the demos and tools directory, and checkout the realsense-viewer application and all of the different demonstrations!

Conclusion

The Intel RealSense T265 is a powerful tool for use in robotics and augmented/virtual reality. Well worth checking out!

Notes

  • Tested on Jetson Nano L4T 32.1.0
  • If you have a mobile robot, you can send wheel odometry to the RealSense T265 through the librealsense SDK for better accuracy. The details are still being worked out.

Thanks, Cheers.

Jetson Nano – RealSense Tracking Camera

 Many people use Intel RealSense cameras with robots. Here we install the real sense-ros wrapper on the NVIDIA Jetson Nano developer kit. Looky here:




Background

There are several members in the Intel RealSense camera family. This includes the Depth Cameras (D415, D435, D435i) and Tracking Camera (T265). There are also more recent introductions which are just becoming available.

The cameras all share the same Intel® RealSense™ SDK which is known as librealsense2. The SDK is open source and available on Github. We have articles for installing librealsense (D400x article and T265 article) here on the JetsonHacks site.

The size and weight of the cameras make them very good candidates for robotic applications. Computing hardware onboard the cameras provide depth and tracking information directly, which makes it a very attractive addition to a Jetson Nano. Plus the cameras have low power consumption. Because ROS is the most popular middleware application for robotics, here’s how you install realsense-ros on the Jetson Nano.

Install RealSense Wrapper for ROS

There are two prerequisites for installing realsense-ros on the Jetson Nano. The first is to install librealsense as linked above. The second prerequisite is a ROS installation. Checkout Install ROS on Jetson Nano for a how-to on installing ROS Melodic on the Nano.

With the two prerequisites out of the way, it’s time to install realsense-ros. There are convenience scripts to install the RealSense ROS Wrapper on the Github JetsonHacksNano account.

$ git clone https://github.com/JetsonHacksNano/installRealSenseROS
$ cd installRealSenseROS
$ ./installRealSenseROS <catkin workplace name>

Where catkin workplace name is the path to the catkin_workspace to place the RealSense ROS package. If no catkin workplace name is specified, the script defaults to ~/catkin_ws.

Note: Version are in the releases section. The master branch of the repository will usually match the most recent release release of L4T, but you may have to look through the releases for a suitable version. To checkout one of the releases, switch to the installRealSenseROS directory and then:

$ git checkout <version number>

e.g.

$ git checkout vL4T32.2.1

The ROS launch file for the camera(s) will be in the src directory of the Catkin workspace realsense-ros/realsense2_camera/launch There are a variety of launch files to choose from. For example:

$ roslaunch realsense2_camera rs_camera.launch

You will need to make sure that your Catkin workspace is correctly sourced, and roscore is running.

Notes

There are dependencies between the versions of librealsense and realsense-ros. The install scripts are also dependent on the version of L4T. Check the releases on the Github accounts to match.

In the video:

  • Jetson Nano
  • L4T 32.2.1 / JetPack 4.2.2
  • librealsense 2.25.0
  • realsense-ros 2.28.0

realsense-ros does not “officially” support ROS Melodic. However, we haven’t encountered any issues as of the time of this writing.

Thanks Cheers

RealSense ROS Wrapper – Jetson Nano

 Back in September, we installed the Caffe Deep Learning Framework on a Jetson TX1 Development Kit. With the advent of the Jetson TX2, now is the time to install Caffe and compare the performance difference between the two. Looky here:



Background

As you recall, Caffe is a deep learning framework developed with cleanliness, readability, and speed in mind. It was created by Yangqing Jia during his PhD at UC Berkeley, and is in active development by the Berkeley Vision and Learning Center (BVLC) and by community contributors.

Over the last couple of years, a great deal of progress has been made in speeding up the performance of the supporting underlying software stack. In particular the cuDNN library has been tightly integrated with Caffe, giving a nice bump in performance.

Caffe Installation

A script is available in the JetsonHack Github repository which will install the dependencies for Caffe, download the source files, configure the build system, compile Caffe, and then run a suite of tests. Passing the tests indicates that Caffe is installed correctly.

This installation demonstration is for a NVIDIA Jetson TX2 running L4T 27.1, a 64-bit Ubuntu 16.04 variant. The installation of L4T 27.1 was done using JetPack 3.0, and includes installation of OpenCV4Tegra, CUDA 8.0, and cuDNN 5.1.

Before starting the installation, you may want to set the CPU and GPU clocks to maximum by running the script:

$ sudo ./jetson_clocks.sh

The script is in the home directory.

In order to install Caffe:

$ git clone https://github.com/jetsonhacks/installCaffeJTX2.git
$ cd installCaffeJTX2
$ ./installCaffe.sh

Installation should not require intervention, in the video installation of dependencies and compilation took about 14 minutes. Running the unit tests takes about 19 minutes. While not strictly necessary, running the unit tests makes sure that the installation is correct.

Test Results

At the end of the video, there are a couple of timed tests which can be compared with the Jetson TX1. The following table adds some more information:

Jetson TK1 vs. Jetson TX1 vs. Jetson TX2 Caffe GPU Example Comparison
10 iterations, times in milliseconds
MachineAverage FWDAverage BACKAverage FWD-BACK
Jetson TK1 (32-bit OS)234243478
Jetson TX1 (64-bit OS)80119200
Jetson TX2 (Mode Max-Q)7897175
Jetson TX2 (Mode Max-P)6585149
Jetson TX2 (Mode Max-N)5675132

The tests are running 50 iterations of the recognition pipeline, and each one is analyzing 10 different crops of the input image, so look at the ‘Average Forward pass’ time and divide by 10 to get the timing per recognition result. For the Max-N version of the Jetson TX2, that means that an image recognition takes about 5.6 ms.

The Jetson TX2 introduces the concept of performance modes. The Jetson TX1 has 4 ARM Cortex A57 CPU cores. In comparison, there are 6 CPU cores in the Tegra T2 SoC. Four are ARM Cortex-A57, the other two are NVIDIA Denver 2. Depending on performance and power requirements the cores can be taken on or offline, and the frequencies of their clocks set independently. There are five predefined modes available through the use of the nvpmodel CLI tool.

  • sudo nvpmodel -m 1 (Max-Q)
  • sudo nvpmodel -m 2 (Max-P)
  • sudo nvpmodel -m 0 (Max-N)

Max-Q uses only the 4 ARM A57 cores at a minimal clock frequency. Note that from the table, this gives performance equivalent to the Jetson TX1. Max-Q sets the power profile to be 7.5W, so this represents Jetson TX1 performance while only using half the amount of power of a TX1 at full speed!

Max-P also uses only the 4 ARM A57 cores, but at a faster clock frequency. From the table, we can see that the Average Forward Pass drops from the Max-Q value of 78 to the Max-P value of 65. My understanding is that Max-P limits power usage to 15W.

Finally, we can see that in Max-N mode the Jetson TX2 performs best of all. (Note: This wasn’t shown in the video, it’s a special bonus for our readers here!) In addition to the 4 ARM A57 cores the Denver 2 cores come on line, and the clocks on the CPU and the GPU are put to their maximum values. To put it in perspective, the Jetson TX1 at max clock runs the test in about ~10000 ms, the Jetson TX2 at Max-N runs the same test in ~6600 ms. Quite a bit of giddy-up.

Conclusion

Deep learning is in its infancy and as people explore its potential, the Jetson TX2 seems well positioned to take the lessons learned and deploy them in the embedded computing ecosystem. There are several different deep learning platforms being developed, the improvement in Caffe on the Jetson Dev Kits over the last couple of years is way impressive.

Notes

The installation in this video was done directly after flashing L4T 27.1 on to the Jetson TX2 with CUDA 8.0, cuDNN r5.1 and OpenCV4Tegra.

The latest Caffe commit used in the video is: 317d162acbe420c4b2d1faa77b5c18a3841c444c


Thanks Cheers

Caffe Deep Learning Framework – NVIDIA Jetson TX2