NVIDIA Jetson AGX Orin Dev-Kit Eval: Inside an AI Robotic Brain

Artificial intelligence and robotics go together like peanut butter and chocolate. To be truly useful, a service robot must be smart enough to perform its assigned tasks, stay out of danger and not run over its human colleagues. Nearly a decade ago, NVIDIA recognized that the massively parallel processing technology in its graphics processors and the more programmable nature of graphics architectures can be leveraged for machine learning. In 2014, that vision began to materialize with the launch of the Jetson TK1, and an AI powerhouse was born.

Fast forward to 2022 and NVIDIA hasn’t let off the throttle. In March, the company pulled the curtain on the Jetson AGX Orin, the latest robotics and AI platform based on the company’s Ampere GPU technology. While COVID-related lockdowns abroad have slowed the company to get its hardware into the hands of developers, those units are now shipping and we have one of the development kits in-house. So without further ado, let’s get acquainted with the Jetson AGX Orin.

Specifications of the Jetson AGX Orin Development Kit

board specs nvidia jetson agx orin

devkit specs nvidia jetson agx orin

We’ve covered the specs of the various Jetson AGX Orin kits in our previous coverage, but the gist is that there’s about one GeForce RTX 3050 notebook GPU with CUDA and tensor cores to do most of the heavy lifting. The GPU horsepower rides alongside a pair of NVIDIA’s Deep Learning Accelerators and a dozen Cortex A78AE cores in a package with a power budget of up to 60 watts. All this is supported by 32 GB of unified LPDDR5 memory with a bandwidth of over 200 GB/s. If that looks a lot like Apple’s M1 Pro or Max on a high level, let’s just say Cupertino didn’t invent the shared memory architectures.

The production kit has a lot of IO for cameras, microphones and other sensors, but the dev kit goes all out. We have 10 Gbps USB 3.2, DisplayPort, 10 Gigabit Ethernet, a 40-pin GPIO array and headers for automation, audio and JTAG programmers. The micro SD slot and two M.2 slots make room for tons of extra storage and wireless connectivity. There is also an eight-lane PCI Express Gen 4 slot in a x16 form factor. Since the Jetson AGX Orin developer kit only has 64 GB of eMMC storage on board, this could be a good place to stash some extra storage, for example.

unboxed nvidia jetson agx orin devkit

The Jetson AGX Orin Developer Kit connects headless to a Linux PC via one of the USB-C or micro USB ports or as a standalone Linux box. We chose to just use it as its own standalone PC; there’s plenty of memory and CPU resources, and it comes with a full install of Ubuntu Linux 20.04 LTS, ready to go. We will soon be talking in detail about our experience with Orin. Whether using the Jetson AGX Orin as a tethered appliance connected to another Linux PC or as a standalone development environment, everything we needed to get started was in the box.

Working with the Jetson AGX Orin Development Kit

This is all fun to read and write about, but NVIDIA sent us a kit so we had to dive in and check it out. As you can see from the pictures above, the Jetson AGX Orin development kit is: small† It’s about 4.3 inches square and three inches long, about the size of one of Intel’s small NUCs, but a little on the larger side. Its size is aided by having an external power supply, which plugs into the USB-C port directly above the cylinder connector, which can instead be used to power from an alternative power supply. Plug in a keyboard, mouse, and DisplayPort monitor (or an HDMI display with a DP-to-HDMI adapter) and turn it on to get started.

To help us test various features, NVIDIA has also included a USB headset and a 720p webcam, although these are not normally included with the development kit. These bits of hardware were important, though, as one of the demos we’ll be showing shortly is looking for these specific hardware IDs.

The Jetson AGX Orin comes with Ubuntu Linux 20.04 pre-installed right out of the box, so the initial boot order will be familiar to Ubuntu veterans. After choosing our language, location, username, and password, we were presented with the Gnome 3 desktop that Ubuntu has been using since the demise of the Unity interface. NVIDIA has added some handy desktop shortcuts that open documentation links in the browser and sample code folders in Gnome’s file browser. But before we could get started, we had to upgrade to NVIDIA Jetson Jetpack 5, which is about a 10GB download on its own, and add our own code editor.

Since Microsoft added support for Arm64 Linux distributions in 2020, it has become a popular choice among Jetpack developers, whether they prefer C++ or Python. This can be downloaded directly from Microsoft’s Visual Studio Code site on the Other Downloads page, or via apt-get on the command line. The VS Code Marketplace has all the language support extensions needed for both supported languages.

The first time we opened a project, Code asked us to install everything we were missing, so it was a pretty painless installation process.

boxed nvidia jetson agx orin devkit
NVIDIA’s toolkit included everything we needed to get started with Jetson AGX Orin

After installing Jetpack 5 and downloading NVIDIA’s benchmark tools and code samples, we only used about 15 GB of the built-in 64 GB eMMC storage. Developers looking to work directly on the system will want to keep that in mind, as it doesn’t leave much room for data and projects, especially data used to draw conclusions about visual AI models. Keep in mind that the Jetson AGX Orin can run in headless mode connected to another Linux PC via USB, so that’s one way around this limitation, or just use an external USB drive for project storage. The 10 Gbps USB-C port should be fast enough that most developers won’t notice a lag.

Exploring the Jetson AGX Orin Demos

The other side of the coin of the development example is demos. In addition to the code samples NVIDIA provides, there are a host of fully-featured demonstrations developers can dive into to see how the company’s trained AI models respond to live input. We dug into each of them to see how they functioned and explore practical use cases. The biggest and potentially most impressive demo NVIDIA gave us was NVIDIA Riva Automatic Speech Recognition (ASR). This one was best served by capturing the machine learning model at work in a video, so that’s what we’ve embedded below.

The ASR demo gave us a blank terminal window and as we talked it started detecting our speech and transcribing it in real time. There are plenty of other speech recognition software tools on the market, but this is just one part of the human-robot interface. Combine ASR with conversational AI and text-to-speech synthesis and you have a robot that can not only talk to you, but also carry on a conversation, something NVIDIA is extremely proud of. We’re actually going to look at that on the next page. Incidentally, this demo isn’t exactly hardware agnostic; this particular ASR application was built for the specific headset NVIDIA sent us, but the source code can of course be extended for additional hardware support.

As you can see in the video, the transcription wasn’t quite perfect, but NVIDIA says it’s enough for a service robot to examine a user’s intent. The video is unscripted and that is intentional. Most conversations aren’t scripted conversations, even if you’ve put a lot of thought into what you want to say. You can see that as I started and stopped, the AI ​​started and stopped with me, and in that regard the transcription was a pretty fair assessment of what I had to say. It’s nice to see in the video how the AI ​​tries to figure out what I mean; words flash in and out of the terminal window as I go and it starts detecting meaning.

There are many solutions on the market, but this one in particular was fun to watch because it worked. This is a pretty big deal, because low-power, automated speech recognition is a big part of what makes robots easier to interact with. Getting meaning and context out of sentences is sometimes hard enough for humans, so training an AI is quite a monumental task. Many companies have solutions, and NVIDIA’s isn’t the first, but as our video — which deliberately avoids talking about technology — shows, it’s still not really a solved problem. It’s going to get better.

Which brings us to NVIDIA’s AI model adaptation framework TAO, or “Train, Adapt, Optimize” toolkit. This means that the model can be modified by developers to get started quickly. For example, NVIDIA offers the Action Recognition Net model, which is trained to identify certain actions that correspond to specific exercises, such as walking and running, and to detect when a person falls. This model is trained on a few hundred short video clips that show these actions from different angles. NVIDIA has provided a tutorial on extending the model to identify additional actions, such as a push-up or a pull-up.

We followed and were then able to deploy the model on the Jetson AGX Orin itself or on NVIDIA’s DeepStream, which has A100 instances on Azure. This is where the difference between a data center accelerator instance even the upgraded Jetson still sticks out. The Jetson was fast enough to run the model and test that the changes we made were correct, while the DeepStream instances are ridiculously fast at searching video files and identifying actions taken in them. It gave us a taste of the model enhancement workflow provided by TAO and gave us an idea of ​​the development deploy-test workflow provided by Jetson AGX Orin and DeepStream together.

Playing with the tools is fun and all, but these parts are made for serious work. Next, let’s look at some practical implications and see what kind of conclusions we can draw from our time with the kit.

Leave a Comment