01What is the NVIDIA Jetson Orin Nano Super?

The NVIDIA Jetson Orin Nano Super is a tiny AI computer that fits in your hand. It runs real AI models right on the device, with no cloud, no internet, and no big servers. NVIDIA, the biggest AI company in the world, released this board to bring AI to everyone, just like the Raspberry Pi made computers accessible to hobbyists.

This board is different from regular computers because it's built for AI. It can run the same kind of AI you use online, but it does everything locally. That means your data stays on your device.

It's fast, private, and works even when you're offline. People are calling it the Raspberry Pi of AI because it makes AI easy and affordable for everyday people.

In this episode, Julian Goldie explains why this palm-sized computer is a big deal. He shows how it's changing AI from something that lives in the cloud to something that can be inside your fridge, car, or robot vacuum. It's the start of a new wave of Edge AI, and this little board is leading the charge.

02Main Features of the Jetson Orin Nano Super

The Jetson Orin Nano Super has some impressive specs for such a small board. It pumps out 67 TOPS of AI power. TOPS means trillion operations per second.

So this tiny board does trillions of AI math problems every second. Before a software update, it did 40 TOPS. Now it's 1.7 times faster.

If you already own one, you just update it, and your old board becomes a super board overnight.

Here are the key features from the episode:

  • Memory speed: 102 gigabytes per second, which is super fast for moving data.
  • Power use: Only 25 watts, less than a light bulb.
  • Model support: Can run Llama 3.1 with 8 billion parameters, a real full-size AI model.
  • Edge AI: Runs AI locally, so no cloud or internet needed.
  • Ecosystem: Free software, tutorials, and a community called Jetson AI Lab.

This board is made for running smaller, optimized AI models. It's not for training huge models or running massive ones like GPT-4. But for its size, it's incredibly powerful and uses very little energy.

03How to Get Started with the Jetson Orin Nano Super

Getting started with the Jetson Orin Nano Super is not like using an iPhone. It's not plug and play. You need to install software and use the command line.

But if you're willing to tinker, it's a great learning tool. Here are the steps to get going, based on what Julian shows in the episode:

  1. Get the board: Buy the Jetson Orin Nano Super from NVIDIA or a retailer. It's a small board that fits in your hand.
  2. Read the docs: Before you start, check the official NVIDIA documentation. You need to understand the basics of the board and its ports.
  3. Download the software: NVIDIA provides free software and tools. Install the necessary drivers and AI frameworks on your computer or directly on the board.
  4. Connect to a monitor: Use a keyboard, mouse, and monitor to set it up like a small computer. You might need a power supply and a microSD card for storage.
  5. Install AI models: Download AI models like Llama 3.1 8B. You can find them from NVIDIA or the community.
  6. Start with a simple project: Build something small and fun first, like a basic AI demo. Then try something useful like a smart camera or a robot.

If you're new to tech, there's a learning curve. But for developers and students, it's a great way to learn about AI. The skills you learn now will be valuable in a couple of years.

04How to Use the Jetson Orin Nano Super for Real Projects

Once you have the Jetson Orin Nano Super set up, you can do amazing things with it. Julian shows five main use cases in the episode. Here's how people are using this board:

  • Robots: Robots need to make decisions fast. They can't wait for the cloud to say stop. With local AI, they can think instantly. This board is perfect for robot builders.
  • Drones: Drones flying through trees can't rely on Wi-Fi. They need on-board AI. Companies put Jetson boards in drones so they can see, think, and fly by themselves.
  • Smart cameras: You can build a camera that tells the difference between your cat and a stranger, without sending video to the cloud. That's local AI at work.
  • Home AI assistants: Create your own private chat GPT that lives in your house. It's completely private and works offline.
  • Factory inspection: Factories use cameras and AI to check products. They need fast, local processing, and Jetson handles it.

In the episode, Julian explains that the board runs Llama 3.1 8B at about 20 to 30 tokens per second. That means it generates text about as fast as a person reads. For a palm-sized board, that's wild.

It's much faster than your phone, which struggles with small AI models, and it uses less power than a light bulb.

05Performance and Real-World Tests of the Jetson Orin Nano Super

Julian doesn't run a benchmark in the episode, but he shares NVIDIA's claims. The Jetson Orin Nano Super delivers 67 TOPS of AI performance. That's 1.7 times faster than the previous 40 TOPS, thanks to a software update.

The memory speed is 102 gigabytes per second, which helps move data quickly.

For running AI models, it can handle Llama 3.1 with 8 billion parameters. NVIDIA says it runs at around 20 to 30 tokens per second. That's about as fast as a person reads, maybe faster.

In comparison, your phone struggles with small AI models. A laptop can do it but drains the battery. The Jetson sips power at just 25 watts, so it can run for a long time.

The key result is that a tiny board can now do what used to require huge data centers. Five years ago, you needed servers and engineers. Now, a regular person can run real AI on something that fits in a lunch box.

This is a major shift in AI accessibility. It's the beginning of Edge AI, where AI runs on your device, not in the cloud.

06Tips, Limits, and Mistakes to Avoid with the Jetson Orin Nano Super

The Jetson Orin Nano Super has limits. It only has 8GB of RAM. That means it can run small to medium AI models, not the giant ones.

You're not going to run GPT-4 on this thing. You're not going to train a huge model on it. It's made for running smaller, optimized AI.

If you try to run a huge model, it won't work.

Another thing to know: this isn't plug and play. You have to install software, use the command line, and tinker. If you're brand new to tech, there's a learning curve.

Don't expect an iPhone experience. But for developers, students, robot builders, and AI nerds, it's a dream.

Here are some mistakes to avoid, based on the episode:

  • Don't expect to run GPT-4: It won't work with 8GB of RAM. Stick to smaller models.
  • Don't skip the learning phase: Take time to learn the basics. Start with a simple project.
  • Don't ignore the ecosystem: NVIDIA provides free software and a community. Use it to learn and share.
  • Don't forget the privacy benefits: Local AI means no cloud, so your data stays private.

Start small. Build something dumb and fun first. Then build something useful.

That's the best way to learn.

07Who Is the Jetson Orin Nano Super For?

The Jetson Orin Nano Super is for a specific group of people. It's not for everyone. But if you fall into one of these groups, it could be a game-changer for you:

  • Developers and builders: If you like building things, this board lets you create AI-powered projects like robots, drones, and smart cameras.
  • Students and learners: It's a great tool to learn about AI and edge computing. You can experiment without needing a big budget.
  • Business owners: If you want to use AI without sending customer data to the cloud, this board lets you run AI locally. That could give you a real edge over competitors.
  • AI enthusiasts and hobbyists: If you love tinkering with tech, this is the Raspberry Pi of AI. It's affordable and easy to play with.

Why does it matter? Because it's making AI accessible to normal people, not just big tech companies. In five years, these boards will be more powerful and everywhere.

You'll see them in your glasses, your car, your watch, and your kitchen. This is the future that NVIDIA CEO Jensen Huang is building. It's a big deal.