Blogs
It's well-known that in academia, the focus of publishing a paper in the field of SLAM is on novelty and achieving the state-of-the-art. While most authors mention their computing platforms, the value of this information is often debatable. For instance, few specify the generation of their Intel Core CPUs, despite the massive performance difference (a 13th-gen i5 can be 5.3 times faster than a 7th-gen i5).
However, in the cost-sensitive industrial world, a more lightweight algorithm can translate into significant savings—renting one less server, buying one less hard drive, or using a cheaper, more compact edge computing platform. For this reason, algorithm efficiency often takes precedence over metrics like pure accuracy.
Another critical factor for industrial applications is real-time performance and low latency. Take SLAM (Simultaneous Localization and Mapping) as an example. To be truly viable outside of a lab setting, it must achieve a frame rate of at least 25 Hz, which means a maximum processing time of 40 milliseconds. Making the most of those precious 40 ms with limited computing power is a major challenge for any SLAM engineer.
This article series will explore how to accelerate computer vision algorithms to help vision practitioners transition from academia to industry.
Why Not GPUs?
In most cases, GPUs are an ideal acceleration tool. Many excellent works in the SLAM field leverage CUDA for acceleration. However, our team decided to forgo GPUs in our odometry pipeline after careful evaluation. This decision was based on our product's specific requirements and a few key factors:
1. Cost: The High Price of Hardware
Industrial computing platforms often lack GPUs to keep costs down. The popular Nvidia graphics cards and Jetson developer boards are notoriously expensive. Even with integrated GPUs like ARM's Mali, you need costly, high-frequency memory to ensure fast data access, which also increases power consumption. So, if you're a student complaining about a slow Jetson, cherish it—you might not have access to such powerful hardware after graduation.
2. Latency: The Data Transfer Bottleneck
Using a GPU introduces significant latency from data transfers between system memory and GPU memory. This delay is catastrophic for real-time applications. For extended reality (XR) companies that need microsecond-level latency, GPUs can be a major bottleneck outside of rendering.
3. Code Optimization: The Power of Software
By meticulously controlling code quality and optimizing at the software level, the performance of a seemingly weak CPU can surprise you and, in some cases, even rival a GPU. This demonstrates the immense potential of software optimization to compensate for hardware limitations.
4. Chained Computations: A Structural Limitation
Chained computations, where the next step depends on the result of the previous one, are fundamentally inefficient for GPUs. Trying to force this type of sequential work onto a parallel architecture is simply a poor fit.
5. Data Volume: Not Every Task Needs Massively Parallel Processing
Apart from tasks involving massive data processing like neural networks or point cloud registration, most algorithms don't handle a large enough data volume to necessitate a GPU. For many vision algorithms, a CPU is more than sufficient.
6. Knowledge Sharing: Exploring Less-Known Techniques
While a wealth of articles on GPU acceleration exists, we prefer to introduce less-commonly discussed knowledge. Our team's decision to avoid GPUs is partly related to RoboBaton-VIOBOT2 positioning, but more importantly, we aim to provide a fresh perspective for vision professionals on how to achieve high-performance algorithms without expensive hardware.
This article aims to provide a comprehensive look at the challenges and opportunities in optimizing computer vision for the industrial world. If you find these topics intriguing, stay tuned for the rest of our series, where we'll dive deeper into more practical methods and tricks for acceleration.
