The innovative manufacturing solution built based on NVIDIA technology enables the robotic arm to conduct autonomous equipment inspections and seamlessly integrate new products into the production line.

By leveraging NVIDIA's digital twin technology, the Amazon Devices and Services division is making significant advancements in the manufacturing sector through a brand-new physical AI software solution.

The innovative "simulation-driven" zero-contact manufacturing solution deployed by Amazon's equipment factory this month enables the robotic arm to perform two tasks through synthetic data: conducting quality audits for multiple types of equipment and seamlessly integrating new products into the production line. The entire process does not require any hardware modifications.

This new technology can integrate the production line process simulation technology independently developed by Amazon's equipment department with the digital twin enabled by NVIDIA. This technology adopts a modular AI-driven workflow, which can achieve faster and more efficient detection compared to the previously used detection equipment.

By simulating processes and products in a digital twin, there is no need for costly and time-consuming physical prototype design. This simplifies the workflow of manufacturers and shortens the time to market for new products.

To achieve zero-sample manufacturing for robot operation, this solution utilizes high-fidelity physical rendering models of Amazon devices and factory workstations to generate synthetic data. Such factory-customized data can significantly enhance the performance of AI models in simulation processes and real workstations, and greatly reduce the gap between simulation and reality before deployment.

This is a significant step towards universal manufacturing: By leveraging automated systems and technologies, it is possible to flexibly handle a wide variety of products and diverse production processes without the need for physical prototypes.

AI and digital twins enhance the understanding ability of robots

The Amazon Devices and Services department was able to build a faster, more modular and easier-to-control manufacturing process by training robots in digital twins to identify and handle new devices. This enabled the production line to seamlessly switch from the inspection of one product to that of another simply through software adjustments.

The movements of the robots can be fully configured based on the training completed in the simulation environment, in order to achieve various stages of product manufacturing, including assembly, testing, packaging and quality inspection.

A complete set of NVIDIA Isaac technologies provides physically accurate simulation-driven solutions and technical support for Amazon Devices and Services.

When introducing new devices, Amazon Devices and Services imports their Computer-Aided Design (CAD) models into NVIDIA Isaac Sim—an open-source robotic simulation reference application built on the NVIDIA Omniverse platform.

NVIDIA Isaac can generate over 50,000 diverse synthetic images for each device using its CAD model, which is crucial for training object and defect detection models.

Subsequently, Isaac Sim processes the data and leverages NVIDIA Isaac ROS to generate robotic arm motion trajectories for operating the product.

Robots trained exclusively on synthetic data can grasp packages and products of varying shapes and sizes for visual inspection. Real-world workstation (left) vs. simulated workstation (right). Image provided by Amazon Devices and Services.

AWS has significantly accelerated the R&D process of this technology. It leverages AWS Batch on Amazon EC2 G6 instances to conduct distributed AI model training based on the product specifications of Amazon Devices. Meanwhile, NVIDIA Isaac Sim runs on Amazon EC2 G6 series instances to perform physics-based simulation and synthetic data generation.

This solution adopts Amazon Bedrock—a service for building generative AI applications and agents. By analyzing product specification documents, it plans high-level tasks and specific quality inspection test cases in factories. Amazon Bedrock Agent Core will be used for autonomous workflow planning across multiple factory sites on the production line, capable of processing multimodal product specification inputs such as 3D designs and surface properties.

To help robots understand their surrounding environment, the solution utilizes NVIDIA cuMotion—a CUDA-accelerated motion planning library. It can generate collision-free trajectories in milliseconds on the NVIDIA Jetson AGX Orin module. Additionally, the nvblox library (a component of Isaac ROS) generates distance fields, providing support for cuMotion’s collision-free trajectory planning.

NVIDIA FoundationPose—a foundational model trained on 5 million synthetic images—enables pose estimation and object tracking. This helps ensure that robots under Amazon Devices and Services can accurately identify the exact position and orientation of devices.

FoundationPose is crucial for the new manufacturing solution, as it can generalize to entirely new, unseen objects without prior exposure to the target items. This allows seamless transitions between different products and eliminates the need to recollect data and retrain models whenever products change.

In the product quality inspection phase, this new solution can be used for defect detection on the production line. Its modular design also supports the future integration of advanced reasoning models such as NVIDIA Cosmos Reason, reserving flexible space for functional upgrades.