AUGMENTED REALITY (AR) APPLICATIONS: TOWARDS SPATIAL COMPUTING THROUGH THE INTEGRATION OF ARTIFICIAL INTELLIGENCE AND DIGITAL TWINS
Augmented Reality (AR) technology is evolving beyond its initial entertainment applications to become a technical tool in industrial management and operations. This shift is driven by the convergence of AR, Artificial Intelligence (AI), and Digital Twins (Digital Twin), establishing the foundation of Spatial Computing – a trend being led by major technology corporations.
1. Differentiating AR, MR, and Spatial Computing
To accurately grasp this trend, it is essential to clearly differentiate between often-confused technology concepts. AR (Augmented Reality) overlays digital information onto the real world, while MR (Mixed Reality) allows virtual and real objects to interact directly with each other.
Encompassing both these concepts is Spatial Computing. This is not merely the sum of AR, AI, and Digital Twin, but a complete computing ecosystem based on three-dimensional interaction. The system architecture of Spatial Computing comprises 4 core layers:
- Sensing: Collects environmental data through the combination of LiDAR, cameras, and IoT networks (Sensor fusion).
- Mapping: Digitizes physical spaces into 3D models or Digital Twins.
- AI Inference: Analyzes context, predicts data, and processes human interactions (such as gesture tracking – gesture, and eye tracking – eye tracking).
- Rendering: Processes 3D images in real-time to respond to users.
2. Automating AR Content with Artificial Intelligence (Generative AI)
The process of creating 3D content for AR environments is being experimentally automated through Generative AI models and Large Language Models (LLM). A typical research prototype in this field is the ImaginateAR system. Instead of requiring specialized 3D programming skills, this system experiments with a pipeline that converts speech to text, generates 2D images from text, and then reconstructs them into 3D mesh models.
However, it is important to note that this technology is still in the research phase. Although models like InstantMesh can quickly create a prototype-level 3D model in a short time (under 1 minute), the quality of these objects is often basic, requiring further human refinement and not yet achieving the necessary stability for industrial production environments.
3. Scene Understanding and Visual Positioning System (VPS)
For digital objects to be accurately anchored in the real environment, the system requires Scene Understanding technology and a Visual Positioning System (VPS).
Unlike traditional GPS, which typically has meter-level error, VPS technology uses image data collected from cameras to directly compare with the point cloud of a Digital Twin, allowing for centimeter-level positioning accuracy. However, this centimeter-level accuracy is only achieved under ideal conditions: environments that have been previously 3D scanned, good lighting conditions, numerous physical features for identification, and a landscape that has not changed significantly since mapping.
4. Spatial Interface: Integrating AR and Digital Twins in Industry
In infrastructure management and Smart City domains, AR is serving as an intuitive user interface (UI) for Digital Twin systems. The use of AR provides engineers with the ability to visualize hidden structures.
Industry Practice: In the digitization project for Metro Line 1 (HCMC), contractor Portcoast utilized Autodesk software platforms (Autodesk ReCap Pro, Autodesk Revit) to convert 3D Laser scan data into Building Information Models (BIM). When combined with AR devices, on-site engineers can directly observe the layout of underground pipe networks, electrical cables, or reinforced concrete structures overlaid onto real-world coordinates, helping to minimize collision risks before excavation or construction. In the industrial sector, Siemens’ platform is also applied at the Binh Duong Smart Industrial Park, enabling simulation and management of infrastructure assets through digital models.
5. Practical Applications in Urban Management and Operations
The integration of 3D data is being practically implemented through national-level urban management projects, exemplified by Japan’s PLATEAU project (providing open 3D data for businesses to utilize).
- Disaster Prevention: 3D maps integrated with GIS are used to visualize flood levels. This data supports authorities in establishing ‘stereoscopic evacuation’ plans (calculating routes to move to higher floors of buildings) instead of merely evacuating on a 2D plane.
- Event Simulation and Broadcasting: In the entertainment sector, 3D urban data is used to organize the AIR RACE X airplane race. This is essentially a mixed reality broadcast event, using 3D data to calculate the depth and occlusion of obstacles in the real sky, rather than being a common AR application for typical mobile users.
6. Differentiating Roles in Autonomous Vehicle and Drone Coordination
In coordinating Unmanned Aerial Vehicles (Drones) and autonomous robots, it is crucial to differentiate the roles of AI and AR. In practice, drones do not use AR for movement. Control and navigation entirely depend on the machine’s perception system through AI, SLAM (Simultaneous Localization and Mapping) algorithms, and 3D maps for automatic route planning and obstacle avoidance. Meanwhile, AR serves solely as a visualization layer for humans, allowing engineers to monitor and evaluate drone flight paths via simulated control screens.
7. Conclusion
The development of Spatial Computing is progressively digitizing and integrating physical environment data into management platforms. Although limitations regarding latency, Generative 3D model quality, and lighting conditions for VPS recognition systems still exist, establishing a synchronized architecture from sensing and AI inference to AR visualization is providing quantitative tools to support effective infrastructure monitoring, predictive maintenance, and operations. For industrial enterprises, integrating an intuitive interface layer into Digital Twin systems is a necessary step to optimize workflows in the digital era.
