Digital Simulation-based Field Scenario Creation for Industrial Quality-, Efficiency- or Safety-related AI Solutions

Introduction

Recently, data-driven artificial intelligence (AI) technology has gained popularity in the Chinese market as a way of helping customers improve product quality, increase production efficiency, and identify safety risks in customized industrial solutions. However, some field-related issues that Hitachi and customers have together faced have always impeded the progress of AI solutions. These issues include (1) a lack of field data, particularly for some negative data samples; (2) expensive field development and testing costs; and (3) a lack of practical demonstration tools for illustrating solution values, especially in the early stages of a project or a solution. To support Hitachi's business of industrial AI solutions in China, Hitachi China Research Laboratory developed "digital simulation-based field scenario creation" technology as an all-phase assistance tool. This technology was applied for different stages of various industrial AI solutions, including quality-, efficiency- or safety-related projects in China with some degree of commercial success.

Methodology and Use Cases

Data Dilemma

For the purpose of AI model training, testing, and field deployment, customized AI solutions inevitably require a large volume of field data. Thus, some AI projects cannot progress without field data and a field environment. At the same time, AI solution providers also are unable to effectively showcase their solutions prior to obtaining field data and accessing to field environment. Nevertheless, it is difficult in many practical projects to satisfy these conditions, especially in the early stages of a project. Our experience shows that this situation impedes the progress of numerous potential projects in very early stages. In other words, without data, solution providers are unable to create their solutions, while without the solution, customers are unable to provide their field data, called a data dilemma.

Methodology

We aim to solve this data dilemma by creating field data and field scenarios through digital simulation to support customized AI solutions. Our field scenario creation technique can generate realistic field data and create a reasonable field environment or process simulation by integrating advanced digital techniques including 3D scanning, AI-generated content (AIGC), digital twins, and digital simulation. This can provide data for AI model training, process simulation for development and testing, and an effective visualization tool for demonstrating the effect of AI solutions in the initial or early stages of a project. We offer three use cases to demonstrate its efficacy in different phases of customized AI solutions.

Use Cases

Product Quality Inspection Solution

Since identifying defective products is a crucial step for controlling product quality, many visual product quality inspection systems with AI algorithms are used to discover product defects. Nevertheless, collecting sufficient data for AI models is very expensive, particularly when it comes to negative (or defective) samples. To overcome this difficulty, 3D digital simulation¹⁾ and AIGC were applied in a “product quality inspection solution” for a manufacturing company to generate unlimited realistic data samples for enhancing the efficacy of AI algorithms during the system development phase. The visual realism of the generated data with two open datasets NEU-DET²⁾ and MVTec³⁾ is depicted in Figure 1.

Figure 1—Data Generation on Open DatasetsFigure 1(a) shows NEU-DET, and 1(b) shows MVTec.

Warehouse Operation Analysis Solution

We recently offered a leading logistics company in China a “warehouse operation efficiency improvement solution” where field workers' status and activity are sensed through the use of sensors (smart bands) to help the warehouse optimize its work procedures. The customer was unable to fully understand our solution and envision its efficacy during the early discussion. Following a brief field investigation, we swiftly created a simulated warehouse field, as shown in Figure 2, to illustrate the values of our solution without the physical involvement of this warehouse, which helps in customer persuasion. Figure 2 (a) is a simulated warehouse field where the working procedure can be configured for finding the optimized ones. In addition, we also use this simulation to assist us in collecting sensor data during the following actual Proof of Concept (PoC) phase. Figure 2 (b) is a simulated worker’s operation in which sensing data is also virtually generated for AI model training.

Figure 2—Scenario Simulation and Data Generation on Warehouse Operation Analysis SolutionFigure 2(a) shows simulated warehouse field creation, and 2(b) shows sensor data generation.

Factory Security Improvement Solution

Currently, we are providing a video analysis-based “factory security improvement solution” to a top brewery enterprise in China, mainly detecting the violation operation in high-place (checking whether workers wearing or utilizing some fall-protection devices in high-place, such as safety belts or hangers). However, the customer raised concerns about our solution's capability to function effectively in certain challenging conditions due to their strict requirements on security regulations. As shown in Figure 3 (a), the majority of high-place works occur in normal environments, whereas a small part occurs in extreme conditions, such as visual obstacles by rainy or snowy weather, or illumination variation at sunset time. This unbalanced data distribution is also called long-tail distribution⁴⁾, which is widely observed in industrial or natural contexts. To dispel doubt, we simulate a variety of field conditions, as shown in Figure 3(b) snowy weather, (c) rainy weather, and (d) sunset time, and verified our AI systems on them. This not only helps us convince customers of the usefulness of our AI solution, but it also produces a large number of useful negative samples that are used to improve our AI algorithm in the system development phases.

Figure 3—Extreme Condition Scenario Simulation on Factory Security Improvement SolutionFigure 3(a) shows the long-tail distribution, 3(b) shows snowy conditions, 3(c) shows rainy conditions, and 3(d) shows sunset conditions.

Conclusions

Hitachi China Research Laboratory proposed "digital simulation-based field scenario creation" technology as a means of advancing Hitachi's industrial AI solutions in China. This all-phase assistance tool supports us at different project stages, including data collection, system development and testing, and value demonstration. From the business perspective, this technology is also successful. Using this approach, Hitachi succeeded in winning a contract for the “factory security improvement solution,” which is now being deployed in 11 factories around China. The contract was for approximately 5M CNY. Hitachi China Research Laboratory intends to keep broadening the use of this approach and bringing down its cost to promote Hitachi's business well and bring extra value to Chinese industrial customers.