Controling and vizualizing a CNC machine using Virtual Reality
The objective of this document is to present the process of creating a digital twin controlled by Virtual Reality. Also to give the main lines to realize reproduce such a process easily. At the end of this document, a general architecture that can be adapted to other case studies is presented.
Poster
Workflow development
| INPUT: CNC Machine - Digital Twin Requeriments |  | OUTPUT: VR environment allowing visualization and control of the CNC machine in real-time |
Workflow building-blocks
| Acitivities | Overview |
|---|---|
| A1 Define the objectives of the Digital Twin | • Resource:: Define the objectives of this digital twin, and what it can be used for, and know if it brings us an added value or not. For that before starting this project, discussions were held to define the objectives and the final result and what it should have in terms of functionality. Two objectives were defined for the digital twin: - Allow the control of the machine from anywhere - Visualize and control the physical twin from a VR environment • Input: CNC Machine, requirements for functionality • Output: Clear objectives for the digital twin including a remote control feature and VR visualization • Control: Validation of defined objectives • Resource : Machine operator, CNC Machine |
| A2 Analysing the physical twin, its functions and its digital interface | • Resource:: Once the objectives are defined, the next step is to analyze the physical machine and understand how it works and the interface that controle it. This approach will help on the modeling. Regarding our CNC machine, it is composed of a PLC and 3 drivers that control the axes, one motor for each axis and 2 limit Switches. In all there are 5 axes, 5 motors, 3 Drivers and 1 PLC that control all of them. All these components come from a single brand which is Beckhoff and controlled by TwinCat3 software.  • Input: Specifications and details of the CNC machine components including PLC, drivers, motors, limit switches, and TwinCat3 software • Output: Comprehensive understanding of the CNC machine’s operation • Control: CNC Machine expert • Resource : Technical documentation of CNC machine, TwinCat3 software, expertise in CNC machines and their operations. |
| A3 3D Modelling | • Resource:: 3D modeling makes it possible to build the digital twin and give it a virtual shape. It was an assembly of several hundred parts. This made it unusable on Unity 3D, because processing so many parts at the same time is one of the limitations of Unity 3D. It is necessaryto simplify and optimize this model. For this purpose: 1) Use SolidWorks to deconstruct the 3D model into individual components, distinguishing between dynamic (moving) and static (fixed) parts. 2) 3) Utilize 3DS MAX to transition the STEP file into an OBJ file. This procedure retains the original positioning and important specifications like dimensions, ensuring data fidelity. 4) As a result of this process, acquire models in OBJ format that are primed for integration within the Unity 3D framework.  As a result the model is imported into Unity 3D ready to be programmed so that it can simulate the physical machine.  • Input: 3D CAD assembly of the CNC machine, SolidWorks, 3DS MAX, Unity 3D • Output: Optimized 3D model of the machine in OBJ format, ready for simulation in Unity 3D • Control: 3D machine model support in Unity 3D • Resource : Modelling in SolidWorks and 3DS MAX for CAD, CNC machine structure and functions |
| A4 Kinematic layer | • Resource:: After having the model, it is necessary to add a layer of kinematics, which allows to simulate the movements of the physical machine. In order to implement the kinematics, it is necessary to use C#. The movement axes are the X, Y and Z axes and the rotation axes also Y, Z. For this purpose, a function has been created for each axis to calculate translations and rotations. Kinematic result • Input: Optimized 3D model in Unity 3D, • Output: Kinematics layer implementation that allows to accurately simulate the movements and rotations of the physical machine • Control: Kinematics accuracy virtual to real • Resource : Unity 3D software, C# scriptinh, understanding of the kinematics involved in the CNC machine operation. |
| A5 Establishing a connection between the PLC and the Digital Twin | • Resource:: In this scenario, machine operations are managed through TwinCat3 software, a tool that allows programming the PLC and triggering specific inputs to execute tasks, such as driving axis movements. To control the machine, a communication line must be established with this software. This interaction facilitates the exchange of data, allowing the machine to interpret and execute the commands transmitted. To achieve this, it is necessary to develop a C# script in Unity 3D. The function of this script is not only to send data, but also to understand the corresponding response using the TCP/IP protocol.  The communication between TwinCat3 and Unity 3D is established. But the problem is that in this way the unity client can only control one machine in a localhost. So to be able to control more than one machine from the same place or remotely, it is necessary to add a server, from which it is possible to control it and visualize it in VR. One solution is to use Mirror, Mirror is a Unity3D library that allows to create servers and clients in Unity 3D with the possibility to make them communicate in real time without losing data. • Input: Virtual machine with kinematics layer • Output: Real-time communication between Unity 3D and TwinCat3 allowing remote control and visualization of multiple CNC machines in a VR environment • Control: Real-time communication with minimal latency • Resource : TwinCat3, Unity 3D and Mirror |
| A5 Virtual Reality Implementation | • Resource:: The final stage incorporates VR to control and visualize the machine. This step requires VR hardware and the SteamVR platform, which drives the VR equipment. SteamVR also provides the additional libraries to create the VR experience, and this coupled with the specific code developed for the control and interactions allows for detailed visualization of the machine and precise execution of the commands to control it. All these activities were based on this architecture.  • Input: Unity enviroment to control the Machine Final result • Output: VR environment allowing visualization and control of the CNC machine in real-time • Control: Successful visualization of CNC machine in VR and accurate execution of control commands in the VR environment • Resource : Unity 3D, VR hardware, SteamVR platform |
Results
Following the proposed workflow, we were able to start this case study, with only one physical machine and here we are with the beginning of a digital twin ready to be used.
The following video justifies the result.
Conclusion
Creating a digital twin of a machine requires going through several steps as we have proposed above. Integrating Virtual Reality has a very important added value, as it should allow to go further by having a more realistic perspective and user experience.
Avatar for me
AVATAR, represents for me an unprecedented experience in the field of industry 4.0. An experience full of added value, collaboration and learning. During this project, I had the opportunity to work with people with exceptional professionalism. Their knowledge in the field of XR technologies allowed me to learn a lot in a short time. Since the beginning of the avatar project, I was autonomous, which gave me the possibility to go and find the best solutions among others on each step. I was involved in the creation ossf a virtual reality scene from existing 3D CAD models, understanding then how to animate them and how to build virtual environment. My activities for this case study focused on experimenting with different tools and file formats. I discovered many tools as well as programming languages and software. After all this work, I think that we have reached the goals we have set with a lot of effort and commitment.