Link Search Menu Expand Document

Physical and Digital Twin

In this use case, the objective was to make a physical prototype of the robot arm piloted by its twin in virtual reality. The main goal is to control the robot arm in different ways as manual or virtual ways.


Poster

Poster


Workflow development

 
CONTROL:
Physical part assembly - Accuracy and responsiveness of the comunication - effectiveness of robot arm control via Simulink - Precision of manual control - Accuracy of angle instructions from VR
 
INPUT:
CAD Models robot arm - servomotors
scenecreation OUTPUT:
A robot arm controllable through the Unity interface in VR
 
RESOURCE:
SolidWorks - 3D printer - Servomotors - MATLAB Simulink - Arduino - Unity 3D
 

Workflow Description

Acitivities Overview
A1
Redesign and production
Description: The redesign and production of the real prototype of the robot arm have commenced using CAD models on Solidworks. Modifications to the CAD models are required to properly integrate the servomotors into the arm’s links. In total, there are six servomotors needed in this prototype. Then, thanks to the use of a 3D printer, each part of the robot arm is efficiently and cost-effectively produced for testing purposes.
Input: CAD models, servomotor specifications (dimentions)
Output: Adjusted CAD models, 3D printed for robot arm prototype assembly
Control: Part assembly and fitting of servomotorss
Resource: SolidWorks, 3D printer, servomotors
A2
Physical-Digital connection
Description: A critical point is the connection between the physical and digital prototype. A connection was designed between MATLAB Simulink and Arduino, which represents an interface card for the communication between the robot and the Simulink project. With this software, the robot is intended to be correctly controlled in both ways thanks to the communication between Simulink and Unity through the local server.
Input: The physical robot prototype, digital twin in Simulink
Output: A bidirectional communication link between the physical and digital robot arm
Control: Accuracy and responsiveness of the comunication
Resource: MATLAB Simulink, Arduino, Robot arm prototipe
A3
Multiphysics model
Description: It is necessary to build the multiphysics model to respect the requirements. The work focused on the connection of the angular instructions with the virtual model and then to establish the link between Arduino and Simulink, to prepare the manual and virtual control of the robot. The advantage offered by Simulink is to propose the entire Arduino library in download. Thus, it is possible to use the Arduino IDE functions in MALTLAB Simulink if necessary. Simulink can recognize these functions servoRead and servoWrite when they are used. The goal is to replace the virtual robot in Simulink with the real one: when we integrate real parts or motors into a Simulink project, it is called Hardware-in-the-loop. The servo motors are linked to the Arduino board on different pins. Simulink recognizes the Arduino board thanks to the compilation of a .PDE code on the board, which allows to use the Arduino library in Simulink.
Input: Arduino functions and library, robot arm prototipe, MATLAB Simulink project
Output: Arduino-integrated Simulink project that can interact with the robot arm prototipe
Control: Accuracy of Arduino-Simulink integration, effectiveness of robot arm control via Simulink
Resource: MATLAB Simulink, Arduino library
A4
Manual control
Description: By implementing the Arduino library and connecting angular instructions to the servo motor pins, it is possible to manually control the robotic arm using a block parameter interface.
Input: Servomotor pins, angle instructions, block parameters interface
Output: A manually controllable robot arm
Control: Precision of manual control
Resource: MATLAB Simulink, Arduino library
A5
Virtual control
Description: The major point of data communication in the project is the two computers that each host a project, one the Simulink project and the other the Unity project. It should be noted that the ports for sending and receiving data must be changed depending on the IP address of the two computers. The objective now is to adapt the manual control to the virtual control. It is necessary to modify the input data interface for the real prototype. The Operator parameter block needs to be converted into a virtual interface in Unity. The angular instructions will come directly from Unity. It is possible to get the data sent by Unity for the angle instructions, after having informed what angle in unity is required. Thanks to 6 different levers, it is possible to choose the angle to be sent to each servomotor and then send the values to the Simulink Project, which will send the converted values to the Arduino and move the servomotor. Now the virtual robot in Unity reproduces the behavior of the real robot with the instructions of the angles and not the real angles with errors certainly.
Input: Unity project, VR headset, control interface, command for angles
Output: A robot arm controllable through the Unity interface in VR
Control: Accuracy of angle instructions from VR, communication latency
Resource: MATLAB Simulink, Unity 3D, Arduino

Results

We can control the real robot thanks to the virtual interface in Unity easily. Two improvements can be made on the robot : to make a more intuitive interface (with joysticks) to facilitate handling of the robot in virtual reality and to establish automation of the data transmission because it could be a requirement of the industrial specifications ; to use a system of recovery of the current position (angle) of the servomotors. Rotary encoders allows to give a feedback of the position of a servomotor by positioning then along the axis of the servomotor concerned. Thanks to Arduino, we can collect this data and send it to Simulink which establish the connection with Unity. Then, Simulink send the date to the input of the virtual servomotor and by this way, the virtual robot in Unity will reproduces the behavior of the real prototype with potential errors of angle (exchange of information).

Conclusion

We can show several uses as manipulation of a robot without any risk for the user for instance because it is remote handling of a robot. It opens to other possibilities as automation and programming series of movements for a use in industrial production line. It can become a low cost remote simulator.


Avatar for me

Avatar project was a great experience for me, over all thanks to the collaboration between nations and projects. I havelearnt about XR technologies and their applications in several fields of the engineering or the industry. I will reuse this experience in my professional career because I think it was a relevant opportunity in my student life.