Physical and Digital Twin
In this use case, the objective was to make a physical prototype of the robot arm piloted by its twin in virtual reality. The main goal is to control the robot arm in different ways as manual or virtual ways.
Poster
Workflow development
| INPUT: CAD Models robot arm - servomotors |  | OUTPUT: A robot arm controllable through the Unity interface in VR |
Workflow Description
| Acitivities | Overview |
|---|---|
| A1 Redesign and production | • Description: The redesign and production of the real prototype of the robot arm have commenced using CAD models on Solidworks. Modifications to the CAD models are required to properly integrate the servomotors into the arm’s links. In total, there are six servomotors needed in this prototype. Then, thanks to the use of a 3D printer, each part of the robot arm is efficiently and cost-effectively produced for testing purposes. • Input: CAD models, servomotor specifications (dimentions) • Output: Adjusted CAD models, 3D printed for robot arm prototype assembly • Control: Part assembly and fitting of servomotorss • Resource: SolidWorks, 3D printer, servomotors |
| A2 Physical-Digital connection | • Description: A critical point is the connection between the physical and digital prototype. A connection was designed between MATLAB Simulink and Arduino, which represents an interface card for the communication between the robot and the Simulink project. With this software, the robot is intended to be correctly controlled in both ways thanks to the communication between Simulink and Unity through the local server. • Input: The physical robot prototype, digital twin in Simulink • Output: A bidirectional communication link between the physical and digital robot arm • Control: Accuracy and responsiveness of the comunication • Resource: MATLAB Simulink, Arduino, Robot arm prototipe |
| A3 Multiphysics model | • Description: It is necessary to build the multiphysics model to respect the requirements. The work focused on the connection of the angular instructions with the virtual model and then to establish the link between Arduino and Simulink, to prepare the manual and virtual control of the robot. The advantage offered by Simulink is to propose the entire Arduino library in download. Thus, it is possible to use the Arduino IDE functions in MALTLAB Simulink if necessary. Simulink can recognize these functions servoRead and servoWrite when they are used. The goal is to replace the virtual robot in Simulink with the real one: when we integrate real parts or motors into a Simulink project, it is called Hardware-in-the-loop. The servo motors are linked to the Arduino board on different pins. Simulink recognizes the Arduino board thanks to the compilation of a .PDE code on the board, which allows to use the Arduino library in Simulink. • Input: Arduino functions and library, robot arm prototipe, MATLAB Simulink project • Output: Arduino-integrated Simulink project that can interact with the robot arm prototipe • Control: Accuracy of Arduino-Simulink integration, effectiveness of robot arm control via Simulink • Resource: MATLAB Simulink, Arduino library |
| A4 Manual control | • Description: By implementing the Arduino library and connecting angular instructions to the servo motor pins, it is possible to manually control the robotic arm using a block parameter interface. • Input: Servomotor pins, angle instructions, block parameters interface • Output: A manually controllable robot arm • Control: Precision of manual control • Resource: MATLAB Simulink, Arduino library |
| A5 Virtual control | • Description: The major point of data communication in the project is the two computers that each host a project, one the Simulink project and the other the Unity project. It should be noted that the ports for sending and receiving data must be changed depending on the IP address of the two computers. The objective now is to adapt the manual control to the virtual control. It is necessary to modify the input data interface for the real prototype. The Operator parameter block needs to be converted into a virtual interface in Unity. The angular instructions will come directly from Unity. It is possible to get the data sent by Unity for the angle instructions, after having informed what angle in unity is required. Thanks to 6 different levers, it is possible to choose the angle to be sent to each servomotor and then send the values to the Simulink Project, which will send the converted values to the Arduino and move the servomotor. Now the virtual robot in Unity reproduces the behavior of the real robot with the instructions of the angles and not the real angles with errors certainly. • Input: Unity project, VR headset, control interface, command for angles • Output: A robot arm controllable through the Unity interface in VR • Control: Accuracy of angle instructions from VR, communication latency • Resource: MATLAB Simulink, Unity 3D, Arduino |
Results
We can control the real robot thanks to the virtual interface in Unity easily. Two improvements can be made on the robot : to make a more intuitive interface (with joysticks) to facilitate handling of the robot in virtual reality and to establish automation of the data transmission because it could be a requirement of the industrial specifications ; to use a system of recovery of the current position (angle) of the servomotors. Rotary encoders allows to give a feedback of the position of a servomotor by positioning then along the axis of the servomotor concerned. Thanks to Arduino, we can collect this data and send it to Simulink which establish the connection with Unity. Then, Simulink send the date to the input of the virtual servomotor and by this way, the virtual robot in Unity will reproduces the behavior of the real prototype with potential errors of angle (exchange of information).
Conclusion
We can show several uses as manipulation of a robot without any risk for the user for instance because it is remote handling of a robot. It opens to other possibilities as automation and programming series of movements for a use in industrial production line. It can become a low cost remote simulator.
Avatar for me
Avatar project was a great experience for me, over all thanks to the collaboration between nations and projects. I havelearnt about XR technologies and their applications in several fields of the engineering or the industry. I will reuse this experience in my professional career because I think it was a relevant opportunity in my student life.