Human Operator Modeling
In this project my group was focused on Human Operator Modeling for industrial environments. This task required us several steps of research and practical application in 3D modeling and analysis. Below you will find the result with some explanations, split into several parts which we identified during the project. Our task was to represent a human operator in a condition of working with the industrial machine, analyze his movements in the space and estimate danger or inefficiencies. The research has been started from a general workflow, heavily used in Computer Graphics, Game Development companies and studios. This pipeline includes some steps for a creation of visually equal model which includes inverse kinematics and can be managed and moved manually, using algorithms, or special systems. We refer to this approach as ‘Human Body Surface’ creation. Therefore, this approach assumes a precise model of a human body represented as a mesh, which, when gone through the process of rigging, can be managed, and moved as a real-human body. However, this approach does not consider specific constraints of a human body and is relied on an assumption of a user that works with a particular model. From the other side, we consider more precise (also more difficult) approach for more precise implementation of a human body into a digital world. This workflow includes some specific software used by experts of ergonomics and medicine. Using this pipeline, it is possible to create and analyze a human body as a system of muscles and joints working together.
Poster
Workflow development
| INPUT: Requirements of the 3D human model |  | OUTPUT: Adjusted and refined 3D Human model |
This section will focus on the first approach to create a human model. There are software and pipelines described. Let us define general steps to create a model that can be represented in a digital environment:
- Mesh modeling, here the main mesh of the human-like model is defined.
- Scaling
- Topology optimization
- Rigging
- Exporting
I will go through some of this steps using several software packages that is used in a real-world production.
Workflow building-blocks
| Acitivities | Overview |
|---|---|
| A1 Formats and Software | • Description: Considering this approach, we can rely on years of development inside the sphere of Computer Graphics and Game Development. Both these spheres were on a frontier of the modeling not only humans, but also complex environments where people live in. It is out of the scope of this report to review if CG used mesh modelling followed by rigging before Game Development. Anyway, this approach, at the moment, is the most widespread. Therefore, during many years of experience there were a lot of software developed for creation of digital people. Some of them are proprietary, but also there are several great open-source analogs on a stage today. First, let’s describe what we want to see as a result of our pipeline. We expect to have visually and logically precise model of a human. It means that the model should look like a human, be a size of a human and textured as a human. Based on the experience of CG world, we can identify that most widespread formats in which human models can be stored are following: - .obj - geometry definition file format first developed by Wavefront Technologies for its Advanced Visualizer animation package. - .fbx - proprietary file format (.fbx) developed by Kaydara and owned by Autodesk since 2006 - .BVH - character animation file format was developed by Biovision, a defunct motion capture services company, to give motion capture data to customers. This format largely displaced an earlier format Biovision providing skeleton hierarchy information as well as motion data. - .dae - or Collada, is managed by the nonprofit technology consortium, the Khronos Group, and has been adopted by ISO as a publicly available specification, ISO/PAS 17506. COLLADA defines an open standard XML schema for exchanging digital assets among various graphics software applications that might otherwise store their assets in incompatible file formats. COLLADA documents that describe digital assets are XML files, usually identified with a .dae (digital asset exchange). - .glb/gltf – 3D data format used for representation of 3D scene in web environments. AVATAR project also focuses on the web representation using Babylon.js, so these formats I will consider separately. • Input: The requirement of a visually of the human model, 3D file format for the human model (.obj, .fbx, .BVH, .dae, or .glb/gltf). • Output: A precise 3D model of a human in a desired file formats (.obj, .fbx, .BVH, .dae, or .glb/gltf). • Control: 3D modeling techniques, the process of conversion or saving the model in the desired file format should be correctly executed, taking into account any software-specific requirements or limitations. • Resource: Software for creating 3D human models, softwares focused on Computer Graphics and Game Development |
| A2 Software Selection | • Description: All software that we will consider in this part is very known not only among specialists in CG but also to many people that used to do any 3D modelling on a computer. Let us mention several software on a stage. At first, it is necessary to mention Blender. It is open-source package that can be used for literally all the steps needed to create a model. However, the pipeline is usually spit between different software. The reason is that some of them are better for a particular step in a workflow. Of course, moving between software is a problem (it can be problem of scale, formats and other) but in a CG studio the workflow is usually a trade-off between using several programs. Let us consider some of them for each step of human modeling. This list if based on the assumptions of the author. 1. Mesh Creation. Here the leaders are ZBrush (allows to create meshes in a sculpting mode and highly tuned for it), Blender and MakeHuman 2. Texturing. Blender, MakeHuman 3. Scaling. Blender 4. Rigging. Blender, MakeHuman Most of the formats and software, as it is usually in the modern world, were created by companies to solve a particular issue. Then these formats and (sometimes) software became open-source and de-facto (sometimes de-jure) standard for the sphere. I will consider approach which is so far the best according to the speed, precision and usability for full creation of a human model. • Input: Requirement for a 3D human model that needs to be created, textured, scaled, and rigged. • Output: Selected software(s) to create a complete 3D human model. • Control: Selection of the most suitable software tools for each step in the human modeling process, • Resource: Software tools for 3D human modeling |
| A3 Creation of a generic body surface model | • Description: There are several ways to start creating a model. Starting from manual mesh sculpting, where the user is free to create any shape or detail of a human body to more general approaches using pre-defined forms and shapes. The AVATAR case is not focusing on visually highly-detailed models. So, the approach was used cover a low-precision model. For this case, the best software defined is MakeHuman. This software allows to go through all the process of model creation. Starting from a base template, user is free to change a big number of parameters using sliders. After that, there are several types of texturing available. However, you can texture the model itself after exporting it using all major formats. Also, this software provides automatic rigging and there are several topologies to use. Rigging in CG is a process of combining skeleton model and a mesh in a way that it behaves as a complete system. One of the main features of MakeHuman is a set of 4 rigging options. They vary by different number of ‘bones’ applied to the human-like mesh. This can be useful when creating models for different reasons. For example, sometimes, only general movements but for many units need to be simulated in a digital world. In this case it is better to simplify bones system and apply as many joints as possible for accurate representation of each human model. From the other side, sometimes more complex behavior is required for a model e.g facial expressions or finger movements. MakeHuman allows to create a maximum number of 163 bones. The final model created in a program can be exported in many formats.  • Input: MakeHuman software for creating a basic low-precision model and desired parameters for the model like body shape, size, proportions, etc. • Output: A generic-adjustable human-like 3D mesh • Control: The level of detail and complexity in the rigging is also adjustable, offering flexibility based on the needs of the project • Resource: MakeHuman software |
| A4 Scaling and Rigging | • Description: However, sometimes there are changes needed for a generated model. In this case all changes can be made directly in 3D modelling software. I was using Blender and Rhinoceros during AVATAR project. Rhinoceros part will be mentioned later in this report. There are many ways the model can be loaded in Blender. Many formats can be used, but sometimes problems of scale or orientation appear. For me it was easier to fix all issues manually in Blender but there are several plugins that can help to work with the model normalization. Export from MakeHuman to Blender is not straightforward but pretty easy. After defining the model, we already can have pretty all the bones and meshes out from the box. Also, Make Human allows to create initiate Pose for your model. You can also check if there is enough precision in the rigging system you have choosen.   The level of precision that you can choose for a bones sysytem can be different. For example, having 163 artificial bones, it is possible to simulate facial expressions or finger movements. Of course, this is not physically-correct representation of how human body works. However, for many tasks this approximation is bearable and is widely used in most of the modern software.   Approximations like this are the reality that is there for any Computer Graphics developments. This allows to always create needed level of precision for a scene required. For the case of visualisation, for example, it is possible to use as less bones as possible. It reduces the size of a model and also makes it easier to manage manually in programs like Blender without motion capturing. I provide a scene that is created in a short time in Blender using MakeHuman model exported as .fbx and tuned directly in Blender.   You can find some example stored as .fbx, .dae and .obj files in this folder. I used two different approaches to create a model of a male and female. Also, two models differ from each other by number of joints and by topologies. Based on the personal experience, the best format is .fbx. It creates less problems when exporting and importing between programs, also it allows to store all the data in a single file when .obj format needs to be supported by .mtl file storing materials data. I consider .glb/.gltf formats separately, because this is not a standart data transfer format for CG and modeling. Glb format was created to optimize the usage of 3D models in Web environments. Last years, web 3D was developing fast and popular engines were created to create static and dynamic 3D scenes. The leaders so far are Three.js and Babylon.js. These engines (I also call it libraries below) have many possibilities and able to work directly with .obj files. However, .glb format is optimized for working with WebGL environments. The difference between glb and gltf is in the view of the data inside. When .glb format store the data in a binary code, .gltf is using JSON representation. JSON allows to store the data in a human-readable format that can be fit into existing databases. From the other side, files stored in .glb require less space on a disk. You can find reliable explanations of working with .glb files on the website of Khronos Group. • Input: Human model generated in MakeHuman • Output: Adjusted and refined 3D Human model • Control: The user has control over the level of detail and complexity of the model, including the rigging system and the number of artificial bones. • Resource: MakeHuman, Blender and/or Rhinoceros for further adjustment and refinement of the model |
Conclusion
During the project I considered several approaches for a realistic human body modeling. First of all, I described already existing and widespread apprach that involves standard software and formats for any CGI production. Secondly, me and my collegues considered a sophisticated approach using a special ergonomic analysis software. At the end, after workung on a second case, I realized that the same functionality can be achieved working only in a 3D design programs. However, Rhinoceros is the best 3D program at the market for many workflows, it is not very known out of the field of architecture and design. So, I could only theoretically describe a system allowing to have the functionality for ergonomic analysis using 3D software extensions. This work can be continued.