A review of heuristic optimization techniques applied for 3D body reconstruction from anthropometric measurements

Reconstructing 3D human models has a variety of applications in areas such as entertainment, medical, manufacturing, and design. Reconstruction techniques are classified based on characteristics such as data input, devices used, and algorithms employed, in which using anthropometric measurements is one of the most widely used methods. Traditional methods of 3D human reconstruction from anthropometric measurements rely on technologies like Convolutional Neutral Network (CNN), and Linear Regression to generate an accurate model in a reasonable amount of time. This paper presents a picture of heuristic optimization methods to find the optimal solution in 3D body reconstructions from anthropometric measurements. In terms of output accuracy, the methods discussed in this paper have the potential to outperform CNN and similar technologies. Results are verified and validated on a real dataset to evaluate the performances of each method.


Introduction
*Due to the global impact of Coronavirus disease , almost all industries are actively transforming their business model to survive and thrive after this unprecedented crisis. One of the most adversely affected sectors is the apparel and fashion industry. Offline retailers and departmentstore chains had already seen massive declines in sales and traffic during the lockdown period. As more consumers find themselves at home scrolling mobile phones and social media, e-commerce is booming into a dominant shopping channel. Fashion companies are quickly trying their best to drive consumers online and rapidly scale e-commerce operations. However, as in online shopping, consumers can not physically try products; the size issues and doubts about their actual appearance are likely to challenge online sales. The term "3D virtual try-on" comes in this regard. This technology allows consumers to try on products without touching them, offering a personalized and immersive shopping experience. Hwangbo et al. (2020) have tested and proved the effectiveness of 3D virtual tryon in boosting sales and decreasing products' return rate.
The main idea of Virtual Try-on technology is to create a 3D virtual model that simulates customers' body shape, and customers can then use "their" model to try-on, mix-and-match, and decide whether a garment looks good and fits them well. There are different technologies developed for Virtual Try-on technology, such as employing 3D scanners, applying machine learning techniques, and using anthropometric measurements. In which, 3D human reconstruction using anthropometric measurements could satisfy three criteria: (1) a method that is easyto-implement, easy-to-use, serving for real applications; (2) a method that protects the privacy of users' personal information; (3) a method that is affordable and accessible for common customers.
To reshape the 3D human body from anthropometric measurements, Zeng et al. (2017) from Sun Yat-sen University introduced a method using a feature-selection-based local mapping technique. First, the authors employ a module called "Imputer" to estimate anthropometric parameters that users do not remember when entering data. The obtained data is then passed to the "Selector" module for mesh calculation, and lastly, the "Mapper" module is used to synthesize a final 3D human body mesh for users. The key point here is using the "Selector" module, trained on the dataset published by Yang et al. (2014). The dataset contains 1531 female and 1517 male meshes, which have the same resolution. However, the weakness of this method is that it requires users to input a large number of parameters (19 parameters, even if they have an "Imputer" to preprocess missing data), and it does not contain the details in realistic localized shape. Wuhrer and Shu (2013) proposed another approach that combines a linear learned correlation with a non-linear optimization to extrapolate the shapes with local variation not presented in training data. The first step is to find a realistic shape describing the measurements and then use a quasi-Newton optimization to deform locally. Their method's efficiency has been demonstrated using both synthetic data and real human measurements. Apart from the above methods, Streuber et al. (2016) developed a new method that focuses on increasing processing speed while maintaining acceptable accuracy in shape generation. The main idea is to have a matrix learned and then employ that matrix to create a mapping between identified body space and geometric body space. Body descriptor words (fit, small, long, skinny, and lean) represent the identified body space. The matrix is created using linear regression and an SMPL-generated dataset. Pujades et al. (2019) contributed to the field by developing a software application for quickly generating 3D body shapes using measurements based on linear regressors. The research's core part is finding a reliable set of body measurements (including Overall Height, Arm Span Fingers, Inseam Height, Hip Width, and Arm Length) to generate a realistic body shape. This approach, however, has a problem with output accuracy since it produces unnatural body shapes and could not be applied in industries that demand great precision. Nguyen and Hoang (2021) suggested a method that uses Diversity Control Oriented Genetic algorithm to find the optimum solution for body shape generation. The research results are evaluated and validated on both synthetic and real data to show the method's usefulness. However, their study did not discuss other aspects of heuristic optimization.
Concerning the ease of implementation and the applicability for non-linear mathematical issues, in this paper, we choose to investigate a collection of heuristic optimization methods to discover the optimal solution for human shape generation. The main contributions of this paper are to (1) Summarize techniques for finding an approximate solution; (2) Analyze the advantages and disadvantages of each heuristic method in generating an optimal 3D body model, (3) Apply heuristic optimization methods to a real dataset and evaluate their accuracy and time consumption. This research is divided into four main parts: Section 2 summarizes the parametric model used, the generally applied optimization model, and problems that must be solved. A review of heuristic methods, including (1) Simulated Annealing, (2) Particle Swarm Optimization, (3) Genetic Algorithm, and (4) Diversity Control Oriented Genetic Algorithm is provided in Section 3. Section 4 contains an evaluation and analysis of the results. Section 5 concludes the paper and opens future works.

Overview of 3D body reconstruction using optimization method
This part is organized as follows: (1) Summarize the parametric model used, (2) Demonstrate the fitness function and constraint when using optimization methods.

Parametric model (SMPL model)
This research uses a parametric model-The Skinned Multi-Person Linear (SMPL) model (Loper et al., 2015). A template model is deformed to generate approximate bodies by controlling the shape and pose parameters. The largest commercial dataset (Caesar) contained approximately 4000 human bodies with different shapes and poses, both males and females, and was used to develop and create SMPL. The body was deformed using a global vertex-based method applied to 3D displacements directly to vertices. Vertices are mapped by shape and pose parameters of SMPL model ( ⃗ , ⃗ , Φ): ℝ | ⃗ ⃗⃗ |×| ⃗⃗⃗ | ↦ ℝ 3 . The SMPL templatebased model is a pre-defined topology with 6890 vertices and the type of surface mesh is a triangle. The vertices ∈ ℝ 3 are described linearly by as follows: with : Shape parameters, : Pose parameters could be variation; : Rest template, : Weight, joint locations, and blend shapes/blend poses are learned from the Caesar dataset. Fig. 1 illustrates the general process of applying the heuristic method for 3D human model reconstruction using anthropometric measurements.

Methodology
Input parameters are nine expected body measurements of users, and output is a 3D model with estimated equivalent measurements. The evaluation function ( ) is created based on the loss function between the input parameters and the estimated measurements which is created from the shape parameter ( ) of the parametric model.
where -the expected parameter of measurements and ̂-the estimated parameter of measurements created from shape parameters. Based on the above-mentioned characteristics of the parametric model, the constraints of are defined in the range [−3, 3]. The process of setting up an initial value and hyperparameters varies is depended on each optimization method and will be discussed in Section 3. The pseudo-code (Algorithm 1) indicates basic implementation steps. If terminating condition is satisfied then 6 STOP 7 End if 8 Generate from optimization method 9 ← 10 End while Table 1 indicates the relationship between linear measurements and volume measurements. The linear measurements and shape parameters have a linear relationship and are approximately by linear regression as Pujades et al. (2019). However, it poses a significant challenge for circumferences such as chest, waist, pelvis, and thigh with a non-linear relationship.

Heuristic optimization in detail
This section discusses four heuristic optimization methods, including Simulated Annealing, Particle Swarm Optimization, Genetic Algorithm, and Diversity Control oriented Genetic Algorithm. The history, main algorithms, and implementation process of each method are described in detail.

Simulated annealing
Simulated Annealing Optimization (SA) was first defined by Kirkpatrick et al. (1983). It is based on the philosophy of physical annealing: A material is heated up until its molecular structure weakens and changes. The temperature of the material is cooled down after archiving the desired structure. The simulated Annealing method mimics the annealing process by applying the Thermodynamic equation. It is used to find the global optimal while avoiding the local. In fact, there are two main factors directly affecting Simulated Annealing: (1) Temperature, which includes the initial and final value for reduction progress, (2) how to choose neighbor candidates after each iteration. Based on the above algorithm, Simulated Annealing provides two key benefits: (1) Simple to implement into an issue and deploy to a large number of applications, (2) Avoids being stuck at the local minimum. However, it has several disadvantages because of the algorithm itself. Finding an optimal solution could take a long time, and many hyperparameters require experience to adjust. The process (Algorithm 2) describes how to apply Simulated Annealing optimization to this issue.
As previously stated, two critical aspects, namely (1) the neighborhood of candidate value and (2) the process to decrease the temperature, would be selected carefully to ensure the issue's requirement. Firstly, values of the candidate are generated as a normal distribution in the range [−3, 3] which is similar to the variety of shape parameters . To identify the neighborhood value of candidates after each iteration, a condition of the vicinity is created as follows: ( , ) = { ∈ ℝ: | − | < }, = 3 , is adjusted parameter, the higher the lower of convergent and vice versa. However, if is too small, the moving process could be fast enough to pass over the global optimal. For temperature reduction, a linear rule is applied as below: = high − ( ℎ ℎ − ) * . The candidate with a worse value than the global value is selected based on the probability = − . If is much larger compared to , a new candidate is frequently updated with a higher probability and vice versa. After investigating and surveying the space solution, the values of ℎ ℎ and are chosen respectively as: 0.005 and 0.

Particle swarm optimization
Particle Swarm Optimization (PSO) is similar to other evolutionary computation techniques. It was first announced by Kennedy and Eberhart (1995) based on the social behavior of birds or fish: A large number of individuals in a swarm and the method they corporate to quickly find food in an area. The particle of Particle Swarm Optimization plays a similar role as birds in a swarm. Each particle has a relevant velocity and is updated in both magnitude and direction via personal and social information. Near to SA, the advantage of PSO is finding the optimal global solution. However, the latter requires fewer hyperparameters used to customize, and the problem is not differentiable. There are two main factors in PSO, including (1) Determining the coefficient for exploitation and exploration of the particle after each iteration, and (2) Defining the random initial value.
Similar to the Simulated Annealing algorithm, the normal distribution is used to initialize the value of particle and velocity of each particle respectively in [−3, 3] and [− 3 , 3 ] with is the customized variable.
The inertia ( ), cognitive ( 1 ) and social ( 2 ) coefficients would represent exploitation and exploration of particles to find the best solution. In this research, adaptive hyperparameters are applied to update the coefficients over the iterations as follows: = − ( − ) * , and 1 = −3 + 3.5, 2 = 3 + 0.5 (according to the research by Sermpinis et al. (2013)). The purpose of this process is to increase the exploration at the beginning with the strong initial value of 1 and , the weak initial value of 2 and to exploit the global optimal by converging towards the best results by decreasing 1 , and increasing 2 .

Genetic algorithm (GA)/diversity control oriented genetic algorithm (DCGA)
Genetic Algorithm (GA) was one of the first developed heuristic optimization methods in 1950 and became well known by Professor John Holland (Holland, 1975) after his book "Adaptation in Natural and Artificial Systems" was published. It is inspired by Charles Darwin's theory of natural selection. Like other meta-heuristic methods, the best candidate or fittest individual (Genetic Algorithm) would be chosen after a large number of iterations. There are five main phases which are known (1) Initial Population, (2) Objective function, (3) Selection, (4) Crossover, and (5) Mutation.
The strong points of the Genetic Algorithm are: (1) It can solve a wide range of problems; (2) it can be implemented with both discontinuous and continuous parameters; (3) it does not require derivatives. However, a time-consuming process is a significant barrier to deploy a Genetic Algorithm, and results could be trapped in local optimal. Parallel implementation and customizing the initial population could handle a part of the issues. Diversity Control Oriented Genetic Algorithm (DCGA) is a variant of the Genetic Algorithm developed to improve the latter's weaknesses. First announced in by professor Hisashi Shimodaira in 1997 (Shimodaira, 2001), Diversity Control Oriented Genetic Algorithm creates diversity for the population and updates the generation using a variable called "hamming distance" to select a better candidate for the next generation.
Algorithm 4: Genetic algorithm 1 Generation of the initial population 2 Repeat 3 Calculation of fitness 4 Selection 5 Crossover 6 Mutation 7 Until terminating condition is satisfied.
To apply Genetic Algorithm and Diversity Control Oriented Genetic Algorithm to this issue, the initial population is generated based on the normal distribution as the above algorithm with a range of shape parameters as [−3, 3]. This research uses Laplace crossover and exponential mutation as ref (Deep and Thakur, 2007a;2007b) for Genetic Algorithm and Diversity Control Oriented Genetic Algorithm. The detailed implementation is available in Nguyen and Hoang (2021). Different from the reproduction process, the selection process of the two methods would be divided into two strategies. A popular selection known as Tournament is chosen to apply to the Genetic Algorithm to select individuals from a population. The selection in Diversity Control Oriented Genetic Algorithm, on the other hand, is divided into three primary steps: (1) Eliminating duplicated individuals in the population, the "duplicated individuals" are defined as follows: 1 , 2 individuals have: | 1 − 2 | ≤ ∀1 ≤ ≤ 10 where 1 ∈ 1 , 2 ∈ 2 , (2) Cross-generational Probabilistic Survival Selection (CPSS) method is used to select individuals. After arranging individuals in ascending order of fitness function's value, the process to select the first individual and the next ones is defined by the following equation: where ℎ is the hamming distance between a candidate individual and the individual which have corresponding genes satisfied: | 1 − 2 | > , 0 < ≪ 1; is the number of genes in an individual; and are the coefficients for shape and exponent whose values are in the range of [0, 1].
(3) After step (2), if the number of individuals is smaller than , new individuals will be generated randomly in the initial population.

Dataset and configurations in detail
The dataset used for validation and verification in this paper is collected by Viettel Military Industry and Telecoms Group (Vietnam). This dataset was generated as part of a project to build a body database of Vietnamese people aged 18-65, living in Hanoi and Ho Chi Minh City, the two largest cities in Vietnam. These cities have about 20 million people, accounting for 20% of Vietnam's total population. The population density is approximately 1000 people per km 2 with thousand business sectors. Data collection is collected using representative samples for gender, mass, occupation, religion, and ethnicity to ensure the distribution and diversity of the population in Vietnam. The database has around 900 file samples, including: (1) object files in the obj format, (2) the anthropometric measurements of each sample, and (3) 2D images of the front view and side view of each sample.

Convergence
As the process mentioned above, the selection of hyperparameters for heuristic methods is approached via Appendix A. It includes two main factors: (1) the average error on the dataset both in males and females, (2) the meantime processing. The results indicate that the processing time is inversely proportional to the accuracy for Simulated Annealing, Diversity Control Oriented Genetic Algorithm, and Particle Swarm Optimization; only the Genetic Algorithm has a minor difference, but the variation is only approximately 0.6%. According to Appendix A, for Diversity Control Oriented Genetic Algorithm and Genetic Algorithm, the difference in the best value and the worst value is 16% and 20%, respectively, and the distances in time processing are 0.3% and 0.6% at the same time. In contrast, Simulated Annealing brings an obvious difference between time and accuracy with 259% and 8%, respectively. Particle Swarm Optimization gives a balanced result with changes of around 1% and 8%. Fig. 2 depicts the convergence time of four methods and Appendix A considers the dependence of them on hyperparameters. It can be seen that Simulated Annealing is pretty straightforward to implement and less dependent on hyperparameters changes; however, its convergence time is rather slow. The Genetic Algorithm is highly reliant on the hyperparameters to produce results and quickly converge to the local optimal. Diversity Control Oriented Genetic Algorithm improves GA's performance, allowing faster convergence and better results thanks to diversifying population after the selection process but this method still rapidly converges to the local optima. From the study's experiment, PSO outperforms the remaining methods as it is simple to conduct, less reliant on hyperparameters, and provides optimal value closest to the global optimal value.

Comparison of time consumption and accuracy between Heuristics methods
First, we investigate the accuracy of nine input anthropometric measurements, including overall height, shoulder breadth, chest circumference, waist circumference, arm length, inside leg length, and thigh circumference. Figs. 3a and 3b illustrate the average errors in these nine measurements. Overall, the Genetic Algorithm produces the least accurate output in both males and females, with the mean errors of roughly 6.36mm and 5.64mm, respectively, nearly double 2.62mm and 3.53mm of Particle Swarm Optimization, which gains the lowest errors among the four methods. Diversity Control Oriented Genetic Algorithm and Simulated Annealing rank second and third with mean errors around 5mm and 4mm, respectively. However, to process time, it can be seen that the more accurate the output, the longer the processing time. In particular, while the Genetic Algorithm has the highest mean error, its processing time is the fastest (about 320.24s). Meanwhile, the most accurate methods, Particle Swarm Optimization and Diversity Control Oriented Genetic Algorithm require the longest time to process (362.7s and 392.88s, respectively). Simulated Annealing maintains a relatively stable ranking of two criteria.
After optimizing 3D human models using four methods, we continue measuring the remaining anthropometric measurements as wrist circumference, forearm circumference, neck width, burst height, etc. Overall, the mean errors of the remaining measurements are 1.5 to 2 times higher than those of input measurements, with 6.68mm for males and 8.57 for females. Interestingly, the rankings of the four methods are completely opposed. If the Genetic Algorithm has the highest mean error in input measurements, the same is true for Particle Swarm Optimization in the remaining measurements. Its most inaccurate output is hip height, which has a mean error of up to 16.5mm. Performance rankings of Diversity Control Oriented Genetic Algorithm and Simulated Annealing remain unchanged on two types of measurements.
Taking all measurements into account, there is no significant difference in mean errors between the four heuristic methods. All differences are in the 6-7mm range for males and 5-6mm for females.
Research results also show a slight gap in the mean errors of linear measurements and volume measurements. In general, four methods performed better for length than circumference: Linear measurements have mean errors of 6.41mm for males and 5.06 for females, compared to 6.71mm and 7.07mm of volume measurements. More specifically, four methods calculate the model height quite accurately, resulting in the lowest mean error for overall height. The opposite is true for forearm circumference with mean errors up to about 10 mm in male and female datasets.
Regarding gender difference, Diversity Control Oriented Genetic Algorithm and Genetic Algorithm always perform more precisely in male models than female models, whereas Particle Swarm Optimization and Simulated Annealing fluctuate in input measurements and other measurements. However, it could not conclude which gender has the advantage in 3D human reconstructions from anthropometric measurements using heuristic optimization because the mean errors of male and female models are only about 1mm apart (6.67mm and 5.59mm, respectively). Fig. 4 shows reconstructing male and female model-Average errors in other anthropometric measurements are 1.5 to 2 times higher than input measurements. Due to a large number of results, the detail would be put in Appendix B.

Conclusion and future work
This research provides a summary of applying meta-heuristics optimization methods for 3D human reconstruction from anthropometric measurements. In the context of this study, Simulated Annealing and Particle Swarm Optimization are simple to implement and require fewer hyper-parameter adjustments. In contrast, changing hyperparameters could significantly affect the results of the Genetic Algorithm. This method, similar to Diversity Control Oriented Genetic Algorithm, could both quickly converge to the local optima. We also provided detailed instructions for deploying four methods under pseudo code to provide a guideline for future research.