Test case prioritization techniques in software regression testing: An overview

, execution time, cost, etc. After TCP the developer and tester have a minimum test case suite with better coverage in all respects. TCP definitely improves the quality of software and brings the best product within a limited time and cost. In this paper, we have provided a detailed survey of the TCP methodologies. This survey includes 2010 to the most recent studies. In this study, we have select carefully the most recent and relevant study of our topic.


Introduction
*Software Testing (ST) plays an important role in the SDLC as it guarantees defect-free software and customer satisfaction. Despite the complexity and importance of this phase when it comes to testing a small amount of time has been left behind which makes this phase costlier. ST techniques involve the execution of software to find if all the components are as per customer need. If any issue is found, it will be considered a defect or bug.
There are lots of stages of ST and lots of ST techniques are used. Among all the ST techniques the most important one is Regression Testing (RT). The importance of RT could never be ignored as this verifies that newly updated code impacting previously build functionality or not? It doesn't matter how many times we make changes in the code we have to run RT to make sure software stability against the code change.
Because of the limited time given for RT, it becomes the difficult phase in the sense of how and which test case to be selected for RT suits. Different methodologies have been used to make the best test suite for regression testing which is called Test Case Prioritization (TCP) or Test Case Selection (TCS). These methodologies include search, coverage, fault, requirement, history, risk and cost-effective based, etc. (Shah et al., 2019). Each of the methodologies has its own algorithms. Best TCP/TCS is dependent upon different criteria like the selection of dataset, incomplete data extraction (Shah et al., 2016). In this study, we have selected the most recent algorithms used for software TCP from the most popular repositories like IEEE Access, Elsevier, and Springer.
In this survey, we have studied 40 articles from 2010 to 2020 on TCP. This study organized as follows. In Section 2 which is a literature review, we will describe all the chosen algorithms in Table 1. In Section 3 we will compare the results generated by using each algorithm. In Section 4 we will describe research methodologies and try to answer the research questions. Section 5 contains state-of-theart challenges and Section 6 contains conclusions and future research topics.

Literature review
Test Case Prioritization (TCP) not only saves time and resources but also stakeholder confidence in us if we used it properly. Various methods and techniques have been used in this regard to obtain the best optimal test suite for Regression Testing. In this study, we will cover and summarize all those algorithms as shown in Fig. 1.

Firefly algorithm
Firefly Algorithm (FA) after getting inspired by the flashy behavior of the fireflies (Shah et al., 2019).
 Fireflies are attracted to each other because all of them are unisexual.  The attraction of fireflies is depending upon their brightness. Both are directly proportional to each other  Fireflies only move randomly if the same/no brightness is found nearby.
Quite a few algorithms were proposed using the Firefly technique.

Optimal firefly algorithm
This algorithm uses a metaheuristic firefly algorithm to generate an optimal path. In this modified firefly algorithm, he used a new objective function/brightness function and guidance matrix to traverse the graph (Shah et al., 2016).
The process starts with creating Control flow graphs (CFG) and Data flow diagrams (DFD) and generating XML files from the CFG diagram. This XML file goes as input into Optimal Firefly Test Sequence Generator (OFTSG). OFTSG perform the following steps (Shah et al., 2016):  Calculate nodes and edges  Calculate Cyclomatic Complexity  Create Adjacency matrix which is the edge between the nodes is considered as 1 everything else is 0.
 Create Guidance matrix using Eq. 1  Path traversal: Algorithm generates best path sequences.  Path Prioritization  Calculation of brightness value at every node using the Eq. 2  Calculation of mean value of brightness.
Higher the mean value of brightness will have higher priority.
The author tested this algorithm using 4 different programs as shown in Table 2.
This shows that as the number of states increases the cyclomatic complexity increases which causes an increase in the percentage of redundancy in path coverage. Results of this study were compared with the state of the art algorithm Ant Colony Optimization Algorithm (ACO). This shows FA generates better paths with reducing or no redundancy as compared to ACO as shown in Table  3.
Following limitations were found during this study, first, the proposed algorithm was not tested on any benchmarks program with a big test case pool. Secondly, it's not clear that which testing criteria are used for example statement coverage, fault coverage, etc. Panthi and Mohapatra (2015) used the same methodology to minimized test cases by exploring the state diagrams. Nor Hashim and Dawood (2018) proposed a similar approach to minimized test cases and the process has been analyzed using a UML statechart diagram. This study also shows prominent results in the form of minimizing the number of test cases.
Both used ATM case studies to test their approaches which provide satisfactory results. Khatibsyarbini et al. (2019) used Firefly Algorithm with string metrics as a fitness function to find the best TCP arrangement.

Firefly algorithm
Steps of Firefly Algorithm:  Extraction of test cases from dataset  Calculating distance and weight of test cases  Perform movement update based on highest values of weight/distance  Selection of test cases based on shortest distance as an optimal sequence.
Two variables Term Distance Inverse Document Frequency (TDIDF) and Term Frequency Inverse Document Frequency (TFIDF) were used to store distance among the test case and frequency (weight) of the test case respectively. The weight of test cases could be calculated as follows given in Eq. 3 (Khatibsyarbini et al., 2019).
The next move decided by the highest value of weight over distance and the shortest distance among test cases was selected as the optimum test case sequence. FA uses average percentage fault detection rate (APFD) to calculate the rate of fault detection which could be calculated using the formula given in Eq. 4 (Khatibsyarbini et al., 2019).
Six Software-artifact Infrastructure Repository (SIR) programs were used for testing. Three of them are UNIX-based programs (flex, grep, gzip) and three are SIEMENs programs (tcas, cs-tcas, j-cas) as shown in Table 2. Comparisons were made in this approach with state-of-the-art algorithms PSO, LBS, Greedy, and GA and among all of them, FA shows prominent results although LBS has quite better results at some places as shown in Table 3. Following limitations were observed during this study. The first one is a selection of datasets. The dataset used for testing this approach is very small SIR benchmark programs. This approach may give different results when applied to a bigger size of test case spools.
The second is less data utilization. For example, only distance and weight of test cases were used as input while we have a lot of other data available like system coding, etc. that could give totally different results. The third and final one is that the only TCP techniques were considered on the other hand we have a lot of other similar techniques available which was not included in validation. Su et al. (2020) proposed this hybrid model of the firefly algorithm, main purpose is to prioritize test cases and reduce the test cost. HFA adopted a hybrid model in which first it correlates all the available test data and test cases and secondly applies enhanced FA on this hybrid model. The new fitness function of this HFA is shown in Eq. 5.

Hybrid firefly algorithm (HFA)
where, Wi-1 shows the test cases weight i-1, a label "Order" is used for the test cases where the priority is known, "un-order" is for test cases where the priority is unknown, Random (·)={N-2<[[Ni-i]-0.1]<N}, where N is the total number of test cases. The steps of this algorithm are as follows (Su et al., 2020):  Define the new fitness function as shown in Eq. 5.  Calculate brightness of firefly using Eq. 6.  Calculate the degree of attraction by creating a distance matrix as shown in Eq. 7.  The movements of the fireflies are based on brightness only. Once all nodes have been visited, the firefly stop moving and their path is recorded.  In the end, the smallest path will be an optimal sequence.
where, β and α are constants, β is the light absorption rate, Є is a random factor of a uniform distribution, ɣ is attraction coefficient, HFA used three Software-artifact Infrastructure Repository (SIR) programs for testing. Which are flex, grep and gzip as shown in Table 2. Comparisons were made in this approach with state-of-the-art algorithms FA, PSO, and Greedy and among all of them, HFA gave the better execution time and best performance as shown in Table 3.
The following limitation was observed during this study is the selection of the dataset. The dataset used in this approach is three very small SIR benchmark programs. This approach may give different results when applied to a bigger size of test case spools.

Hypervolume genetic algorithm (HGA)
The method uses GA with Hypervolume Indicator to prioritized test cases with more than three testing criteria. He proposed an enhanced genetic algorithm which is called HGA (Hypervolume-based Genetic Algorithm). Earlier GAs was used to solve only single-objective optimization problems. But hypervolume allows us to combine multiple-objects to teat as a single object. HGA consist of two parts (Di Nucci et al., 2018;. Part-I 1. Initialization of cumulative coverage scores. 2. Computation of cumulative coverage scores (cost, branch, and statements) for each testing case. 3. Computation of actual IH using equation8. 4. Normalized IHP using Eq. 9.
HGA used five Software-artifact Infrastructure Repository (SIR) programs for testing. Which are bash, flex, grep, gzip and sed as shown in Table 2.
Comparisons were made in HGA with state-ofthe-art algorithms Additional Greedy Algorithm, GA (AUC metrics), NSGA-II, GDE3, MOEA/D-DE, and among all of them HGA shows the prominent results as shown in Table 3.
Following limitations were observed during the study of this algorithm. The first one is the type of construction used to validate the algorithm depends upon the results obtained. In order to validate the results APFDc matrix was used and only three testing criteria were used which include statement coverage, fault coverage, and execution cost. On the other hand, if we include the time parameter the results could be different. The second one is internal validity. a) Only different types of GA were used to compare the results of this algorithm. It would be highly appropriate to choose some other algorithm as well. b) Tuning of the parameter could really affect the validity of results Study shows that the value of parameters was kept the same as used in basic GA. Results should be tested by changing the values of parameters. The third which could affect the results is the dataset used in this approach which are six trademark programs from SIR only. The algorithm should be validated against real-time industry programs to test results.

Ant colony optimization (ACO)
ACO is another prominent approach used for test case prioritization. Following different types of the algorithm has been proposed based on ACO.
Ant colony optimization is a metaheuristic optimization methodology that was first introduced by Márquez et al. (2016). Zhang et al. (2019) used requirement-based criteria in his proposed ACO technique to solve the prioritization problem. He divided it into two ways TCP-ACA-1 in which he calculated the distance between two test cases and TCP-ACA-2 which is used to update the value of pheromone continuously (Zhang et al., 2019).
This method was successfully used in solving various optimization problems and the Test case prioritization problem is one of the combinatorial optimization problems where ACO showed impressive results. As per the technique used by Gao et al. (2015) during the test case prioritization problem, there are two rules that must be followed while using ACO methodology.
ACO algorithm works as follows as at the start ants are placed on the initial Test case list from where they randomly select the next node until all the faults are identified. It updates the pheromone value once each iteration is finished. The two most important steps in this algorithm are given below (Gao et al., 2015):  Pheromone updating rule: Pheromone is a kind of variable used in ACO which is representing the Ants pheromone. Which they deposit on their path during food search when the next ant comes in contact with pheromone they know that they are on right track and add more pheromone for the next group and so on (Márquez et al., 2016).
This Algorithm updates the pheromone values after each iteration by using this Eq. 10 (Gao et al., 2015).
where, is evaporation rate;  Test case selection rule: As per Ants' behaviors, the movement of ants is irrelevant to all other ants. Similarly, an ACO algorithm selecting the next node is the most important step which uses pheromone quantity and then decided which node to move. So, the next test case selection is calculated using Eq. 11 (Gao et al., 2015).
where, ( ) is visibility function; +1 is a list of alternative test cases. Where α and β are the parameters to regulates the influence of pheromone and visibility respectively. Both ACOs techniques used really small programs with a set of 8-10 test cases having 10 faults only as shown in Table 2.
In TCP-ACA comparisons were made with particle swarm optimization (PSO) algorithm, genetic algorithm (GA), and random test methods which showed better results. On the other hand, Gao-ACO was only compared with random test case selection none of the other comparison algorithms were mentioned as shown in Table 3.
Following limitations were observed during the study of this algorithm. The first one is the type of construction used to validate the algorithm depends upon the results obtained. In order to validate the results, the APFDc matrix was used. Secondly, a new technique was not compared with any of the stateof-the-art methodologies and the third but most important one is that the technique is tested with a very small program with eight test cases having 10 faults only. Solanki et al. (2015; proposed a modified ant colony optimization technique (m-ACO). In this algorithm when an ant finds food, it brings food back home and goes again to collect and this process repeats until all food sources are finished. Moreover, if the path is short all ants follow the same path until food finishes. This technique used the following test criteria code coverage, fault detected, and execution time. In order to achieve the maximum fault coverage, the APFD metric was used proposed by Elbaum et al. (2002). Algorithm proceeds as follows:

Modified ant colony optimization (m-ACO)
 Initialization of Ant variable  Randomly search food source  Calculate food fitness  Calculate the probability of selection  Calculate the reduction in pheromone In order to validate m-ACO author used 3 small case studies were used. 1) 10 test cases with 10 faults 2) 9 test cases with 5 faults 3) 5 test cases with 5 faults. All the test case was executed on a Perl program as shown in Table 2. In m-ACO none of the comparisons were made with any state-of-the-art methodologies to verify the correctness of this algorithm. Instead of that only APFD was calculated with prioritized and un-prioritized test cases with showed comparatively better results than unprioritized test cases as shown in Table 3.
This technique has the following limitations. First is only 3 small case studies with a limited number of test cases were used and it is highly recommended that this methodology should be validated with realtime and big data sets. By doing this we can get a real picture of results. The second is external validity. This approach is not compared with any of the state-of-the-art methodologies. It would be good to perform a comparison with some state-of-the-art prioritization methodologies. The third is the type of construction used to validate the algorithm depends upon the results obtained. In order to validate the results only the APFDc matrix was used, in which the APFD value of prioritized and un-prioritized results was compared.

Epistasis-based
Ant Colony Optimization technique is Multiobjective search-based regression Test Case Prioritization (MoTCP). This technique is a Metaheuristics that is inspired by nature (movement of ants to find food) (Gao et al., 2015). Study shows that epistasis theory also improved the effectiveness and efficiency in a single-objective GA (Deb et al., 2000). The use of epistasis theory in the multiobjective problem also showed promising results. ETS-Based ACO Algorithm for TCP is given below (Yuan et al., 2015).

Algorithm
Steps of ETS-Based ACO Algorithm for TCP.
 Setting of parameters of ACO  Initialization of all the new sequences.  Initialization of pheromone (weights of transitions).  New test case sequence creation based on ant method equation 12.  Test sequence evaluation using a fitness function.  Ranking of test sequences as per NSGA-II method (Deb et al., 2000).  Comparisons and Updation of the test sequence.  Updating pheromone.
where, =pheromone (weight of the transitions); =number of objectives; =heuristic information of an object; =heuristic factor; =relative weight of heuristic value; =is the set of candidate test cases; =ant.
In order to validate E-ACO author used the SIR program for unit test and the V8 program for the smoke test. The size of the V8 program is quite large which is good but it's still not that large enough for big experiments as shown in Table 2.
E-ACO showed very impressive results as compared with the above-explained ACO algorithms. This new methodology was also compared with the state-of-the-art algorithms NSGA-II for MoTCP and ACO which shows the prominent results as shown in Table 3. E-ACO showed very prominent results still it has some limitations that must need to address to make it a standard algorithm. The first one is instrumentation accuracy, which could yield different results. It would be better if we test this using different software instead of using only gcov (A GCC tool). Secondly, since the real-time program was not used in testing, in compensation to this test was run 10 times. It would be much better to use the real-time big program to get accurate analysis results. Third swarm sizes of different scales should be used. So we could easily compare the efficiency of ACO and E-ACO. The fourth and last one is the parameters configuration used in E-ACO which is the same used by NSGA-II. It would be rather good to change the parameters to verify the conclusion.

Hybrid ACO
This approach is proposed by Ahmad et al. (2018) which is the combination of two different approaches.
Step-I: In the first section by using test factors priorities are assigned to the test cases.
Step-II: Once we have priorities assigned to test cases we apply ACO to calculate the optimal sequence which has the lowest execution time but the highest fault rate.
The approach seems interesting but the study doesn't show anything about testing or if any comparisons were made with any state of art techniques. (2015) proposed a novel Local Beam Search Algorithm (LBS) which is an inputbased search technique. LBS technique uses lightweight input as compared to other techniques like code-coverage. In each iteration of this algorithm width of the beam is kept constant that's the reason this approach is much more efficient than the other. For example, the greedy algorithm doesn't select the new test cases from uncovered test cases only directly increases cost and time (Elbaum et al., 2002). This input-based LBS algorithm takes randomly selected test case set as input, and processes it, and generates the set prioritized set of test cases.

Jiang and Chan
This approach uses distance matric f to calculate the distance between prioritized and un-prioritized test cases (Gusfield, 1997). The steps of the algorithm are given below (Jiang and Chan, 2015).

Algorithm
 Selection of candidate set which is not yet selected  Calculation of distance matrix between the candidate and already prioritized ones  Test cases are sorted in descending order.  Appending the top test case in the successor pool.  Repeating the process until all the test case has been prioritized.
LBS used 4 Software-artifact Infrastructure Repository (SIR) programs for testing. Three of them are UNIX-based programs (flex, grep, gzip, and sed) as shown in Table 2. Comparisons were made in LBS with state-of-the-art algorithms ART, Greedy, and Genetic Algorithm (GA). LBS showed more impressive results than GA and Greedy but less than the Adaptive Random Test case (ART) as shown in Table 3.
Results shown by LBS are very impressive but this algorithm has some limitations. 1) Platform used, during the testing of this approach only a C program was used. In order to cross-validate the finding other programming languages should be considered. 2) Testing Suites, during the analysis of this approach only branch-adequate test suites were used. The use of different test case suites could improve the results. 3) Use of distance metric, there are a lot of other metrics available like Hamming distance matric could also be tried to analyze results. 4) Since the LBS follows a randomized procedure so, the author repeats the testing 50 to get better results. It would be much more appropriate to use a big population of the test case to avoid such issues. 50 study shows that search-based TCP techniques don't use fault rate detections. While this research used APFD which make it difficult to access that whether the results produced by this research are consistent with previous studies or not.

Particle swarm optimization (PSO)
Particle Swarm Optimization (PSO) is a multiobject TCP technique which prioritizes test case based on changes in the code. PSO is an optimization methodology that was first introduced by Eberhart et al. (2001). Souza used the Binary Constrained PSO approach to prioritize test cases using the following three steps (Tyagi and Malhotra 2014;de Souza et al., 2010):  Removal of extra test cases.  Selection of test cases with the highest coverage but lowest execution time.  Prioritizing test cases. Hla et al. (2008) proposed an algorithm that performed test case prioritization based on changes in software sections by the PSO algorithm. This algorithm has been used a lot in various functional optimization problems because of its fast convergence property. This algorithm uses multiobjects to calculate fitness values using statement, branch, function, and code coverage as shown in Eq. 13. Also, it uses the velocity and position of the test cases for the next fitness evaluation which can be calculated using Eqs. 14 and 15. PSO algorithm has two parts (Hla et al., 2008).

Part-I: Tokenizer
 Take code as input  Converts into token line by line  Save into database
where xi is the position vector and vi is rate of change of position vector. c0, c1, and c2 are weight factors. PSO used 20 test cases from JUnit as shown in Table 2. A JUnit test suite provides automatic test case creation and execution as shown in Table 2.
Comparisons were made in PSO with the state-ofthe-art Greedy algorithm. PSO showed less run time complexity as shown in Table 3. Following limitations were observed during this study. 1) In order to keep internal validity in mind. This approach was tested using 20 test cases from JUnit which is really less. 2) Second is external validity, this new approach is only compared with the Greedy algorithm. It is highly recommended to validate the procedure using some real-time application and make a comparison with some other state-of-the-art techniques to get a better picture of results. Khatibsyarbini et al. (2017) proposed this weight-hybrid string distance technique. The main purpose of this technique was to get a higher APFD rate and efficiency in execution. A hybrid string distance was created using weight and distance between test cases. String distance is of two types; character-based and term-based. Two characterbased string metrics used in this study are Manhattan Distance by Ledru et al. (2012) Eq. 16 and Levenshtein Distance by Jiang and Chan (2015) Eq. 17.
Hybrid string distance is calculated by multiplying the distance with the next test case weight.
"Term frequency-inverse document frequency" (TFIDF) shows the importance of a word in the whole document.
= log The whole process is divided into three phases: A. Information Extraction phase: In this phase test cases and their inputs are extracted. The extracted inputs are saved into a separate document which will be used so, that they could be utilized in the next phase. After that, the test cases from both programs (original and version) are executed and a fault matrix sheet is prepared to compare both results. B. String Distance Calculation: The extracted in the previous phase is used to calculate the string distances using Eqs. 16 and 17. The weight of test cases is also calculated using Eqs. 18 and 19 and the results of both calculations are populated into the test case distance matrix. Then hybrid string distance is calculated. After the completion of calculations, PSO is used to prioritize the distance matrix. C. Evaluation phase: In this phase, a matrix sheet is created using results obtained in an earlier phase. WH-PSO used 3 Software-artifact Infrastructure Repository (SIR) programs for testing. Three of them are SIEMENs benchmark programs (tcas, jtcas, and cs-tcas) as shown in Table 2. This new approach was not compared with any state-of-the-art algorithms only comparison was made among character-based, term-based, weight-hybrid character-based, and weight-hybrid term-based results as shown in Table  3. Results shown by weight hybrid PSO are very impressive but it would be really good to test this approach with other states of the art methodologies. Allawi et al. (2020) proposed this novel hybrid greedy and PSO algorithm (GPSO) which is the combination of both greedy and PSO algorithms.

Greedy particle swarm optimization (G-PSO)
In this approach, the author creates the fitness function using Particle Swarm Optimization (PSO) and then applies the Greedy approach to find the partial best and makes them as global best and select the optimal solution from there. After this step new algorithm moves back to the PSO approach where it finds the velocity of each particle is calculated using Eq. 22 and the location of the next partial using Eq. 23. This process repeats until the termination condition occurs.
Algorithm proceeds as follows (Allawi et al., 2020):  Random Initialization of particles.  Calculation of fitness function using PSO.  Greedy part: o Determines and selects the particle with the partial Best position value to be placed as the global Best solution. o Selects the optimal solution from the available candidate solutions at a particular point.
 Each particle velocity can be calculated using Eq. 22.  According to Eq. 23 move to the next position.  The algorithm will be stopped if the termination criteria are satisfied; otherwise, return to Step 2.
Six different Java card applications (JCS Applets) were used as data set to test the algorithm which includes a Network connection tracker, HelloWorld, PKI, RSACrypto, Calculator, OATH, Passport, CoolKey as shown in Table 2.
The comparison was done between the GPSO algorithm and standard genetic algorithm (GA), which showed that GPSO exceeds the GA in various ways. GPSO takes a fewer number of iterations and time but gives us high coverage percentage as shown in Table 3.
Following are the limitations found during this study. 1) Real-time application was used to test this approach but that includes small programs with few hundreds of branched was used. In a real-time program, there are thousands of branches of code that could give a different point of view about this approach. 2) Discussion about fault matrix is missing in the study, whether it's created or not. 3) Only one comparison was done with GA, it would be better to test this new approach with other state-of-the-art methodologies.

Greedy algorithm (GA)
3.13.1. Additional greedy algorithm (AGA) Yoo and Harman (2007) proposed the Additional Greedy Algorithm. This technique uses NSGA-II and its variant which is called vNSGA-II algorithms.
The vNSGA-II has two main modifications as compared to NSGA-II which are as follows:  vNSGA-II uses a separate group of sub-population which is good because different objectives can be assigned to each sub-group.  vNSGA-II keeps the record of the best subpopulation.
The additional greedy algorithm only supports up to two objects. In order to work with more than two objects, we have to combine the objects to make one by using the weight sum approach. That is the reason AGA doesn't show good results for multi-object TCP.
AGA used 4 Software-artifact Infrastructure Repository (SIR) programs SIEMENs based programs (printtokens, printtokens2, schedule, schedule2), and 1 European Space Agency (ESA) program (space) as shown in Table 2. Comparisons were made in this study among two genetic algorithms, NSGA-II, and its variation. It has been seen that the greedy algorithm doesn't work well for multi-objective TCP it is only good for single objects as shown in Table 3.
Following limitations were found during this study.1) Accuracy of tools used to find the coverage information. The author used professional software in order to minimize the risk but it would be highly recommended to try to use some other software to just see the variety of results. 2) Most of the testing data used in this study is from the SIR repository. Only one program was used from ESA. 3) Results of this approach also depend upon the type of algorithm used for multi-object prioritization. The author tried to use so far best algorithm NSGA-II but it is highly recommended to test the approach with some other multi-object test case prioritization techniques like firefly etc. to test the variation in results.

Enhanced additional greedy algorithm (EAGA)
This technique is proposed by Hsu et al. (2014), EAGA is a modified form of AGA. In this technique control flow is used over the data flow criteria. Experimental analysis shows that the EAGA technique performed well in "fault severity/unit cost". EGA used 8 Software-artifact Infrastructure Repository (SIR) programs SIEMENs based programs (Tcas, Totinfo, Replace, Schedule, Schedule2, Printtokens, Printtokens2, and Space) as shown in Table 2. Comparisons were made in this study with AGA which shows that EAGA with GA is far much better than AGA for large programs only but when we use small programs there is not a difference between AGA and EAGA as shown in Table  3.
Following limitations were found during this study: 1) Same as AGA, EAGA is not fit for multi-object TCP. 2) Results are not compared and checked with any other state-of-the-art techniques to get clear pictures of results.
Graphite TCP: One of the most popular prioritization techniques is greedy. The greedy technique used code coverage criteria. In this technique test cases with higher code coverage are considered better than the lower coverage (Do et al., 2008;. Azizi and Do (2018) proposed a novel graph-based greedy technique that utilizes a graph traversal algorithm to prioritized test cases. This algorithm consists of two parts, graph generator, and graph traversal.
Part-II: Traversal Procedure: 1) Select node with maximum value as a start point.
2) Calculate the highest gain using the traversal algorithm.
3) Repeat this process until all connection edges will eliminate.
In order to test this approach, the author used 4 open-source programs, among them 2 are obtained from SIR (jmeter, jtopas) and two of them are none SIR (nopCommerce, Umbraco-CMS) as shown in Table 2. Experiment results showed indicated approach is effective and efficient as compared with the unprioritized approach only as shown in Table 3.
There are few findings of this new methodology. 1) This new technique is not compared with any state-of-the-art TCP methodology to get a clear picture.
2) The choice of the distance function. The most popular method was used for it but it could give different results with a different function. It is highly recommended to test the new approach using other distance functions as well so, we could get a clear picture of this algorithm.

Results
In this section each research questions were answered: Q1: How TCP is important in Software Testing? In Regression Testing (RT), the TCP is the most difficult phase of the RT that allows testers to risk management, test planning, cost-value analysis of the testing phase. Different approaches are used to prioritized test cases for example time, cost, effort, code coverage, and fault coverage which increase RT performance. By using these techniques test cases with the highest priority are selected for RT suites that give us the highest rate of fault detection. Q2. What are different classifications of test case prioritization techniques? Fig. 2, shows the detailed classification of the test case prioritization techniques.

Q3.
What are single object and multi-object TCP techniques? Table 4 shows which techniques are multi-object TCP and single-object TCP. This table also shows which metrics were used to validate the algorithm. Q4. What are the latest studies in TCP? Table 5 shows the latest studies in Test Case Prioritization techniques. Q5. Which metrics are commonly used for TCP? About 70% of the multi-object test case prioritization techniques used the Average Percentage Fault Detection (APFD) metric as shown in Table 4. Q6. Which prioritization methods are used often and what are their proportions? Table 6, shows the percentage of each algorithm from different repositories.    Table 7 shows the inclusions and exclusions criteria of this study.  (Wang et al., 2011). It is highly recommended to keep these things in mind before start prioritization so we could get accurate cost and computation time.

Data utilization
Study shows that effectiveness and efficiency of a logarithm depend upon various factors for example programming language and its framework, coverage, types of faults, etc. . Other than these factors the most important thing that could affect the effectiveness and efficiency of an algorithm is as follows:  Size of data set: Most of the above describe studies are only using a small set of SIR programs like bash, flex, grep, gzip, sed, etc. and some of them even used very small programs having 8-10 test cases (Khatibsyarbini et al., 2019;Su et al., 2020;Di Nucci et al., 2018;Jiang and Chan, 2015;Yoo and Harman, 2007).  Incomplete data: Second most important thing is input data used in the algorithm. For example, the results of the APFD matrix depend upon the distance and weight of test cases which depends upon input given to the system (Khatibsyarbini et al., 2019). Test cases have a lot of other data available in them. But it is very difficult, timeconsuming, and expensive to utilize all the available data. The best example is system coding.
The extraction and utilization of complete data could be a big challenge in the test case prioritization techniques which could give us different results.  Parameter Tuning: The tuning of parameters is could also affect the effectiveness and efficiency of an approach. For example, in few studies, parameter values were not changed and kept as it is as in the previous work (Li et al., 2007). Tuning to that parameter could influence the overall results. Lu studies the influence of change in code or its parameter. Study shows that change in code and parameter could influence the effectiveness an algorithm (Lu et al., 2016).

Execution time
The study shows that most of the techniques are tested using the SIR program which is a small set of programs. These programs have test cases whose execution time is very low. It would be worth testing the above-given prioritization techniques on a test case where execution time is high. But testing of high execution time test cases could increase the cost. It would be better to consider the individual cost of test cases instead of the total cost.

Conclusion and future work
In this paper, an empirical survey has been conducted on Test case prioritization techniques. In this study, we review 40 TCP papers published in journals and conferences with a systematic process. The main objective of this study is to explore and determine which approaches have been studied in detail and which of them are left behind and where is more chance of improvement. We divided this study into the following stages. In Stage-I we created six research questions. Stage-II we selected repositories from where we are going to get our literature. Stage-III we created a search methodology in which we search the most recent papers on state-of-the-art techniques. Stage-IV we setup inclusion and exclusion criteria which we utilized to search our data as shown above. The following suggestion was provided in this study:  Latest public data set utilization: Almost 60% of the papers selected in this study are using a public data set which is good but data is being updated regularly which is not satisfactory. By the development of different programming languages and programming techniques updated code must be utilized for testing. It looks like becoming a trend of using SIR data instead of actual data or updated public datasets. Although there are challenges of getting actual data set and re-testing by other researchers.  Consideration of multi-object TCP approaches: Study shows that 35% of the approaches are welldefined multi-object and 20% of them are purely single object TCP techniques and the rest of them are not properly stated as single or multi objects. It would be highly recommended to create a multiobject model-based approach that could set standards.  Consideration of cost criteria: Survey showed that that most of the studies focus on code base techniques and less focus is given to cost criteria. Some studies used cost criteria as a total cost of execution of test cases. It would be better to consider the individual cost of test cases instead of the total cost.  State of the Art Techniques Comparison: Study showed that 46% of the techniques have only performed a comparison with the well-defined state-of-the-art methodologies. 26% of the approaches have not compared any state-of-the-art technique. It would be highly recommended to set up such models which accept the studies having compared with the state-of-the-art techniques. Also, it is good to conduct more studies for comparison of TCP techniques.  Survey study for Multi-Object TCP: Study showed that none of the review/survey studies were conducted on multi-object TCP techniques. It would be highly recommended to separate publishing the review study of multi-object TCP techniques.  Utilization of real-time dataset: Study showed that none of the approaches has been tested using any industrial or real-time dataset. It would be highly recommended to validate the test case prioritization techniques using some big industrial or real-time data set to get a clear picture of the new approach. However, only 10% of the studies used non-SIR data set but not industrial data set.

Conflict of interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.