Development and evaluation of a mobile game as an English learning tool for ESL learners

Learning and memorizing irregular verbs are some of the most difficult aspects of the language for non-native English learners. Research concerning games for education reveals that educational games have a large positive impact on learners’ skills and abilities. This research was divided into two phases including 1) the design, development and deployment of a mobile game used as an English learning tool for ESL learners and 2) an evaluation of the outcome. An Android mobile game was developed and then deployed on Google Play. In the evaluation phase, a quasi-experimental set-up with a non-equivalent pre-test and post-test control group design, was used to evaluate the impact of the use of the mobile game. The sample consisted of sixty-four tenth-grade students who studied English in a Thai school. There were two groups; the treatment group (N=32) and the control group (N=32) selected by cluster sampling. Both groups were given a pre-test to measure their prior knowledge. The sample then learned English in the class using conventional methods over a six-week period. During that period, the treatment group used the mobile game as a supplementary tool. Finally, both groups were given a post-test to measure the learning outcome. In this study, an independent t -test at the significance level of 0.01 was conducted. The results revealed that students who used the mobile game as a supplementary tool had statistically significantly higher learning outcomes.


*The
Partnership for 21 st Century Skills has presented a framework entitled '21 st Century Learning' that focuses on eighteen different skills (P21 Network, 2020). They are necessary but nontechnical and non-specific with regard to jobs available in the workplace. According to reports of Zainuddin et al. (2019); employers are increasingly recruiting people who have 21 st Century Skills specifically the 'four Cs' which relate to critical thinking, communication, collaboration, and creativity, all of which are considered to be the most important skills in the modern age (Rao, 2019). The ability to communicate information effectively, clearly and as intended is one of the most essential skills. Both good verbal and written communication skills are required for the workforce, whether they are sending correspondence, emails, communicating with clients, offering business opportunities, or others. A survey of hiring managers revealed that communication skills are the most in-demand skills (Meganck et al., 2020) on the part of employers (Rios et al., 2020).
English is not the most commonly spoken language in the world, but it is used as the language of international communication on the part of fiftythree countries and spoken by more than 400 million people across the globe (Rao, 2019). Moreover, the Exit Exam has been implemented in many institutions. This exam is a specific form of assessment aimed at examining the English level obtained by students before leaving for the workplace. In Thailand, many universities have announced that all students who plan to graduate are required to take the Exit Exam, and they must achieve the standard needed to ensure that they have adequate English communication skills. The exam is typically given to students towards the end of their higher education tenure, and is used to measure the general level of education gained at an institution, rather than assessing the skills obtained from specific courses. Verbs, including irregular verbs, tend to be the main body of a sentence. Without them, sentences become mere fragments and devoid of meaning. Irregular verbs are an important feature of English; people often use irregular verbs when speaking and writing (Salawazo et al., 2020). Learning and memorizing irregular verbs are some of the most difficult aspects of the language for English learners. Irregular verbs are normally presented in the form of alphabetical lists which are difficult to remember. However, ESL (English as a Second Language) learners need to study them since they are used to form passive and perfect sentences. The best way to memorize vocabulary is to review and repeat what is presented. However, ESL learners tend to become discouraged, and fail to learn when they try to memorize and review irregular verbs. To overcome this, games can be used as a supplementary tool to engage and improve the learning experience of the learner (Yunus and Saifudin, 2019).
In this research, we developed a mobile game to be used as an English learning tool to help ESL learners memorize irregular verbs. We then evaluated the outcome after learners had used the mobile game as a supplementary tool for six weeks.

Multi-memory model
The multi-memory model or multi-store model is a psychological model proposed by Atkinson and Shiffrin (1968). The model suggests that human memory consists of three stores: A sensory register (SM), short-term memory (STM), and long-term memory (LTM). When people listen, see, sense, or learn, the incoming sensory information first enters the sensory register where it is stored for a very short period before it disappears. The STM then receives selected information from the sensory register. It is stored for fifteen to thirty seconds before it begins to decay. If people rehearse the information they have received, it will be encoded and transferred into their LTM which is able to store for a long time. As a result, this information can be recalled at a later date.

Educational games and framework
Educational games and gamification are the application of games designed to assist students and learners in learning certain topics or developing their skills. Over the years, educational games and gamification have been widely used to facilitate learning in almost all education levels. The benefits of educational games have drawn the attention of educators around the world. Research into games for educational purposes shows that an educational game as a learning tool is able to affect many aspects of education, and also has a large positive impact on learners' skills and abilities (Herwinarso et al., 2020). Games and related applications can be used as a supplementary tool for a classroom course, or used independently on an e-learning system (Alshammari, 2020). A game consists of content, user interface, control, story, gameplay (feedback and experience), objective, and game mechanics (Elaish et al., 2018). A game mechanic is a rule or method designed for interaction with the game state, providing the gameplay as well as the game's response. It effectively specifies how the game will work for the people who play it (Roy, 2016). Many game mechanics are used by game designers and developers for both non-educational and educational games. Elaish et al. (2018) defined some of the cores and very popular game mechanics that are used when designing and developing an educational game. These include objectives and goals, feedback, achievements and rewards, countdown timers, competitions, and progress indicators. The objectives and goals should be challenging but achievable.
When designing a long-term objective/goal, it is better to include a short objective/goal to ensure that the learners/players do not leave the game prematurely because they feel the objectives/goals are not achievable (Portales, 2015).
Smartphones and mobile devices are now an integral part of daily life. They have made learning more accessible, convenient, and portable than ever. Studies have revealed that educational games on a mobile device (mobile applications or mobile games) have a positive effect on the learning process, as they tend to increase motivation and improve students' learning outcomes and skills (Herwinarso et al., 2020).

Mobile game engines
A game engine is a software development tool designed to reduce the cost, complexity, and time needed for the development process. It provides several components and tools, as well as the complex math and physics functions used in games. This allows developers to create games faster and with less effort. Since the mobile game industry is constantly developing, there now exist mobile game engines, game development platforms, and tools for developers. Table 1 shows popular game engines that support mobile platforms (Rawendy et al., 2017).

Mobile game development and deployment
The first phase of this study was to develop and deploy a mobile game application to assist students to memorize irregular verbs. The development was based on a Software Development Life Cycle (SDLC) (Pressman and Maxim, 2014) consisting of five stages in the form of requirement gathering, analysis, and design, development, testing, and deployment. The requirements were analyzed and determined from the information gathered from two groups of stakeholders with the use of in-depth interviews as part of the requirement engineering process. The first group consisted of thirty-two students. The second group consisted of a panel of seven experts who had experience in teaching English for ESL purposes. They were non-native and native English speakers. The goals of the game were divided into levels, thus the contents of the game; irregular verbs (Murphy, 2019) were grouped into fifteen levels to ensure that the players do not leave the game because they feel that the goals are unachievable. The content levels then were approved by the panel of seven experts who had experience in teaching ESL.

Experiment and assessment
In this study, a quasi-experimental set-up with a non-equivalent pre-test and post-test control group design (Creswell, 2014) was used to investigate the impact of the use of the application. The sample consisted of sixty-four tenth-grade students who studied English in a Thai school. There were two groups in the form of a treatment group (N=32) and a control group (N=32) selected by cluster sampling.
Initially, both groups were given a pre-test to measure their prior knowledge. Then, the sample groups learned using conventional methods in a class setting for six weeks. During that period, the treatment group used the mobile game as a supplementary tool. Finally, both groups were given a post-test to measure the learning outcomes (Table  2).

System design and architecture
This section describes the system architecture and the components that employed game programming patterns and android native applications (Portales, 2015). It is comprised of a game loop and system architecture as shown in Fig.  1 and Fig. 2.

Game loop
A game loop is the core procedure that controls the overall gameplay. The game runs repeatedly on a series of tasks in this loop, known as a frame, until the user quits the game. Generally, a game loop runs at 30 to 60 frames per second (FPS). The game loop allows the player to control the game objects by reading input via the I/O unit. In this study, it reads the coordinate when the player touches the screen to select an answer.

Start Initialize
Destroy End Input Process Update Render Game Loop As illustrated in Fig. 1, the game starts from the initialization state where all the required resources (assets) such as images to be used in the form of sprites, sounds, and setting data, are loaded into the system. It then starts a game loop. The game loop performs four main tasks. Firstly, it reads player inputs from SurfaceView (a user interface in the application's view hierarchy) on the touch screen. Then, the game processes the inputs, and updates the properties of all the active sprites and objects, including life status, scores, the position of sprites, and plays sound according to the game status and time. Finally, the updated objects are rendered and displayed on the screen. The presentation of the series of frames in the game loop creates the illusion of animation. The game will be terminated when the time is up, or the player sends an input command to exit the game. When the game loop terminates, all objects are destroyed, and the program unloads all the resources then exits the game. GameView. The Main Activity View employs a main thread to handle the Settings, Controller, Content Manager, Audio, and Database, while the GameView handles sprites.

System architecture and components
A sprite is a two-dimensional bitmap graphic object that can be a static image or an animation that is integrated into a game scene (Roy, 2016). Each sprite is used to represent different objects such as choices, targets, effects, characters, enemies, and background objects. There are many methods available to draw and animate sprites. In this study, we used bitmap images and threads to render and animate sprites on the game stage. A thread is a light-weight process consisting of instructions running independently of one another. Each sprite is individually controlled by a thread. The controlled functions include sprite animating, collision detecting, and scene rendering. Sprites are implemented under Android SurfaceView, where the game controller is able to process collision checking, touch events, and can respond to the sprites' status. To play a game, the player needs to start with an unlocked level. During the game, questions are displayed in the baskets located at the bottom of the screen. To be successful, the player needs to choose the correct answer by tapping the choices falling from the top of the screen. When the player obtains 80% or higher at the current level, the mission/objective of the level is achieved. The next level will be unlocked to allow the player to play the next level.

Application implementation and deployment
We developed an Android native application (mobile game) using Android Studio SDK. A native application is an application written in a language specific to the operating system. As a result, it benefits from a seamless and high-quality performance on the device and is optimized for a specific platform as it is able to leverage the full capabilities of the operating system. The applications are compiled using a platform's core programming language and the APIs cause them to be both fast and responsive. Fig. 3 illustrates the UI of the game levels. Fig. 4 shows mobile game screens.
The testing stage was performed by the developer, seven teachers, and thirty-two students. The application was then distributed to the sample users. Distributing an application is available through several channels. It is possible to send the application (Android Package Kit: APK) to the users by email, share it on a website or a cloud, or publish it on an official marketplace called Google Play. This latter is the most famous channel for distributing applications.
In this study, we distributed the application via Google Play. The following are steps needed to publish (upload) the application to Google Play.
Register for a developer account: the registration process needs a one-time registration fee of $25.
Configure the application for release: Provide the necessary attributes; the android: Version code, android: Version Name and remove the android: Debuggable located in the manifest element.
Generate the APK file: Perform a digital signature signed with two keys, the app signing key, and the upload key then build the APK.
Prepare store listing: Prepare application details, graphic assets including screenshots, images, videos, promotional graphics that show application's features and functionalities.  Upload the APK: Upload the APK file and add application details on Google Play. After that, the Trust and Safety team will verify the application. This may take up to three days. If there is no problem, the application will be available on Google Play.
In terms of the usage statistics, the Google Play Console reports that the application has been downloaded and installed from the store more than 16,662 times from users in fifty-five countries around the world, as shown in Fig. 5.   Fig. 6 shows the top fifteen countries' usage statistics. The impressions in the report indicate the number of times (counter) that users have used the application. The top country is Thailand (73,342 times, 70.24%) followed by Laos (11,012 times, 10.55%) and Australia (3,584 times, 2.85%). Thailand is top in the terms of usage statistics since we set the meta-data to focus on Thai students when we published the application on Google Play.

Results
In this study, an independent t-test at the significance level of 0.01 was conducted. Before the t-test was carried out, the pre-test scores were tested for normality and homogeneity using the Shapiro-Wilks test (Statistic=0.95, p=0.21) and the Levene's test (F=0.11, p=0.741), respectively. These results indicated that the scores were normal and homogeneous.
The results in Table 3 show that there was no statistically significant difference in terms of the pretest scores between the treatment group (M=3.12, SD=1.26) and the control group (M=2.97, SD=1.18); t(62)=0.512, p>0.01. This finding indicated that both groups had the same prior knowledge level or skill, before engaging in the learning experiment. To evaluate the impact of the mobile game, an independent samples t-test was conducted. The post-test scores were tested for normality and homogeneity using the Shapiro-Wilks test (Statistic=0.96, p=0.33) and the Levene's test (F=0.02, p=0.894) respectively before the t-test was carried out. These results indicated that the scores were normal and homogeneous.
The results in Table 4 show that there was a statistically significant difference in the post-test scores between the treatment group (M=23.43, SD=2.90) and the control group (M=19.91, SD=2.30); t(62)=0.512, p<0.01. Based on these results, the alternative hypothesis was accepted. It can be concluded that using the mobile game as a supplementary tool yields significantly better learning outcomes than occur through conventional learning methods as applied to the control group.

Conclusion
In this study, a mobile game was developed to act as an English learning supplement for ESL learners. The development is based on an SDLC and game framework. The goals of the game were divided into fifteen levels to ensure that the players did not leave the game because they felt that the goals were unachievable. The content levels were approved by a panel of seven experts who had experience in teaching English for ESL purposes.
A quasi-experiment with a non-equivalent pretest-posttest control group design was used to investigate the impact of the use of the mobile game. The sample used in this study consisted of sixty-four tenth-grade students in a Thai school who were studying English. There were two groups; a treatment group (N=32 students) and a control group (N=32 students) selected by cluster sampling. Initially, both groups were given a pre-test to measure their prior knowledge. The results showed that both groups had the same knowledge level prior to the experiment. The sample groups then learned English using conventional methods in a class setting for six weeks. During that period, the treatment group used the mobile game as a supplementary tool. Finally, both groups were given a post-test to measure the learning outcomes. Based on the results, it can be concluded that using the mobile game as a supplementary tool yields significantly better learning outcomes than does the sole use of conventional learning methods.

Conflict of interest
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.