Design and evaluation of an adaptive framework for virtual learning environments

Traditional virtual learning environments may not always be suitable as they overlook the diverse requirements of students and are designed generally to support certain learning activities. Adaptivity is often proposed as a promising solution to overcome that limitation. However, it is still challenging to find the proper way to design such systems in order to adapt learning material in accordance with the students’ characteristics. This paper, therefore, provides an adaptive framework to design different instances of adaptive virtual learning environments. An implementation based on the proposed framework resulting in an adaptive virtual learning environment is also presented. The adaptive environment incorporates learning style and student performance. These two student characteristics are used to produce personalized learning paths as the main adaptive feature. An illustrative example is also offered to highlight how the framework can be used and implemented. The paper also presents an evaluation of the developed adaptive virtual learning environment in terms of perceived usefulness and learning engagement. A controlled experiment was managed with seventy-five participants in a learning environment. The results indicate that the adaptive virtual learning environment can be better to support students in terms of their perception and better in engaging them in the learning process than when they interact with a non-adaptive version. The framework can be valuable as a foundation in designing such systems and in enhancing future adaptive online-learning research. Future directions of research are also highlighted.


Introduction
*Effective Virtual Learning Environments (VLEs) are designed to offer suitable and relevant learning resources to support the learning process (Gunathilaka et al., 2018). Traditional VLEs may not always be appropriate as they overlook the diverse requirements of students and are designed generally to support pre-defined learning activities (Rodrigues et al., 2019). The main challenge in designing VLEs is to find out the appropriate approach needed to meet the needs, motivation, and preferences of students, and to deliver a more personalized learning experience (Truong, 2016). Adaptivity is often proposed as a promising solution to enhance traditional VLEs so that they can adjust their behavior and output according to the user's requirements (Brusilovsky, 2001).
Adaptive Virtual Learning Environments (AVLEs) can provide personalized learning resources and their order according to a student model that can represent and maintain different students' features such as motivation, knowledge, performance, and learning style (Normadhi et al., 2019). An AVLE may offer adaptive learning paths, underline particular learning content fragments or lessons, and may adjust a graphical interface according to specific students' features. However, the current AVLEs are usually designed based upon pre-defined adaptation rules linked to the learning material of a specific course (Normadhi et al., 2019). The main issue is that the generalization and usage of such systems can be limited to specific learning purposes. In addition, when such adaptive frameworks proposed, they are rarely implemented and appropriately evaluated (Xie et al., 2019). Therefore, this paper aims to address this issue in providing an adaptive framework for designing and implementing AVLEs, followed by an evaluation. This paper contributes to the current literature by offering a generic adaptive framework for designing AVLEs taking into account related work in building such frameworks (Bremgartner et al., 2017;De Bra et al., 2013;Feigh et al., 2012). Furthermore, the paper contributes by offering validation of the framework through a specific implementation that results in an adaptive virtual learning enlivenment. Also, the paper offers an experimental evaluation in a learning environment exploring the effect of adaptivity on students. The adaptive framework has three fundamental modules, including the content domain model, the student model, and the adaptation model. The content domain model facilitates the process of representing, storing, and maintaining learning resources where different application domains can be considered. The student model also represents stores and maintains student features.
The adaptation model considers information stored in both the content domain model and the student model to deliver personalized learning paths in order to enhance learning. The framework was validated through an implementation that results in an AVLE according to the learning style and learning performance. These two student features are uniquely related to this particular implementation. The AVLE was also evaluated through a carefully controlled and thorough experiment in a learning context. The main variables of the experiment were the perceived usefulness and learning engagement when using the adaptive version of the AVLE. Subjective feedback was also collected from students who used the developed AVLE.
In this paper, three main research questions are addressed as follows: RQ 1. How can we design an adaptive framework for developing different instances of virtual learning environments? RQ 2. How can the proposed adaptive framework be used to implement a virtual learning environment? RQ 3. What is the effect of adaptivity when using the developed virtual learning environment by students in terms of perceived usefulness and learning engagement?
The paper is structured as follows. Section 2 offers theoretical foundations related to adaptive virtual learning environments. Section 3 conceptualizes the concept of learning style. Section 4 details the proposed framework for the AVLEs. Section 5 details the implementation according to the proposed framework. Section 6 provides an illustrative example. Section 7 details the evaluation method. Section 8 presents the results and discusses them. Section 9 concludes the paper with future directions of research.

Adaptive virtual learning environments
Adaptivity can be described as a process or an approach of adjusting a system, a graphical user interface or content in accordance with the user's requirements (Brusilovsky, 2001;Klašnja-Milićević et al., 2015). For instance, learning strategies can match and be modified to the students' learning styles, performance, and cognitive ability. The personalization concept can also be connected to adaptation; personalization means that designing an object or something according to the needs of a specific human/user. Adaptive systems can be described as systems that modify their behavior and output based on different useful features such as preferences, motivation, behavior, emotion, knowledge, and skills (Feigh et al., 2012). Adaptive systems construct user models, process their data, make decisions, or inferences generated as output to offer the adaptive and personalized experience to the users (Jameson, 2009).
Adaptive technologies can be applied to different domains, such as e-commerce, e-health, and elearning. For example, the AEADS system was implemented, as an application related to the ecommerce domain, to offer relevant advertisements based upon a user's preferences and behavior (Qaffas et al., 2018). Adaptivity is also applied to graphical user interfaces as an important application domain. For example, an approach was developed called CHAIN to provide adaptive help and support to assist users in completing their tasks when interacting with the graphical user interface (Akiki, 2018).
Regarding the domain of e-learning, adaptivity is fundamental to meet the student's needs and requirements; so, an AVLE can recommend suitable learning strategies, offer relevant learning content and support the navigation through learning resources (Brusilovsky, 1996). Moreover, a more recent definition of effective VLEs highlights the significance of adaptivity as a vital concept that needs to be incorporated into modern learning environments (Rodrigues et al., 2019). Based on a recent systematic review of the field, it has been argued that adaptive learning based on knowledge level, preferences, and learning style has always been an attractive topic in education (Xie et al., 2019). Adaptivity in VLEs is still a challenging research area because of the different factors involved, including the diversity of student features and characteristics, the complexity of matching learning material and their sequences to specific features, and the need for such systems to follow sound instructional models to support learning.
AVLEs are an improvement to the traditional approach, which has the assumption that 'one size fits all' in the design and development of e-learning systems. The areas that inspired the evolution of AVLEs include Intelligent Tutoring Systems (ITSs), adaptive hypermedia, and Web-based educational systems (Park and Lee, 2003). ITSs utilize artificial intelligence techniques in order to simulate the teacher's role in offering personalized instruction (Self, 1998). In the middle of the 1990s, a large number of students were able to acquire personal computers and could access learning resources available on the Web. Accordingly, adaptive Webbased learning started to meet the different requirements of those students.
The main aim of AVLEs is to adapt instructional material and their arrangements to meet the requirements of a student closest possible in order to enrich the learning experience. AVLEs can take into account student features such as performance, emotion, skills and learning style in order to offer more personalized features and to deliver relevant learning material (Brusilovsky, 2001;Normadhi et al., 2019). An AVLE may emphasize important learning content fragments, offer feedback on what should be studied, or generate personalized sequences of learning resources.
There are different research directions related to AVLEs (Brusilovsky, 2012). Adaptive models and frameworks are seen as a key research theme (Feigh et al., 2012). Student modeling also represents another research direction that follows the steps of representing, storing, and maintaining student features such as learning style, performance, knowledge level, and motivation (Normadhi et al., 2019). Another research direction is related to content domain modeling and developing authoring tools for AVLEs (Hsu, 2012). Developing and utilizing adaptive methods and techniques can also be considered a vital research direction in AVLEs (Brusilovsky, 1996;Klašnja-Milićević et al., 2015).
Many AVLEs have been designed and deployed, concentrating on the different research directions (Akbulut and Cardak, 2012;Brusilovsky and Millán, 2007). An adaptive framework was proposed by De Bra et al. (1999) to support the design of adaptive systems; it has three main modules, including the user model, the domain model, and the adaptation model. In a relevant project called GRAPPLE, these modules were also incorporated into learning management systems so that instructors can create adaptive courses (De Bra et al., 2013). These three modules are essential, even to modern AVLEs, as evidenced by a recent work offered by Bremgartner et al. (2017). Regarding student modeling, Normadhi et al. (2019) recently conducted a systematic review focusing on how student features are identified, used, and evaluated in AVLEs. An approach for automatic domain modeling in AVLEs was proposed so that different forms of adaptation can be generated in order to recommend personalized and adaptive instructional resources (Simko and Bielikova, 2019).
About adaptive methods and techniques, a pioneering example is the adaptive link annotation. It shows a specific metaphor behind links such as changing the font color, size, or icons of a specific link to familiarize the student with the lesson behinds that link . The LS-Plan system, another example, changes the sequence and arrangements of learning material based on the knowledge level and learning style of students (Limongelli et al., 2009). The eTeacher learning system also groups students into different clusters based on their learning styles so as to offer suitable learning strategies to each group of students (Schiaffino et al., 2008). Another example is the Protus system that teaches computer programming; it is based on a student model that represents both knowledge level and learning style (Klašnja-Milićević et al., 2011). An adaptive approach that matches learning resources to the learning style of students is proposed by Dorça et al. (2016), yielding positive findings. The APELS system was also developed recently according to prior knowledge and learning style, adapting freely available learning material on the Web (Aeiad and Meziane, 2019). The work presented by Gunathilaka et al. (2018) confirmed the positive learning effect by the generation of personalized learning paths that are aligned with the knowledge level, performance, and learning style of students.
The work presented in this paper differs from previous attempts by building an adaptive framework, by a validation of the framework through an implementation that resulted in an AVLE, and also by a carefully conducted experimental evaluation of the effect of adaptivity on learning. Keefe (1979) described learning style as a student characteristic influenced by two main aspects including affection and cognition which determine the way that an individual recognizes, understands and interacts with factors involved in a learning context. Learning style can also be defined as the favored method to approach learning and gain some knowledge (Honey and Mumford, 1989). Cognitive style is a concept related to learning style; however, cognitive style can be considered as a subset or a certain aspect of learning style. Also, many learning style models exist; some aspects of such models overlap with and similar to those existing in other models; other dimensions can be unique to specific models. Popular learning style models are the Felder-Silverman learning style model (Felder and Silverman, 1988), the Honey and Mumford model (Honey and Mumford, 1989), and the Kolb model (Kolb, 1984).

Learning style
Learning style is an essential subject in education (Honey and Mumford, 1989;Keefe, 1979;Klašnja-Milićević et al., 2011). Despite some disputes questioning the added value of the concept of learning style in enhancing learning (Curry, 2000;Pashler et al., 2008), many researchers claim that learning resources should be tailored in accordance with the learning style of students, and encouraging findings were also obtained (Akbulut and Cardak, 2012;Felder and Silverman, 1988;Klašnja-Milićević et al., 2011;Labib et al., 2017). Coffield et al. (2004) pointed out that "there is a strong intuitive appeal in the idea that teachers and course designers should pay closer attention to students' learning styles." A comprehensive model of learning style still needs to be established even though various models exist (Coffield et al., 2004). However, the Felder and Silverman (1988) Learning Style Model (FSLSM) is used frequently as the preferred model, particularly in AVLEs (Akbulut and Cardak, 2012;Alshammari et al., 2014). The dimensions of FSLSM are comprehensively detailed, and each learning style dimension has a number of teaching strategies (Felder and Silverman, 1988). FSLSM also comes with a reliable and validated tool called the Index of Learning Style (ILS) to identify the students' learning styles (Felder and Spurlin, 2005). The model is comprised of four learning style dimensions (information processing, input modality, information understanding, and information perception).
According to FSLSM, the dimension of information processing (active/reflective) details the technique that students use in order to process information. Active students may interact with and manipulate something and may collaborate with their peers for effective learning. Reflective students can think deeply about something before actively involved in order to gain knowledge. AVLEs can indirectly support both active and reflective students by integrating collaborative and interactive features in addition to problem-solving features (Jeong and Lee, 2008).
The dimension of input modality (visual/verbal) focuses on the presentation of information and the information multimedia types. For instance, pictures, videos, graphs, and diagrams can be used to support visual students. Verbal students might be supported by offering them spoken information and written details. A large body of research examined the learning effect when taking this particular dimension into account mostly without any positive learning enhancements (Kollöffel, 2012).
The dimension of information understanding (sequential/global) concerns the preferred structure and arrangement of information. Sequential students can understand learning content if they are delivered in a linear and more logical manner where each learning step is carefully explained. Global students can be supported by offering them the big picture and overview of information before going into their details. Global students typically prefer to study in a random learning approach. This dimension has been incorporated in an AVLE without obtaining positive results questioning its feasibility in learning provision (Brown et al., 2009).
The dimension of information perception (sensory/intuitive) deals with favorite types of information. Concrete learning content might be more beneficial for sensory students while abstract content may better support intuitive students in understanding and grasping knowledge. Facts, examples, simulation and interactive lessons are examples of concrete information. Abstract types of information can involve, for instance, theories, definitions, and mathematical notations. An application of this dimension was developed in a game-based VLE, and the central aim was to derive the information perception style in accordance with the observed behavior of students when interacting with the game (Feldman et al., 2014). Their results showed that students indeed differ in their favored types of information matching their observed behavior and their information perception styles. Though, the learning efficacy when integrating this particular dimension was not evaluated. As a result, this particular dimension has been taken into account in the work presented in this paper.

The adaptive framework
An adaptive framework is proposed to be used as a basis in the design of different instances of AVLEs. By using the framework, it is achievable to construct different modes and forms of adaptation, to represent different learning resources and to consider various student features. The framework is depicted in Fig. 1, and it involves three main modules including the content domain model, the student model, and the adaptation model.
The domain model represents learning resources in a specific structure that enables the AVLE to deliver adaptation. The student model concerns information about student features such as knowledge level, performance, learning style, and motivation. The adaptation model has the ability to deliver proper learning resources to students by considering data represented in both the student model and the content domain model.
The framework also has supplementary modules including the module of interaction data monitor and the interaction module. The module of interaction data monitor keeps tracks of all student actions in the system and then feeds into both the student model and the adaptation model for updates. The interaction module is fundamentally the graphical user interface that the student interacts with to gain some knowledge and understanding of the learning content.
The framework is designed primarily to be completely adaptable; its modules are not linked to a particular learning domain; student features or some adaptive mechanisms. The framework can be used as a basis for the design and deployment of AVLEs for different application domains. Therefore, the first research question (How can we design an adaptive framework for developing different instances of virtual learning environments?) is answered by the proposal of the adaptive framework in this paper taking into account previously available models (De Bra et al., 1999;2013). A specific implementation approach is offered in the next section to verify the applicability of the framework by offering an instance of an AVLE focusing on the components of the framework.
The auxiliary components include the interaction component (i.e., the graphical interface), interaction data monitor that feeds into the adaptation model and the student model for updates, and instructional material (i.e., the adaptation output).

Framework implementation
An implementation based on the proposed adaptive framework is presented in response to the second research question (How can the proposed adaptive framework be used to implement a virtual learning environment?). The three major modules of the framework (the content domain model, the student model, and the adaptation model) are detailed in the following sub-sections.

Fig. 1:
The proposed adaptive framework containing the core components including the student model, the adaptation model, and the domain model

Content domain model
The content domain model organizes and represents the learning material to be provided to students. Some critical points are taken into account when representing and building the content domain model in this implementation. First, the domain model should be independent of the application domain. In other words, any learning resources related to any course can be integrated. Second, the content domain model should allow for easy and usable management of learning resources. These resources can be updated without affecting the generation of adaptation. Third, the model should also use a simple description of learning content properties since current standards of learning resources may not well support adaptation (Simko and Bielikova, 2019). These standards were proposed chiefly for traditional VLEs. Sharing learning resources between different adaptive environments is out of the scope of this work. So, the provided description of the content domain model is used mainly for illustration. Although the provided description offers useful management of the domain model and can be used in other systems, different representations can be used in future developments of instances of AVLEs based on the proposed framework.
In order to complete the process of modeling the domain content, a few steps are needed including content domain model representation, specifying its properties and identifying the modeling approach. These steps are discussed as follows.

Domain model representation
The domain model is represented in a tree-like structure of four levels as shown in Fig. 2. Level one is the root and is called the course. Level two connected with the course node involves a number of Learning Units (LUs). Level three contains Learning Objects (LOs) connected with corresponding LUs.
A number of multiple-choice questions (quizzes) together with supplementary learning resources can also be associated with each LO in level four in order to offer applicable adaptive feedback on learning misconceptions.

Domain model properties
The domain model representation and content properties are fundamental to provide adaptation and to manage the learning content and resources efficiently. Current e-learning standards, such as IEEE LOM, do not fully support adaptation and are not flexible because of the complexity of adaptation (Rey-López et al., 2008). The content domain model in this approach is designed to be lightweight and independent of the application domain. In other words, any application domain and learning resources can be organized, stored and maintained within the content domain model to offer a personalized studying experience to students. A set of simple properties to describe the domain model content is proposed and presented in Table 1. For example, the course has three simple properties including the course ID, course title, course description and the language of instruction of the course. Similarly, the learning units that are related to the course have the properties of ID, the title of the learning unit and a description. The properties of LOs associated with learning units involve the set of the ID of the LO, LO description, LO type (abstract, concrete or mixed), location in the file system and prerequisite(s).

Domain modeling approach
The domain modeling approach is presented based on the proposed properties. This approach has two fundamental phases to provide adaptation and to construct personalized learning paths. The properties of the domain model are applied to calculate the content similarity degree between LOs using methods related to Natural Language Processing (Chowdhury, 2003) and Information Retrieval (Frakes and Baeza-Yates, 1992). Once the properties of the domain model content are provided, a pre-processing phase of these properties will be accomplished. Then, a content similarity degree between LOs will be calculated. The main objective of the modeling process is the smallest element in the domain model representation which is the LO in view of learning units. That is essentially what will be provided to students. In order to pre-process LO metadata, the title and description of its corresponding learning unit are retrieved and combined with the LO's title and description as well. This is accomplished for all LOs to ensure that learning objects that are related to a specific learning unit will have high content similarity degrees.
When the data are thoroughly combined, a number of steps are accomplished to the data including removing common linking words (e.g., of, a, the, it, etc.), punctuation symbols (e.g., ?, and, !, *…etc.) and performing word stemming (e.g., the word "played" becomes "play"). These data do not usually have useful meaning when estimating the content similarity degree between LOs so that they can be removed.
The Vector Space Model (VSM) (Salton et al., 1975) is employed to represent each learning object as a vector. VSM is one of the most popular models in the field of Information Retrieval (Frakes and Baeza-Yates, 1992). VSM is simple, yet it enables for more inexpensive computation, can intuitively calculate the content similarity between LOs and can priorities the retrieval process of relevant LOs to the student.
The content similarity degree between each two LOs can be computed based on Eq. 1. The weight of each word is calculated using TF-IDF (term frequency-inverse document frequency).
Where, Wt,lo is the weight of the term t in the learning object lo; tft,lo is the frequency of the term t in the learning object lo; LO is the total number of learning objects in the course; Nt is the number of LOs containing the term t; log ( ) is inverse learning document frequency (IDF). The content similarity degree of the learning object (LO1) vector to the learning object (LO2) vector is normally computed by the cosine of the angle between them based on Eq. 2. This equation is called the Cosine Similarity (Sim). . (2) Assuming that there are n LOs stored in the content domain model, the similarity degree between all LOs can be expressed by the matrix R:

] ×
For example, r12 indicates the content similarity degree between LO1 and LO2. The matrix R will be used by the adaptation model to provide relevant LOs and to construct personalized learning paths which are the main adaptivity feature.

Student model
Since most AVLEs in published research focus on a single student feature (Normadhi et al., 2019), the student model in this implementation varies in combining two student features including learning style and student performance. The student model representation is technically based on a combination of the stereotype model (Rich, 1989) and the overlay model (Brusilovsky and Millán, 2007). The learning style is represented as a stereotypical model. The student performance is tracked by maintaining the knowledge level which is represented as an overlay model. This model is assumed to be a subset of the content domain model. These two features are discussed as follows.

Learning style
The information perception dimension of the learning style (sensory/intuitive) of FSLSM, presented in Section 3, will be mainly used (Felder and Silverman, 1988). This dimension is considered to be the most applicable dimension of FSLSM, and it can be found in many models of learning style (Coffield et al., 2004;Felder et al., 2002;Kolb, 1984). Also, this dimension has a relationship to other factors such as management approaches, behavior characteristics, learning style and even career competencies (Feldman et al., 2014). On the contrary, this particular dimension did not receive much attention in related work in comparison to other dimensions of FSLSM (Akbulut and Cardak, 2012). So, the question is still open about the way needed to offer adaptivity according to this specific dimension of learning style either alone or when it is combined with other features such as knowledge level. This information perception dimension categorizes students into two groups: Sensory or intuitive. Students can have mild, moderate or strong characteristics toward a specific category. A student can be assigned to one of the defined four Stereotypes (S) for representing the perception dimension of learning style in the student model as follows:  S1: students who have a strong or moderate sensory style  S2: students who have a mild sensory style  S3: students who have a mild intuitive style  S4: students who have a strong or moderate intuitive style Learning style can be identified using a learning style questionnaire that is associated with FSLSM (Felder and Spurlin, 2005); it has 11 questions with two multiple-choices (a or b). Based on the answers of students, the value and type in the learning style dimension for each student can be determined. The key strategy is to calculate the number of responses for each choice.
For example, a student may select the option a for 8 times, and the option b for 3 times. Then, the value of the learning style dimension will be 8a-3b=5. The value 5 means that the student has a moderate sensory learning style based on equation L(value) below. Therefore, the student will be assigned to stereotype 1.

Student performance
The student model takes into account student performance by maintaining the knowledge level of each student in addition to learning style. The overlay model (Brusilovsky and Millán, 2007) is mainly used to represent the knowledge level of students. The knowledge level is assumed to be a subset of the content domain model. The student performance is dynamically tracked based on student-system interaction. In this representation, a qualitative overlay model-that represents the degree of how a student knows and understands such a LOclassifies the knowledge level into one of four degrees including (1) unknown, (2) partially learned, (3) learned and (4) mastered. Fig. 3 presents an example of an overlay model with the annotated knowledge level of LOs for a student. This is to target the drawback of the pure overlay model which categorizes the knowledge level as either be known or unknown. A possible approach to identify the knowledge level is by asking students to respond to some test items (quizzes). This method is also enhanced in this approach by allowing students to explicitly rank their understanding level (i.e., self-assessment) of LOs. Each LO is associated with a quiz (i.e., contains one or more test questions) and a feature that allows students to rank the LO. Once a student attempts a quiz and ranks its related LO, the student-system interaction data are collected, processed and then stored in the student model as follows:  A quiz. A student attempts a quiz related to a specific LO. The result of the quiz can be classified as: {Poor, average or excellent}.  Understanding ranking. The student explicitly rates a specific LO as: {Not understood, partially understood or fully understood}.
Once the student-system interaction data are stored, four production rules are applied to update and maintain the knowledge level in the student model, and the data will also be transferred to the adaptation model for timely adaptive feedback. Simply put, the production rules take into account quiz results and understanding ranking provided by the student as the main input, and a decision is made to identify the knowledge level based on the predefined four rules, as shown in Table 2.

Adaptation model
The adaptation model generates suitable sequences of LOs for each student. The model generates personalized learning paths based on student-system interaction with a view of data represented in both the content domain model and the student model. An initial learning path that contains a set of ordered LOs in accordance with learning style is generated. When the first learning path is completed by the student, another personalized learning path based on the knowledge level is generated and continually updated based on student performance. The adaptation model constructs two forms of personalized learning pathways: (1) an initial learning path and (2) a dynamic learning path. They are described as follows.

Initial learning path
The key purpose of the initial learning path is to provide a quick adaptive learning experience and to overcome the cold start problem when recommending appropriate LOs. The property of the LO type is mainly considered when constructing the initial learning path. The LO type may have three possible values: abstract, concrete or mixed. Abstract LOs can be more appropriate for intuitive students while concrete LOs are more appropriate for sensory students. A mixed LOs are provided to students who have a mild tendency to either category.
Based on the learning style representation in the student model (i.e., Stereotype (S) representation), an initial and automatic learning path is generated for each S based on the LO type. The orders in which LOs are placed in the learning path are different for each S as follows:  S1: (concretemixedabstract)  S2: (mixedconcreteabstract)  S3: (mixedabstractconcrete)  S4: (abstractmixedconcrete) S1 considers students who have a strong or moderate sensory learning style. The adaptation model orders the LOs that have "concrete" type first, then "mixed" LOs, and finally "abstract" LOs and insert them into a list that is generated and then offered as a learning path. S2 considers students who have a mild tendency to sensory learning style while S3 takes into account students who have a mild affinity to intuitive learning style. S4 considers students who have a strong or moderate intuitive style.
Once the initial learning path is built, a further step is made to check the prerequisites between learning objects in the constructed learning path. If there is a prerequisite found between any two LOs, their positions will be accordingly changed, and the sequence of LOs in the learning path is updated, and then offered to students as an initial learning path.

Dynamic learning path
The dynamic learning path is made according to the knowledge level of students. Once a student completes the first learning path, and that the student model updates the knowledge level of each LO studied, a new learning path is organized by the adaptation model to be recommended.
The approach of creating the learning path is to begin with the LO that a student has less knowledge first (i.e., unknown LO). If there is more than one LO with the knowledge level "unknown", the adaptation model will select a LO based on the LO identifier number (ID).
Then, the similar LOs (i.e., based on the similarity matrix between LOs) are ordered from highrelevancy to low-relevancy and added them to the learning path. Prerequisites between learning objects are also checked and the learning path is further re-constructed. The updated learning path can then be ready to be offered, and students will follow their learning paths until the knowledge level of all LOs available in the content domain model become "mastered."

An illustrative example
We assume that the content domain model contains six learning objects (LOs) and a subset of LOs properties (type and prerequisite) is provided as shown in Table 3. Mixed -5 Concrete -6 Concrete 4 The initial learning path for each stereotype based on the student model representation will be produced as presented in Table 4. The adaptation model will consider the learning style of a student and the LO type to construct the initial learning path. For example, student R has a sensory learning style, and the initial learning path for that student will start with concrete LOs, then mixed LOs and finally abstract LOs. The adaptation model will construct the learning path taken into account the LO metadata producing the sequence of LOs as follows: 356412.
The adaptation model will also analyze the prerequisites in that sequence and may found that to study LO3 the student should first study LO2, the path is then updated by placing LO2 before LO3 resulting in a new sequence: 235641. Still, LO4 is a prerequisite to LO6 in the path; so, LO4 will be placed before LO6 and the learning path will be further updated resulting in the sequence: 235461. The adaptation model will ensure that there is not any prerequisite left in the learning path and then present it to the student as an initial personalized learning path. It should be noted that the set of LOs in the learning path is relevant to a specific learning unit. The procedure will continually be produced for all LOs related to all learning units represented by the content domain model. When the student R completes the full learning path and studies each LO in the same order as provided by the AVLE, the knowledge level is updated based on student-system interaction and then saved in the student model. We assume that the knowledge level of R for each LO is provided as shown in Table 5.
The adaptation model will start with the LO that the student has less knowledge of. In this case, LO5 is "unknown" by the student R. The learning path will be re-constructed by ranking similar LOs to LO5 according to the similarity table as shown in Table 6 (the table is randomly generated for illustrative purposes). Based on the similarity table, the LOs will be ranked from high relevance to low relevance starting from LO5. The learning path will be constructed taken into account similarity measures as follows: 516324. Then, the adaptation model will check the path if there is any "mastered" LO by the student R; it will be omitted from the learning path to optimize the learning time. In this case, LO1 will be taken out from the learning path generating the sequence: 56324.  The next step is to check the prerequisites of each LO in the updated path. If there is a prerequisite found between any two LOs, the knowledge levels of both objects will be diagnosed. If the student has a better knowledge level for the prerequisite LO than the targeted LO (i.e., provided before its prerequisite in the learning path), no action will be performed by the adaptation model. Otherwise, the prerequisite LO will be placed before the targeted LO.
In this case, the student R has a better knowledge level of LO4 (i.e., a prerequisite for LO6) than LO6 (i.e., provided before its prerequisite LO in the path). Then, the adaptation model will keep the same learning path unchanged (56324). Still, the adaptation model may find that LO2 is a prerequisite for LO3, and the student R has a less knowledge level of LO2 than LO3, and that LO3 is placed before its prerequisite, LO2. Subsequently, the adaptation model will place LO2 before LO3 and update the learning path to be: 56234 and then recommend it to the student.
Once the student R finishes the learning path successfully, a new learning path will then be built with a new sequence of LOs. This is achievable iteratively until the student has a "mastered" knowledge level of all the LOs represented in the content domain model reaching the optimal case.

Evaluation method
An AVLE was developed based on the proposed adaptive framework. The AVLE was also evaluated through a carefully controlled experiment to answer the third research question (What is the effect of adaptivity when using the developed virtual learning environment by students in terms of perceived usefulness and engagement?). According to the research questions, two research hypotheses are put forward for this research as follows: Hypothesis 1: Adaptivity in virtual learning environments enhances the perceived usefulness of students. Hypothesis 2: Adaptivity in virtual learning environments enhances the learning engagement of students.
The experiment was managed with seventy-five Male undergraduate students studying in a computer science degree. All the participants were Males in order to control the gender variable. This is to eliminate the effect of variances between the experimental groups and to eliminate confounding factors on the experimental results.
About the experiment procedure, a number of experimental sessions were managed, and each session was lasted for about 100-120 minutes. There are two conditions in the experiment being the adaptive condition (treatment) and the non-adaptive condition (control). The treatment condition involved thirty-nine participants, and the control condition consisted of thirty-six participants. In the adaptive condition, the participants used the AVLE while the participants in the non-adaptive condition used a traditional VLE with there was no adaptivity. All participants studied learning material related to some concepts of Cryptography (i.e., the application domain). The material was new to all the participants in both experimental conditions, and the difference between these two conditions was the delivery of adaptation.
In order to explore the perceived usefulness of AVLEs, a questionnaire of ten items using a 5-point Likert scale was developed, as presented in Table 7. The creation and improvement of the questionnaire items involved three Human-Computer Interaction experts. Also, a Cronbach's alpha was managed to confirm the reliability of the questionnaire items having 0.769, which indicated good reliability. The questionnaire aims to assess the degree to which students perceive the usefulness of the AVLE and that it recommends them with relevant learning resources. In both conditions, the participants completed the questionnaire to find out whether students-irrespective of their assigned condition being adaptive or non-adaptive-think that the AVLE meets their needs and preferences so that the variable of perceived usefulness can be determined.
In addition, we report on the learning engagement of participants calculated based on the time spent on learning using the AVLE as an important factor in learning success. It is assumed that the more students spend on their learning process, the more they are engaged in learning (Kuh, 2009). Furthermore, we collected subjective comments by some participants who used the AVLE at the end of the experiment. The system recommends you to revise a specific lesson. 2 The system provides you with additional learning content to support your learning. 3 The list of learning lessons changes continually as you progress through the learning process. 4 The list of learning lessons is ordered according to your knowledge level and preferences.

5
The system provides you with relevant learning content at the appropriate time. 6 The system provides you with helpful feedback for your learning. 7 The provision of rewards motivates you to learn.

8
The system allows you to have some control over the learning process.

9
The system is better than other e-learning systems that you have experienced. 10 The system will be useful when applied to other topics such as Java programming.

Results and discussion
The overall results of the questionnaire indicated a statistically significant difference between the treatment (adaptive) condition (M= 4.26, SD= 0.52) and the control (non-adaptive) condition (M= 3.86, SD= 0.66); t (73)= 2.93, p= 0.004, as assessed by an independent sample t-test at the level of 0.05. These results suggest that the students perceive that the AVLE meets their needs and preferences by providing more relevant learning material than the non-adaptive version.
As to further investigate the questionnaire items, each item results are reported, as shown in Table 8. Out of the ten items, five items have statistically significant findings. For example, the results of the item 'the system recommends you to revise a specific lesson' in the adaptive condition is better than the non-adaptive condition. This indicates that participants perceive that the adaptive version provides them with relevant learning material in comparison to their peers who used the nonadaptive version. According to the results of the item, 'the system provides you with additional learning content to support your learning,' there was a statistically significant finding. It was expected that the adaptive version provides participants with more supportive material based on their progress through learning than those who used the nonadaptive version. Also, positive findings were also obtained for the item 'the system provides you with helpful feedback for your learning,' the item 'the provision of rewards motivates you to learn' and the item 'the system is better than other e-learning systems that you have experienced.' However, five items have no much difference between the treatment (adaptive) condition and the control (non-adaptive) condition. For example, the item 'the system allows you to have some control over the learning process' had similar results comparing the two conditions. This is not surprising since participants were requested to study the learning material in the same sequence as recommended by the AVLE. Therefore, their control of the learning process is limited and is out of the scope of this study. Nevertheless, the overall results of the ten items indicated better results for the AVLE in comparison to the non-adaptive version. According to the results of the questionnaire, hypothesis 1 is confirmed. It can be suggested that adaptivity in virtual learning environments enhances the perceived usefulness of students.
Regarding the results related to learning engagement (based on time spent on learning in seconds), there was a statistically significant difference between the adaptive condition (M= 3668.33,SD= 967.278) and the non-adaptive condition (M= 2520.17, SD= 863.490); t (73)= 5.406, p= 0.000, as assessed by an independent sample ttest at the level of 0.05. It was found that participants were better engaged in learning using the adaptive version of the AVLE in comparison to the participants who used the non-adaptive version. Therefore, hypothesis 2 is confirmed, and it can be suggested that adaptivity in virtual learning environments enhances the learning engagement of students. Furthermore, there were some positive comments provided by participants who interacted with the adaptive version. Based on the results of the perceived usefulness, learning engagement, and the participants' comments, it can be suggested that the proposed framework was successfully validated by the implemented AVLE generating positive findings.

Conclusion
This paper contributed to the current literature by the proposal of an adaptive framework for developing adaptive virtual learning environments, by the implementation based upon the framework and also by the experimental evaluation.
The proposed adaptive framework can be used to design and implement different instances of adaptive virtual learning environments. The paper also offers an implementation based on the adaptive framework. The implementation confirmed the applicability of the framework by covering its three modules including the content domain model, the student model, and the adaptation model. The content domain model was designed to be flexible and manageable to integrate different learning resources that can be linked to any application domain. The specified aspects of the design of the content domain model allowed for the provision of adaptivity. The student model incorporated two student features including learning style and knowledge level. A specific dimension of learning style called the information perception (sensory/intuitive) was integrated into the student model in addition to the knowledge level. This particular integration is original and uniquely related to the implementation offered in this paper. The main adaptive feature was the production of the personalized learning pathways by the adaptation model in order to be recommended to students in addition to adaptive feedback. The adaptive virtual learning environment was capable of generating different sets of learning paths tailored to the learning style and knowledge level of students.
Since the paper presented a successful validation of the framework by the implementation, it is also possible to generate different instances of adaptive virtual learning environments by integrating different student features, by providing different adaptivity mechanisms and by representing different learning resources that are not only related to a particular application domain. This validation demonstrated the usefulness and applicability of the adaptive framework in designing such adaptive environments and in contributing to educational technology research. Additionally, an experimental evaluation was conducted with seventy-five students in a learning context yielding promising results in terms of student perception toward using the adaptive virtual learning environment and learning engagement. Positive comments were also obtained from the students who used the adaptive virtual learning environment. The two hypotheses proposed in this research are confirmed. It can be suggested that adaptivity in virtual learning environments enhances the perceived usefulness and learning engagement of students.
Although the work presented in this paper generated positive findings on the effect of adaptivity, more studies are needed to generalize the results to different application domains, different types of students and to explore other experimental variables. This work presented a foundation for future directions of research either by improving the proposed framework to integrate collaborative, adaptive, and gamified features by generating different instances of virtual learning environments focusing on the main components of the framework or by replicating the offered experiment following the same procedure. This paper provided an initial experimental evaluation that can be further extended in future experiments. Specifically, future work will involve a longer-term experimental evaluation in a learning context with a larger sample size that incorporates both Males and Females using the implemented adaptive virtual learning environment based on the adaptive framework. Different application domains and experimental variables can also be considered such as learning gain, student satisfaction and perceived usability.