Students perceptions of audience response system in classroom feedback: A qualitative study

Article history: Received 2 December 2017 Received in revised form 7 February 2018 Accepted 13 February 2018 The aim of this study was to explore students’ attitudes to formative feedback from classroom questions, peer interactions, and post-lecture sessions. We used a qualitative approach involving focus groups interviews. A complete cohort of second-year nursing students (n = 120) attending a therapeutic communication course as part of a Bachelor of Nursing Program participated in this study. Of them, 49 volunteered to contribute to focus group interviews. NVivo (QSR International, Melbourne, Australia) was used to organize data using the content analysis method. All students were women aged 18–24 years. Most considered the ARS a useful tool that aided learning by providing instant, anonymous feedback. Some felt that there were barriers to peer interactions, including lack of interest in participating or not knowing the answers. In conclusion, ARS use improved students’ learning and encouraged a deep learning approach. Post-lecture feedback was most influential, followed by feedback from classroom questions, and lastly feedback from peer interactions.


Introduction
*Lectures are the most commonly used method of transferring information to nursing and medical education students (Bove, 2008;Charlton, 2006). Traditional or "didactic" lectures are a one-way means of delivering knowledge to students, and are increasingly considered an ineffective method of teaching (Abdel and Collins, 2017). Didactic teaching is based on the assumption that teachers bring about learning, and no power is given to the learners (Charlton, 2006). However, studies have suggested that this method of teaching causes students to adopt a surface learning approach, in which they simply memorize material divulged in lectures to pass assessments (Biggs, 1989). Didactic lectures are further criticized because they inhibit peer interactions, which enable the construction of new information, and student's attention spans, which lead to poor recall of information (Abdel and Collins, 2017).
In contrast, in interactive lectures, students adopt a deep learning approach through interaction with the lecturer and with their peers (Biggs and Tang, 2003;Ko et al., 2017). Interactive lectures encourage students to communicate and interact, which leads to a better understanding of the subject (Abdel and Collins, 2017;Egelandsdal and Krumsvik, 2017;Ko et al., 2017). The outcomes of interactive lectures are influenced by students' approaches to learning (Biggs, 1989;Marton et al., 1984). Many studies have reported that learning approaches differ from one student to another (Biggs, 1989;2003;Marton et al., 1984). This is due to students' learning preferences and the method of teaching material delivery (Biggs and Tang, 2003). Therefore, it is good practice for teachers to be aware of this fact in order to facilitate, encourage, and influence students' learning through the implementation of "student-centered" approaches (Biggs, 1989;2003;Marton et al., 1984). Meyer (2004) summarized factors for learning among students as individual differences in their learning intentions, prior knowledge, epistemologic beliefs, and motivations. All of these factors influence the overall intended learning outcomes and hence affect the quality of learning and assessment results (Marton et al., 1984;Lizzio et al., 2002).
Audience Response System (ARS) technology enables the delivery of interactive lectures. ARS tools, such as Learning Catalytics (LC; Pearson UK, London, UK), provide students with individualized feedback via their own smartphones, laptops, and tablets (Abdel and Collins, 2017). This innovative technology communicates wirelessly through an application (https://learningcatalytics.com). Questions are created in or uploaded to the LC platform, and can be delivered to students at any point during the teaching session. ARS tools can be used to check and reinforce understanding, and any problems can be addressed immediately (Sawdon, 2009). Students make an interactive choice, yet retain the security of anonymity (Sawdon, 2009). This technology benefits both the teaching faculty and students. ARS tools allow the teaching faculty to engage large numbers of students by ensuring their anonymity, perform formative and summative assessments, gain insight into common errors, and provide instant feedback. ARS use has been shown to improve students' satisfaction through the provision of instant feedback and increase knowledge retention (Ismaile et al., 2017;Sawdon, 2009).
Indeed, formative feedback aids the learning process (Grzeskowiak et al., 2015). Providing formative feedback in classrooms allows learners to reorganize their understanding and adopt a deep learning approach (Biggs and Tang, 2003). Formative feedback is given not only by educators, but also by peers. Peer interactions during interactive lectures also provide formative feedback that aids students' learning. However, formative feedback may inhibit learning in some circumstances (Hattie and Timperley, 2007;Shute, 2008). This depends on how the information is given and understood by students (Hattie and Gan, 2011).
Although ARS technology is commonly used to provide formative feedback in higher education (Grzeskowiak et al., 2015), there is limited information on its efficacy in the Arab region. In particular, no studies have been conducted to date to investigate students' perceptions of ARS tools in nursing education in the Arab region (Grzeskowiak et al., 2015). Therefore, in this study, we explored students' attitudes to formative feedback provided by an ARS. We used a qualitative approach involving focus groups to explore students' attitudes to formative feedback from classroom questions, peer interactions, and post-lecture sessions. The use of focus groups enabled students to share, interact, and discuss the utility of ARS technology in the learning process. The objectives of this study were: first, to explore students' experiences of ARS use to support their learning needs; and second, to identify students' perceptions of formative feedback from pre-lecture classroom questions, peer interactions, and post-lecture sessions.

Design
We performed a qualitative study using focus groups to explore students' views on the efficacy of ARS use in the learning process. Focus groups were used to facilitate communication and interaction between students. This allows the expression of different thoughts and experiences, and may reveal important information (Shute, 2008).

Population and context
The study took place at the College of Nursing of the Princess Nourah bint Abdulrahman University (PNU), Riyadh, Saudi Arabia. A complete cohort of second-year nursing students (n = 120) enrolled in a therapeutic communication course as part of a bachelor of nursing program were invited to participate in this study. All of the students were women.
Fourteen lectures were delivered as part of the therapeutic communication course between September and December 2017. One hundred fiftysix questions in both multiple-choice and true-orfalse formats were uploaded to the LC platform. The questions assessed the learning outcomes of each lecture. Before the start of the lecture, the students were invited to login to the LC platform using an ID and password given to them by the lecturer in the first lecture, which they continued to use for the rest of the course. The educator started each lecture by highlighting its learning outcomes, and then delivered the teaching material. The ARS questions were posted at the start of each lecture to estimate students' prior knowledge (Sawdon, 2009). The ARS also assessed students' understanding of topics already covered by adding questions from previous lectures to the questions on the current lecture. This linking of prior knowledge with the knowledge gained in each lecture encouraged a deep learning approach (Biggs and Tang, 2003). By evaluating students' responses to these pre-lecture questions, the teacher was able to adapt the content of the teaching material in areas where more or less information was required. Also, students were encouraged to interact with their peers in discussing the questions.
In addition to its implementation pre-lecture, the ARS was also used in a post-lecture revision session (Eggert et al., 2004). Students were allowed to retake the test and answer each question as often as required, with the aim of enhancing the learning experience and improving their knowledge (Egelandsdal and Krumsvik, 2017). This revision session provided the students with feedback on their progress (Eggert et al., 2004). If a student did not score well or felt that they were not grasping the concepts, they were encouraged to attend supportive tutorial sessions (Eggert et al., 2004).
Of the 120 participants, 49 volunteered to take part in focus groups interviews. The students were divided into seven groups of seven students. To allow a more open-ended discussion of students' perceptions of ARS use, semi-structured questions were used. The researcher (SI) was the moderator of all focus groups, and audio recordings were made of all focus group interviews. Copies of the audio recordings were made available if requested by students. Focus group interviews took place in a pre-booked classroom at the College of Nursing of the PNU. The seating was arranged in a circle.

Qualitative data collection
Each focus group session lasted 30-45 minutes. Upon arrival, the students were informed of the purpose of the study, session format, and confidentiality related issues. A consent form was signed by each student before the start of each session.

Qualitative data analysis
In qualitative research, data analysis aims to examine issues, understand phenomena, and answer related questions. Such studies use unstructured information, such as focus group interview transcripts, to produce key themes and subcategories. In this study, we used the qualitative data analysis software NVivo (QSR International, Melbourne, Australia) to analyze the large amount of raw data collected as summarized in the preceding section. The software enabled us to manage, shape, and spot correlations in the information gathered rapidly and effectively. NVivo was also used to encode, structure, store, and organize the categorized data.
The content analysis method was used based on textual data or "data mining" from the audio recordings of the interviews. The interviews were analyzed by meaning categorization to identify major issues, and sampling continued to saturation where possible. Thematic saturation, with no different themes emerging or themes replicating, was attained after seven focus groups. Data saturation was confirmed with the research team. Data collection and analysis were predominantly conducted by SI and FA. Transcription was performed by an administrator. The participants were provided with unique identifier to retain anonymity, and no identifying details were present during the transcription process. All students were given the opportunity to approve their transcripts before publication. Ethical approval was provided by the institutional review board of the College of Nursing of the PNU.

Results
In this study, we explored the attitudes of undergraduate nursing students enrolled in a therapeutic communication course to formative feedback provided by an ARS. Table 1 presents the students' ages and their distribution in the focus groups. All of the students were women, and most ranged in age from 18-20 years (mean: 19.67 years; standard deviation: 1.83 years).
We examined the data collected from seven focus group interviews to determine the students' attitudes to ARS feedback. Three principal themes emerged from this analysis: students' attitudes to formative feedback from pre-lecture questions, peer interactions, and post-lecture revision sessions. However, it should be noted that these themes affect each other and, consequently, it was impossible to completely separate them (Fig. 1).

Formative feedback from pre-lecture classroom questions
The most prominent theme to emerge from the analysis of the focus groups interviews was related to students' perceptions of formative feedback from the pre-lecture classroom LC questions. The students considered the ARS a useful tool that aided them in the learning process by providing instant, anonymous formative feedback. Participants stated, "It's great; it helps us to understand the main concepts. We are aware of what we need to learn." (Student 3A) and "I understand more and focus on my mistakes." (Student 7C). A student agreed, "The lecture material is clear to us and we understand it more." (Student A). Another added, "I enjoyed it because I understood the information very well." (Student 1F).  As stated earlier, literature is limited regarding the use of ARS tools to provide formative feedback in nursing education. This is particularly true in the Arab region (Grzeskowiak at el., 2015). In this study, participants stated that, "We understood the class [using ARS] more than any class… in this program." (Student E5). ARS technology was introduced into one course in the knowledge that it had never been used in the nursing program or in any other medical or non-medical program at the university where the study took place.
Also, the students reported that the feedback given by the ARS helped them to be aware of their learning needs and made the main concepts clearer: "I thought I understood, but then it came out that I didn't when I received the answer on my mobile… Then it was clear to me." (Student 1B). Some students asserted that, if they were given more time (more than 45 seconds per question), they would have answered more questions correctly.

Fig. 1: Feedback themes from focus group interviews
The students reflected on the types of question uploaded to the LC platform. They stated that the multiple-choice questions were well written and a challenge to answer, which improved their focus. Many students reported that they found it a fun way of learning. A student stated that she learned more when using her mobile, and that the platform was easy to use: "It's easy… using my mobile and logging into the session." (Student A7). Another stated that she loved to use her tablet to learn, and that she would have preferred more questions: "I love using my iPad, it helps me to focus." (Student F2). However, one student stated that she had poor internet connection, which affected her experience in getting formative feedback. Also, another student stated that her mobile was almost out of battery. However, it lasted and she was able to experience the provision of formative feedback by the ARS.
Further advantages to the use of ARS technology included its anonymity, instant feedback, and interactivity. Participants stated, "It's private, we can answer and no one knows it that it is." (Student G4). A student added, "We immediately have the answers, it's fast, our teacher explains it even further." (Student G1), and another agreed, "It's active… I learn better this way." (Student G3).

Formative feedback from peer interactions
The students in the focus groups perceived peer interactions to be a valuable part of the formative feedback provided by the ARS. Three main themes emerged from the analysis of these data: that peer interactions enabled reflection on their own understanding, interactive learning, and face-to-face explanation. The students stated they learned from each other, and that this gave them the opportunity to reflect on their understanding. However, a few students disagreed, and asserted that there were barriers to peer interactions, such as a lack of interest in taking part or not knowing the answers.
The students discussed the benefits of peer interactions facilitated by ARS use. One student stated, "The best thing is, we can discuss with other students what the correct answer could be, then we can explain it. I understand it better." (Student E2). Another student agreed, "Yes, usually I sit alone during the lecture, I think am I getting this point correct or not? Having [the ARS] in our course made me understand the lecture much better because I can ask my friends in the class and they help me to understand." (Student E7). Another participant added, "I misunderstand few things… by asking my friend to explain my misunderstanding, things become clear to me." (Student E1).

Formative feedback from post-lecture revision sessions
Post-lecture revision of previous and current knowledge was greatly appreciated by the participants. Moreover, the students stressed that the formative feedback provided by their educator and explanations of both correct and incorrect answers helped them to understand concepts even better. In addition, the students agreed that ARS use helped their teacher to track students' understanding: "We revise previous lectures and this is cool. It helps us to memorize knowledge and connect it to our current lectures. We love this." (Student F7). Another student agreed, "Our teacher explains the correct answers and also tracks students' performance. She explains all the multiplechoice options and clarifies why they weren't correct." (Student A1). The participants also reported that the ARS was a great tool for revising for their summative exams. They added that it helped them to be aware of their individual learning needs. Finally, all the students in the focus groups agreed that they valued being active in lectures, and suggested that the ARS should be implemented in other courses in their nursing program.

Feedback from Learning
Catalytics Action

Students' perceptions of formative feedback given by the audience response system
ARS technology is utilized for its effective feedback and ability to enhance learning (DeBourgh, 2008). Nursing students are expected to reflect on feedback and set goals in order to improve their learning and adopt a deeper and more active approach to learning. According to Hughes (2005), feedback influences learning; in fact, it may be the single most important factor. Students need both positive and negative formative feedback, which allows them to reflect on their knowledge and stimulates future learning (Hughes, 2005). It is essential to encourage students to reflect on feedback and develop targets for future work, because not all students do this (Higgins et al., 2002;Hughes, 2005). ARS technology distributes instant negative and positive feedback to large numbers of students during lectures to improve their quality of teach (Higgins et al., 2002). This enhances students' learning experience by providing individualized feedback on their progress throughout the course (Higgins et al., 2002). In this study, all of the students stated that they were satisfied with the feedback provided by this technology, which aided their understanding. In fact, many students stated that they achieved deep learning by employing previous knowledge to aid in the understanding of new knowledge. Consistent with this, previous studies have reported that ARS use improves students' satisfaction via the provision of instant feedback, in addition to improving their knowledge retention and encouraging a deep learning approach (Biggs and Tang, 2003).
The nursing students reported that the formative feedback highlighted areas in which they were struggling as well as providing positive feedback on areas in which they were excelling. According to Hughes (2005), this enables students to adapt their self-directed study to meet their learning needs.
Regarding formative feedback from peer interactions, although some students agreed that peer interactions provided them with positive feedback that aided the learning process, others reported concerns regarding peer feedback. The barriers to peer interactions that they identified included a lack of interest in taking part in answering questions or a lack of preparation or knowledge on the topic of the lecture. Such circumstances may inhibit peer interactions and render the formative feedback ineffective. Similar results were reported by Abdel and Collins (2017).
The most valued and cited formative feedback was that from the post-lecture revision sessions. All seven focus groups agreed that post-lecture revision enhanced their understanding, and they particularly praised the explanation and clarification given by the teacher. Similarly, Atlantis and Cheema (2015) reported that the provision of formative feedback and explanation of both correct and incorrect answers by the educator increases students' understanding of the knowledge learned. This improves participants' awareness of their individual learning needs (Atlantis and Cheema, 2015). Moreover, students stated that teachers were able to track their understanding through use of the ARS. Many studies have confirmed these results (Atlantis and Cheema, 2015;Egelandsdal and Krumsvik, 2017).

Limitations
The results of this study were based on focus group interviews. The participants were all women, and they related their experiences of using the ARS in just one course. Therefore, this may have introduced bias into our results. The focus groups were run by the researcher (SI), who was also the teacher of the communication course. This may have affected the results of the study. However, it was emphasized from the outset that the interviewer's role was that of a researcher and not of a teacher. This relationship and sensitivity was achieved throughout the focus group interviews.

Conclusion
In conclusion, this study explored students' attitudes to formative feedback from classroom questions, peer interactions, and post-lecture sessions. The students reported that using the ARS improved their learning and encouraged them to adopt a deep learning approach. The most influential formative feedback was given in post-lecture sessions, followed by feedback from classroom questions, and lastly feedback from peer interactions.
We recommend a follow-up study on the barriers to and promoters of the use of students' mobile devices, such as smartphones, tablets, and laptops, as tools to facilitate ARS use. Moreover, a study on the perceptions and attitudes of members of the teaching faculty regarding the use of mobile devices, especially smartphones, during lectures may be beneficial.