Examination of Student Satisfaction with e-courses by Clustering Analysis

With the developing technology, information-and-communication technologies-oriented e-learning practices have increased in education institutions. E-courses, one of the e-learning applications, are becoming increasingly widespread as a complement and alternative to formal education. Due to the pandemic that emerged in the world and in Turkey in 2019 and the subsequent transition to distance education by suspending face-to-face education, awareness and prevalence of e-courses have grown even more. During the confinement period resulting from the COVID-19 virus, distance, isolation, and uncertainty were inevitable among the general population (Bautista et al., 2022; Vásquez et al., 2023). In the e-learning environment, courses take place synchronously and asynchronously (Yildirim, 2020). While synchronous courses are conducted in an interactive online learning environment, allowing students and teachers in separate places to meet simultaneously, asynchronous ones are delivered at separate times and in separate locations. Student satisfaction, which provides feedback to make e-courses effective, efficient, and attractive, is one of the primary factors in ensuring the sustainability of these courses (Bolliger et al., 2010; Moore & Moore, 2005). High levels of satisfaction may be associated with a higher likelihood of success in the learning process, and students with high levels of satisfaction are likely to exhibit better academic performance (Chen et al., 2008; Martín et al., 2015).

Student satisfaction can be expressed as the “perceived value” of the education they receive (Bollinger, & Erichsen, 2013; Ilgaz, 2008). When the literature is reviewed, it is observed that the factors affecting students’ satisfaction levels regarding the e-learning environment are evaluated from different perspectives. Chua and Montalbo (2014) address these factors as learner interface, learning community, content, and usefulness. Gülbahar (2012) discussed learner satisfaction in four dimensions: teaching process, interaction with the teaching content, delivery and usefulness, and evaluation. Kolburan and Deveci (2015) analyzed the factors affecting students’ satisfaction levels in five dimensions. These dimensions are the materials and communication tools used in the online environment, design, attitude towards the e-course, course content, teacher-student interaction, and teaching process. A sustainable system of higher quality can be achieved by examining the factors that affect the students’ satisfaction levels in the e-learning environment, making arrangements for a more effective and efficient teaching process (Donavant, 2009; Gülbahar, 2012; Martín et al., 2015). Today, the continuous increase in the number of online courses and student demands indicates the requirement for further relevant research. Determining the clustering trends of the satisfaction variables of students taking e-courses can contribute to the elimination of the deficiency in the literature.

In today’s world, where data have been multiplying rapidly, the searching process for correlations that allow accessing information from large-scale data, that is, making projections about the future from large piles of data through software, denotes data mining (Terzi et al., 2011). Data mining (DM), which is employed in every field from education to finance, from health to business, is the extraction of implicit, not-so-clear, previously unknown yet potentially useful information from the available data. Having its basis in statistical methods, DM is used in the analysis of structured and unstructured data. Among the data mining models, clustering analysis is a multivariate method that classifies grouped data based on similarities and differences. This classification allows for an easier understanding of data stacks and more efficient use of information (Çetintürk, & Gençtürk, 2020). In the literature, there are numerous studies using cluster analysis, but it has not been used sufficiently in education. Antonenko et al. (2012) stated that cluster analysis could function as a data mining technique in educational technology research, yet the clustering algorithm should be chosen carefully, and cluster validation analysis was essential for obtaining valid and reliable results. Satisfaction-related studies conducted with cluster analysis in the literature are as follows:

In their study investigating the social dimensions of distance education in high schools, Lidegran et al. (2021) identified three clusters: urban upper-middle-class, immigrant working-class, and rural working-class.

Agyapong (2021) divided students into three clusters, which were highly, moderately, and least satisfied based on preferences (price, quality, packaging, and social bonding) that influenced their distance education program choices. Students in the highly satisfied cluster were driven by the education program’s service quality and competitive fee structure.

In their study, Çakir et al. (2018) determined the motivation levels of distance learners using cluster analysis and obtained three clusters as low, medium, and high levels. They reported that the reason for the dissatisfied group with low motivation was the lack of interaction, negative perceptions, and dependence on traditional education, while the students with high motivation had high satisfaction due to reasons such as being independent of time and space, investigation-suitable, effective, and contemporary learning.

Unsihuay and Blanco (2021) aimed to characterise students taking a basic statistics course in the first cycle of virtual classes and to determine which variables affect their academic performance. They identified three clusters of students who differed according to age, anxiety and stress scores, academic background and depression. They also found that the most influential attributes in predicting academic performance were the weighted semester average of the previous semester, the course they were enrolled in, age, the number of credits enrolled, stress and anxiety.

Kaya (2022), using Ward’s minimum variance clustering method, classified the differences in attitudes toward the use of distance education environment into three clusters: effectiveness and satisfaction, motivation, and dissatisfaction.

1.1. The aim of the research

The present research aimed to analyze the satisfaction levels of undergraduates for e-courses by K-Means cluster analysis with reference to different variables. In this context, the researchers determined the following questions:

1. What is the satisfaction level of the students regarding e-courses?
2. Do the clustering tendencies of students differ with respect to their satisfaction with e-courses, gender, class, computer ownership, computer and internet literacy, internet usage duration, way of connecting to the internet, and the tools students use in the learning management system?

This research was conducted with a descriptive approach and using a correlational research model. The correlational research is a research method that aims to determine the presence and/or covariation between two or more variables (Karasar, 2006). The research took place on a voluntary basis with 530 students studying in different faculties of a state university during the spring semester of 2021-2022. Due to missing data in the questionnaires of some students, the number of attendees whose data were processed was 522. Ethics committee approval was obtained before starting the study. Among the students participating in the study, 1st graders received at least four courses via distance education, 2nd graders received at least two semesters, and 3rd and 4th graders received at least three semesters of all courses via distance education. Descriptive statistics on the demographic characteristics of the students are given in Table 1.

TABLE 1. Demographic details of the students.

2.1. Data Collection Tools

The research data were collected through the Satisfaction Scale for E-Courses developed by Kolburan and (2015). It is a 5-point Likert scale and comprises 35 items and 5 sub-dimensions. Materials and communication tools utilized (8 items), Attitude towards e-course (6 items), Environment design (8 items), Teacher-student interaction (4 items), and Course content and teaching process (9 items) make up the sub-dimensions. The data were first analyzed for reliability, and Cronbach’s alpha (α) was found to be =0.923, indicating a “good” level of reliability.

2.2. Data Analysis

The data were analyzed using the SPSS 21 software. Mean, standard deviation, and clustering analysis were used in data analysis. The significance level was taken as 0.05. The range of arithmetic means for the E-Courses Satisfaction Scale was interpreted as: 1.00-1.79 (strongly disagree), 1.80-2.59 (disagree), 2.60-3.39 (somewhat agree), 3.40-4.19 (agree), 4.20-5.00 (completely agree). K-means clustering, one of the data mining techniques, divides a data set consisting of n data objects into K clusters given as input parameters. Partitioning mainly aims to group the units, based on their characteristics, in a way to have the highest level of intra-cluster similarity and the lowest level of inter-cluster similarity. This grouping renders the data into useful summative information that is convenient to research.

The descriptive statistics of the Satisfaction Scale for E-Courses used in the research and its sub-dimensions are depicted in Table 2. According to this table, the students’ mean satisfaction score in “Environment Design”, one of the sub-dimensions of the satisfaction scale, was found to be at a good level (Agree), while it was determined to be at a moderate level (Somewhat agree) in the overall scale and in other sub-dimensions. This finding indicates that student satisfaction with distance education is low across the whole scale.

TABLE 2. Perception levels of students regarding their satisfaction with e-courses.

The findings of the analysis performed with K-Means to reveal the satisfaction of students taking online courses in e-learning environments are presented below. In the study, the findings with regard to the variables of “gender, grade, computer ownership, computer and internet literacy, internet usage duration, connection way to the internet, and the tools used in e-learning environments” are as follows. Table 3 depicts the inter-cluster distances found as a result of the analysis. When this table is examined, it is observed that Cluster 3 (the group with low computer literacy and internet usage durations) and Cluster 4 (the group with the highest number of females, first graders, and the ones with the “good” and “very good” internet literacy levels) are the closest clusters (0.574), while Cluster 1 (the group with the highest computer ownership rate and the highest computer literacy) and Cluster 2 (the group with the lowest computer ownership rate and the lowest computer literacy) are the most distant clusters (2.021). Accordingly, students in Clusters 3 and 4 indicate far more similarity, while those in Clusters 1 and 2 are the least similar.

TABLE 3. Distances between final cluster centers.

Table 4 gives the ANOVA test results indicating the difference between the clusters created based on K-Means clustering. The value obtained from the Bonferroni test, one of the post hoc techniques of the ANOVA test, reveals that all clusters differ significantly from each other (F=1375.2; p<.001).

TABLE 4. ANOVA test results demonstrating the difference between clusters in terms of E-course satisfaction.

Table 5, which depicts the clusters the students belong to and the demographic characteristics of the students in the clusters, reveals that there are 4 clusters, and Cluster 3 has the highest number of students (196) while Cluster 1 has the lowest number of students (64). The Final Cluster Centers column in the table reflects the mean values of the students attending e-courses in 4 clusters. Accordingly, it is evident that the mean value of the students in Cluster 1 is the highest while the mean value of those in Cluster 2 is the lowest.

TABLE 5. Clusters formed as a result of K-Means analysis with respect to the learning environment satisfaction levels of the students attending e-courses.

When Tables 5 and 6 are analyzed together, it is noted that Cluster 1 has the highest mean (4.26), and their satisfaction is at the “good” level. It is the group with the highest rate of computer ownership (90.6%) and the highest computer literacy (57.9% at a good - very good level). The duration of using the internet within the cluster along with Cluster 3 is lower than other clusters (46.9% between 1-5 hours). This group makes the least use of the faculty’s internet connection service (23.4%), and they usually have their own internet packages. It is the Cluster 1 that watches lecture recordings (73.4%), animations (14.1%), and videos (40.6%) the most, does homework the most (62.5%), and uses questionnaires the most (21.9%).

The satisfaction level within the 2nd cluster, which has the lowest mean (2.24), is low. This group has the lowest rate of computer ownership (72%) and the lowest computer literacy (good - very good level 35.4%). Their internet usage durations are higher than other clusters. Moreover, this group takes the longest time for synchronous courses (90.2%), while spending the least time on word processing documents (11%), web pages (15.9%), exams (39%), graphics and pictures (15.9%), homework (45.1%), animations (6.1%), videos (23.2%), and emails (9.8%). Thus, one can infer that this group follows the course just to ensure the course attendance requirement.

Cluster 3, having a moderate satisfaction rate (2.92), is the second-most populous cluster. The students in this cluster have low computer literacy (59.2%) and internet usage duration (45.9%), while the 1st graders’ rate is higher than the other grades, and they mostly use their own internet packages. The group spends the least time watching the course records (44.4%) and uses the questionnaires the least (11.7%).

Cluster 4, with a “good” satisfaction level (3.5), is the most populated one. This cluster has the highest rate of females (80.6%), 1st graders (33.9%), and those with internet literacy at good and very good levels (77.8%). This cluster ranks second in terms of computer ownership (83.9%) and computer literacy (good - very good 43.3%). At the same time, the group has the longest time staying connected to on the internet (over 5 hours by 65%). It is the group that follows synchronous courses the least (83.3%) and uses the chat tool (31.7%), videos (48,3%), word processing documents (21.7%), and web pages (35.6%) the most.

TABLE 6. Learning Management System (LMS) tools used by students in clusters generated by the k-means method.

In this study where it was aimed to analyze the satisfaction levels of undergraduate students attending distance education courses during the pandemic by using cluster analysis with respect to different variables, it was found that student satisfaction with distance education was at a moderate level. Upon the review of relevant studies in the literature, it is observed that there exist different levels of satisfaction. Kaya (2022) identified that university students adapted to distance education during the COVID-19 period, and the satisfaction rate was high. Harsasi and Sutawijaya, (2018) reported that the influencing factors on students’ satisfaction levels are the course structure, flexibility of online education, and quality of technology; they also noted that more attention should be paid to the online education quality, particularly in terms of image quality, ease of use, and teacher-student interaction. Zhou (2022), on the other hand, stated that inadequate learning environments and poor student-teacher interaction are the foremost factors that prevent student satisfaction.

According to the results of the K-Means cluster analysis, 4 separate clusters were specified. Among these clusters, the students in Clusters 3 and 4 were revealed to be more similar to each other, while those in Clusters 1 and 2 were the least similar to each other.

It was observed that the students in Cluster 1 had the highest satisfaction level (agree - good), while those in Cluster 2 had the lowest satisfaction (disagree). Regarding the characteristics of the students in the obtained clusters, the students in Cluster 1 have the highest computer ownership rate and the highest level of computer literacy, and along with cluster 3, the shortest internet usage duration, compared to the other clusters. This group uses the Faculty’s internet connection service the least, and they usually have their own internet packages. Also, this cluster watched lecture recordings, animations, and videos the most, did the homework assignments the most, and used the questionnaires the most. The fact that the students in this group frequently use the resources relevant to the course content and have a high level of satisfaction in personalizing the education they receive indicates that they feel extremely interested in the course content when using this system with distance education. According to the literature, student-content interaction is the strongest predictor of satisfaction level (Kuo et al., 2014). Karabatak et al. (2020) stated that the homework-supported-distance-education process enhances academic success more, students’ academic satisfaction and attitudes are positively affected, distance education alone is not enough, and it should be supported with diverse techniques and methods to increase its effectiveness.

Having the lowest mean scores, Cluster 2 is the group with the lowest computer ownership and the lowest level of computer literacy. Their internet usage durations are higher than other clusters. It is the cluster where the highest number of synchronous courses are taken, yet the least time is spent on word processing documents, web pages, exams, graphics and pictures, homework, animations, videos, and e-mails. Thus, one can infer that this group follows the course just to ensure the course attendance requirement.

Cluster 3, which has a moderate level of satisfaction, is the second-most populated cluster. The students in this cluster have low computer literacy and short internet usage duration. Among the cluster members, the rate of 1st graders is higher than other grades, and they mostly use their own internet package. Apart from these, this cluster watched the course records the least, and they used the questionnaires the least.

Being the most populated and having a “good” level of satisfaction, Cluster 4 is the one with the highest rate of females, 1st graders, and those whose internet literacy is at a “good” or “very good” level. It ranks second in terms of computer ownership and computer literacy. At the same time, it is the group with the longest duration staying connected to the internet. Within the same group, synchronous courses are followed the least, while chat tools, videos, word-processing documents, and web pages are used the most.

Nortvig et al. (2018) reported that the educator presence in online environments, student-teacher-content interactions, and intentional connections between online and offline activities were the most prominent factors that affected students’ e-learning, online learning, and blended learning experiences in higher education. Likewise, Goh et al. (2017) identified three learning experiences as predictors of learning outcomes and satisfaction: course design, interaction with the instructor, and interaction with peer students.

Considering these findings, it can be inferred that computer literacy and using the internet for reasonable periods significantly affect satisfaction, and satisfaction increases once the interactive and individualized learning tools in the environment increase. Several studies on distance education also reveal that experience with technological tools contributes to students’ distance education experiences and thus to their satisfaction levels (Kuo et al., 2013). Bossman and Agyei (2022) stated that the factors of technology anxiety, instructor, course quality, technology quality, and ease of use significantly affect distance learners’ e-learning satisfaction and performance levels.

In future studies, it may be recommended to further expand the dataset through different demographic variables, student performance, and behavioral and psychological variables that might apply to satisfaction with e-courses, and to further improve the clustering performance by using diversified parameter settings.

Agyapong, K. (2021). Students’ perspectives on satisfaction with distance education in Ghana: A cluster analysis. African Journal of Teacher Education, 10, 365-391. https://doi.org/10.21083/ajote.v10i1.6287

Antonenko, P.D., Toy, S., & Niederhauser, D.S. (2012). Using cluster analysis for data mining in educational technology research. Education Technology Research Development 60, 383–398. https://doi.org/10.1007/s11423-012-9235-8

Bautista, E., Quintana, N. L., & Sánchez, O. A. (2022). Distance education with university students quarantined by COVID-19. Innoeduca. International Journal of Technology and Educational Innovation, 8(2), 5-13. https://doi.org/10.24310/innoeduca.2022.v8i2.12257

Bollinger, D.U., & Erichsen, E.A. (2013). Student satisfaction with blended and online courses based on personality type. Canadian Journal of Learning & Technology, 39(1), 1-23. https://doi.org/10.21432/T2B88W

Bolliger, D. U., Supanakorn, S. & Boogs, C. (2010). Impact of podcasting on student motivation in the online learning environment. Computers & Education, 55(2), 714- 722. https://doi.org/10.1016/j.compedu.2010.03.004

Bossman, A., & Agyei, S.K. (2022). Technology and instructor dimensions, e-learning satisfaction, and academic performance of distance students in Ghana. Heliyon, 8(4), 1-16. https://doi.org/ 10.1016/j.heliyon.2022.e09200.

Cakır, O., Karademir, T., & Erdoğdu, F. (2018). Psychological variables of estimating distance learners’ motivation. Turkish Online Journal of Distance Education, 19(1), 163-182. https://doi.org/10.17718/tojde.382795

Chen, N. S., Lin, K. M., & Kinshuk (2008). Analyzing users’ satisfaction with e-learning using a negative critical incidents approach. Innovations in Education and Teaching International, 45(2), 115-126. https://doi.org/10.1080/14703290801950286

Chua, C., & Montalbo, J. (2014). Assessing students’ satisfaction on the use of virtual learning environment (VLE): An input to a campus-wide e-learning design and implementation. Information and Knowledge Management, 3(4), 108-116.

Çetintürk, I., & Gençtürk, M. (2020). The classification of the health expenditure indicators of OECD countries through clustering analysis. Süleyman Demirel University Visionary Journal, 11(26), 228-244. https://doi.org/10.21076/vizyoner.650681

Donavant, B. W. (2009). The new, modern practice of adult education: online instruction in a continuing professional education setting. Adult Education Quarterly, 59(3), 227-245. https://doi.org/10.1177/0741713609331546

Goh, C., Leong, C., Kasmin, K., Hii, P., & Tan, O. (2017). Students’ Experiences, Learning Outcomes and Satisfaction in e-Learning. Journal of e-Learning and Knowledge Society, 13(2). https://doi.org/10.20368/1971-8829/144

Gülbahar, Y. (2012). Study of developing scales for assessment of the levels of readiness and satisfaction of participants in e-learning environments. Ankara University, Journal of Faculty of Educational Sciences, 45(2), 119-137. https://doi.org/10.1501/Egifak_0000001256

Harsasi, M., & Sutawijaya, A. (2018). Determinants of student satisfaction in online tutorial: A study of a distance education ınstitution. Turkish Online Journal of Distance Education, 19(1), 89-99. https://doi.org/10.17718/tojde.382732

Ilgaz, H. (2008). The contribution of technology acceptance and the sense of being community to learner satisfaction in distant education [Thesis doctoral, Hacettepe University, Ankara]. Institute of Science, Ankara.

Karabatak, S. Alanoglu, M., & Karabatak, M.(2020, June 1-2). Effects of homework supported distance education on academic satisfaction, academic achievement, and attitude towards distance education [Conference paper]. 8th International Symposium on Digital Forensics and Security (ISDFS), Beirut, Lebanon. https://doi.org/10.1109/ISDFS49300.2020.9116372

Karasar, N. (2006). Scientific Research Method. Nobel Publication.

Kaya, M. (2022). Attitudes of higher education students about the use of distance education environments during the covid-19 outbreak process. Fırat Üniversitesi Sosyal Bilimler Dergisi, 32(3), 995-1011. https://doi.org/10.18069/firatsbed.1103453

Kolburan, A., & Deveci, A. (2015). Development of satisfaction scale for e-course: Reliability and validity study. Journal of Theory & Practice in Education, 11(4), 1272-1287.

Kuo, Y., Walker, A. E., Belland, B. R., & Schroder, K. E. E. (2013). A predictive study of student satisfaction in online education programs. The International Review of Research in Open and Distributed Learning, 14(1), 16–39. https://doi.org/10. 19173/irrodl.v14i1.1338.

Kuo, Y., Walker, A.E., Schroder, K.E.E., & Belland, B.R. (2014). Interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. The Internet and Higher Education, 20, 35-50. https://doi.org/10.1016/j.iheduc.2013.10.001

Lidegran, I., Hultqvist, E., Bertilsson, E., & Börjesson, M.(2021). Insecurity, lack of support, and frustration: a sociological analysis of how three groups of students reflect on their distance education during the pandemic in Sweden. European Journal of Education, 56(4), 550-563.  https://doi.org/10.1111/ejed.12477

Martín, O., Fernández, J. C., Montero, M. A, & González, F. (2015). The main components of satisfaction with e-learning. Technology, Pedagogy and Education, 24(2), 267-277. https://doi.org/10.1080/1475939X.2014.888370

Moore, C., & Moore, J. (2005). The Sloan Consortium Quality Framework and the Five Pillars. The Sloan Consortium.

Nortvig, A.M., Petersen, A.K., & Balle, S.H. (2018). A Literature Review of the Factors Influencing ELearning and Blended Learning in Relation to Learning Outcome, Student Satisfaction and Engagement. The Electronic Journal of e-Learning (EJEL), 16(1), 46-55.

Terzi̇, Ö., Küçüksi̇lle, E., Ergi̇n, G., & Ilker, A. (2011). Estimation of solar radiation using data mining process. International Journal of Technological Sciences, 3(2), 29-37.

Unsihuay, J. E. G., & Blanco, A. Z. (2021). Modelos de minería de datos aplicados al rendimiento académico universitario: Educación virtual durante pandemia COVID-19. Tierra Nuestra, 15(1), 18-28. http://dx.doi.org/10.21704/.rtnv15i1.1812

Vásquez, M.-S., Nuñez, P., & Cuestas, J. (2023). Teachers’ Digital Competences in the context of COVID-19. A quantitative approach. Pixel-Bit. Revista De Medios Y Educación, (67), 155-185. https://doi.org/10.12795/pixelbit.98129

Yildirim, Ö. (2020). Canli ders öncesi çevresel düzenlemeler ve ortam hazirliği. In S. Karaman, E. Kurşun (Ed.), Uzaktan öğretimde canli ders uygulama ilkeleri ve örnekleri (p. 24-38). Atatürk Üniversitesi Yayinlari.

Zhou, G., Yang, L., Liu, W., Shi, J., & Shen, J. (2022).  Exploratory research on satisfaction degree in distance education. Applied Sciences, 12(15), e7889. https://doi.org/10.3390/app12157889