Data analytics play their part in all aspects of work and industry these days, and there’s no question that data analytics are also here to stay in the world of higher education. Through tracking, aggregating, and analyzing student activity captured in learning management systems, universities are hoping to “open the black box of education” using learning analytics technologies (Jones, 2019). Of course, in order to analyze data, there must be data to look at in the first place, and as the number of students participating in online learning has increased exponentially (both in K-12 and higher education) during the COVID-19 Pandemic, the amount of educational data instructors and administrators readily have access to has increased as well. In fact, perhaps unsurprisingly, according to Wong 2021 and a 2020 survey conducted by EDUCAUSE, demand for student success analytics, particularly in relation to online teaching/learning activity, increased by 66 percent during the pandemic.
Yet the use of data in any capacity brings with it a whole host of questions: where is the data coming from and is it ethically sourced? For what purpose is the data being used? Is the data capturing the ‘big picture’ or is it only one piece of the puzzle? Are there biases in the data that need to be reckoned with? Data, in all its forms, is hardly neutral, and thus we must proceed carefully as we look to data to influence decision-making in education. In my mind, data will only ever tell part of the story, but it certainly can be a helpful tool in the educator toolbox when handled with care and context.
Perhaps it is also helpful to clarify what I’m referring to when I say ‘student data.’ For the purposes of this post, I’m referring to certain biographical and socio-economic information related to a student’s background (e.g. whether or not a student is first-generation, financial aid information, etc.), student behavior and participation in courses and campus life, and student performance in particular courses in the form of grades. When it comes to analyzing this data and using that analysis to improve teaching and learning, what’s working in postsecondary education?
Identifying At-Risk Students at the Institutional Level:
An oft-cited use for student data in higher education in this moment is identifying students who might be at-risk of dropping out in order to offer early intervention and support. This is often done at the administrative/academic services level as opposed to the level of individual course instructors.
Since 2017, Gannon University, a private Catholic University in Pennsylvania, has been using a “homegrown application” which collects and aggregates data points from applications across campus, including data related to a student’s academic performance, financial well-being, and engagement in campus activities/community (Wong, 2021). In other words, both qualitative and quantitative data points are observed. A computer model helps determine which data points are most significant, and then summarizes the important data for a student dashboard format. Staff and administrators then check the data dashboard four or five times each semester, including at key grading periods, and if students are flagged as struggling, the advising center or the student development and engagement offices reach out to check on them (Wong, 2021).
Using a similar three-pronged approach, staff and administrators at the University of Kentucky use Tableau software to help interpret student data and identify students who may need support with academics, financial stability, or health and social well-being (Wong, 2021). Based on the data and the populations of students who are flagged as at-risk, staff and faculty have formed outreach protocol, including the ability to increase financial aid support for specific students through grant funding when needed.
Both Gannon and the University of Kentucky have seen retention rates increase over 4% as a result of their meaningful use of student data (Wong, 2021). Of particular note here is the use of data from multiple sources, all of which help tell a fuller story of student success. Grades aren’t the only indicator of a student’s level of risk; financial stability and social well-being are treated as equally important information sources.
Improving Classroom Instruction:
Student data can also be helpful to individual instructors as they look to monitor the effectiveness of their instruction and better meet the needs of students who may be struggling. In an earlier post titled Using Canvas Analytics to Support Student Success, I specifically looked at the student data analytics capabilities native to the Canvas LMS platform, but there are certainly plenty of comparable features in other LMS platforms which would assist instructors in the efficient analysis of student data.
When it comes to student engagement and indicators of successful course completion, information gathered in the first weeks of the course can prove invaluable. Rather than being used solely for instructor reflection or summative ‘takeaway’ information about the effectiveness of the course design, course analytics may be used as early predictors of student success, and the information gleaned may be used to initiate interventions from instructors or academic support staff (Wagner, 2020). For example, if a student in an online course is having internet access issues, the instructor can likely see this reflected early-on in the student’s LMS analytics data (not logging in to the course, not accessing important posted materials, etc.). The instructor would have reason to reach out and make sure the student has what they need in order to engage with the course content. If unstable internet access is the issue, the instructor may then flex due dates, provide extra downloadable materials, or continually modify assignments as needed throughout the quarter in order to better support the student.
In addition to student performance, LMS analytics tools may be used by the instructor to think about the efficacy of their course design, especially in online learning environments. Course analytics tools can help instructors see which resources are being viewed/downloaded, which discussion boards are most active (or inactive), what components of the course are most frequented, etc. Technology can also help instructors save valuable time. For example, course analytics tools can quickly cull through quiz results to identify which concepts remain hazy in students’ minds, helping instructors to efficiently discern which of their lesson plans is most effective, and which concepts need more attention and/or a different teaching approach (O’Bryan & Shah, 2021).
Gathering Student Feedback:
Finally, student surveys have proven to be another effective way to access student data and meaningfully use that data in support of student success. Student surveys elevate the use of student voice within the data, and they are much easier to use where data privacy management is concerned. “Since learning analytics often rely on aggregating significant amounts of sensitive and personal student data from a complex network of information flows, it raises an important question as to whether or not students have a right to limit data analysis practices and express their privacy preferences as means to controlling their personal data and information” (Jones, 2019). Within a survey context, students have choice around when and how they participate in providing data, and they often have greater insight into how the data will be used afterward. This is not always the case when it comes to data used in and through learning management systems, and many scholars and researchers feel that the ethics behind data collection/use/privacy in learning analytics have yet to be properly addressed (Viberg et al., 2018; Jones, 2019).
To that end, University of Connecticut offers a great example of elevating student “voice and choice” in the data collection process. UC has developed a software suite in-house called Nexus that is designed to involve the entire campus community in improving student retention and success, especially the students themselves. Students can choose to log in to a UC campus application at any time to create study groups with classmates, schedule advising and tutoring appointments, and connect with mentors and other resources as needed. The university also occasionally asks students to fill out a short online survey when they log in to the app; the 60-second survey asks critical questions such as how they are doing and whether they are contemplating dropping out for any reason (Wong, 2021). Thus, in this approach, students are able to volunteer data relevant to their learning needs and connect to available resources when it feels appropriate to them; they are not passive in the data collection process.
Course completion surveys are also commonly used by higher education institutions, and these surveys provide important student-sourced feedback about the effectiveness of individual instructors and courses. However, since the feedback is summative/reflective in nature, its ability to have any impact on an individual student’s learning at a point of accute struggle during a term of study is limited, if not completely obsolete. Additionally, these course surveys are usually more focused on growth and improvement for instructors and course design, and the data collected gives little additional insight on individual student performance.
Summary:
To be sure, there are likely many other examples of places and spaces where data analytics are working well in higher education, and I’ve only touched on a few key areas in this post, but generally speaking, data or learning analytics seem to be proving helpful at the institutional level to improve retention rates, at the instructor level as a way of efficiently identifying student needs in real time within a course, and at the student level when student feedback data, often via surveys, is used meaningfully in support of student success. Learning analytics have not been, and will never be, some kind of computer-aided substitute for sound pedagogical assessment in a classroom. Furthermore, as mentioned above, educators are wise to bear in mind that any single data set is only part of a larger story. Learning analytics seem to be at their best to the extent that they are truly used in support of individual student growth and flourishing in all aspects of education. As Viberg et al. (2018) would posit, the more the use of learning analytics in higher education shifts focus away from a prediction emphasis and towards a dynamic understanding of students’ real-time learning experiences, the more we’ll be able to see authentic and substantive improvements in student outcomes.
References:
Jones, K.M.L., (2019). Learning analytics and higher education: a proposed model for establishing informed consent mechanisms to promote student privacy and autonomy. International Journal of Educational Technology in Higher Education, 16(24). https://doi.org/10.1186/s41239-019-0155-0
O’Bryan, C. & Shah, B. (2021, September 8). Higher education has a data problem. Inside Higher Ed. https://www.insidehighered.com/views/2021/09/08/using-data-holistic-way-support-student-success-opinion
Wagner, A. (2020, June 6). LMS data and the relationship between student engagement and student success outcomes. Airweb.org. https://www.airweb.org/article/2020/06/17/lms-data-and-the-relationship-between-student-engagement-and-student-success-outcomes
Wong, W. (2021, October 18). Higher education turns to data analytics to bolster student success. EdTech. https://www.edtechmagazine.com/higher/article/2021/10/higher-education-turns-data-analytics-bolster-student-success
Viberg, O., Hatakka, M., Balter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior 89, 98-110. https://doi.org/10.1016/j.chb.2018.07.027
Your explanation of the power of online analytics makes it clear that these tools can be useful in addressing the needs of at risk students, or allowing the educator to gain insights into whether students might have difficulties connecting to online resources. I am not sure I see how I might use these tools to determine if the content I am offering, and the instructional strategies I am using, are effective. Has this been the focus of much research? You might also find Justin Reich’s book, Failure to Disrupt to be valuable.
Thanks for raising up this question, data analytics sure is one of the important knowledge we need to know. With technological advancement, our lives are inseparable from data. Many different data are generated in our lives, work, and studies. However, how to make good use of these data to help us improve our lives, the analysis techniques involved are very important. In my working environment, I often use different data to help customers analyze the future development of the business and matters needing attention. However, the use of data analysis in education is very easy to be forgotten, but in fact, if the effective use of data can definitely improve the quality of students’ learning, it can also help analyze their future needs. It can also be expected that some things that have not happened now will be prepared as soon as possible.
Especially what you shared about collecting student feedback through surveys. This is an interesting part. I believe that in higher education, this method has a very old history, but how can it be provided with current technology such as artificial intelligence? The accuracy and practicality of the information will be a direction worth knowing
I appreciate how you were careful to callout the need to balance use of the data collected with care and that data should be used for the support of students. One of the lines I’ve been straddling and haven’t necessarily figured out a “right” way to handle, is this idea of surveillance under the guise of care – http://hackeducation.com/2020/04/30/surveillance that by monitoring and seeing what you’re up to, this is in support and care of the student. Such as seeing what time a student logs in the most. Sometimes I find myself asking why do I need that data, what should I do with that data? Is it even my place to say something? I wonder where and if a line can be drawn?