Using Canvas Analytics to Support Student Success

Though online teaching/learning are hardly new concepts in education, the pandemic has necessitated a massive shift to online learning such that educators worldwide–at all levels–have had to engage with online learning in new, immersive ways.  Online learning can take many forms (synchronous, asynchronous, hybrid, hyflex, etc.), but regardless of the form, educators with access to an LMS have been forced to lean into these platforms and leverage the tools within in significant ways, continually navigating (perhaps for the first time) how to best support students in achieving their learning goals using technology.

Without consistent opportunities for face-to-face communication and informal indicators of student engagement that are typically available in a classroom (e.g. body language, participation in live discussions, question asking) a common challenge faced by educators in online learning environments–especially asynchronous ones–is how to maintain and account for student engagement and persistence in the course.  Studies using Educational Data Mining (EDM) have already demonstrated that student behavior in an online course has a direct correlation to their successful completion of the course (Cerezo et al., 2016). Time and again, these studies have supported the assertion that students who are more frequently engaged with the content and discussions in an online course are more likely to achieve their learning goals and successfully complete the course (Morris et al., 2005).  This relationship is, however, tricky to measure, because time spent online is not necessarily representative of the quality of the online engagement.  Furthermore, different students develop different patterns of interaction within an LMS which can still lead to a successful outcome (Cerezo et al., 2016). Consequently, even as instructors look for insights into student engagement from their LMS, they must avoid putting too much emphasis on the available data, or even a ‘one style fits all’ approach to interpreting it.  Instead, LMS analytics should be considered as one indicator of student performance that contributes to the bigger picture of student learning and achievement.  Taken in context, the data that can be quickly gleaned from an LMS can be immensely helpful in identifying struggling or ‘at-risk’ students and/or those who could benefit from differentiated instruction, as well as possible areas of weakness within the course design that need addressing.

Enter LMS analytics tools and the information available within.  For the purposes of this post, I’ll specifically be looking at the suite of analytics tools provided by the Canvas LMS, including Course Analytics, Course Statistics, and ‘New Analytics.’

Sample Screenshot of Canvas New Analytics, https://sites.udel.edu/canvas/2019/11/new-canvas-analytics-coming-to-canvas-in-winter-term/
  • Course Analytics are intended to help instructors evaluate individual components of a course as well as student performance in the course.  Course analytics are meant to help identify at-risk students (i.e. those who aren’t interacting with the course material), and determine how the system and individual course components are being used.  The four main components of course analytics are: 
    • Student activity, including logins, page views, and resource usage
    • Submissions, i.e. assignments and discussion board posts
    • Grades, for individual assignments as well as cumulative
    • Student analytics, which is a consolidated page view of the student’s participation, assignments, and overall grade (Canvas Community(a), 2020).  With permission, students may also view their own analytics page containing this information.
  • Course Statistics are essentially a subset of the larger course analytics information pool.  Course statistics offer specific percentages/quantitative data for assignments, discussions, and quizzes.  Statistics are best used to offer quick, at-a-glance feedback regarding which course components are engaging students and what might be improved in the future (Canvas Community(b), 2020).
  • New Analytics is essentially meant to be “Course analytics 2.0” and is currently in its initial rollout stage.  Though the overall goal of the analytics tool(s) remains the same, New Analytics offers different kinds of data displays and the opportunity to easily compare individual student statistics with the class aggregate.  The data informing these analytics is refreshed every 24 hours, and instructors may also look at individual student and whole class trends on a week-to-week basis.  In short, it’s my impression that ‘New Analytics’ will do a more effective job of placing student engagement data in context.  Another feature of New Analytics is that instructors may send a message directly to an individual student or the whole class based on a specific course grade or participation criteria (Canvas Community(c), 2020). 
Sample Screenshot of Canvas New Analytics, https://sites.udel.edu/canvas/2019/11/new-canvas-analytics-coming-to-canvas-in-winter-term/

Of course, analytics and statistics are only one tool in the toolbelt when it comes to gauging student achievement, and viewing course statistics need not be the exclusive purview of the instructor.  As mentioned above, with instructor permission, students may view their own course statistics and analytics in order to track their own engagement.  Beyond viewing grades and assignment submissions, this type of feature can be particularly helpful for student reflection on course participation, or perhaps as an integrated part of an improvement plan for a student who is struggling.

Timing should also be a consideration when using an LMS tool like Canvas’ Course Analytics.  When it comes to student engagement and indicators of successful course completion, information gathered in the first weeks of the course can prove invaluable.  Rather than being used solely for instructor reflection or summative ‘takeaway’ information about the effectiveness of the course design, course analytics may be used as early predictors of student success, and the information gleaned may be used to initiate interventions from instructors or academic support staff (Wagner, 2020). Thus, instructors who use Canvas will likely find that their Canvas Analytics tools might actually prove most helpful within the first week or two of the course (University of Denver Office of Teaching & Learning, 2019).  For example, if a student in an online course is having internet access issues, the instructor can likely see this reflected early-on in the student’s LMS analytics data. The instructor would have reason to reach out and make sure the student has what they need in order to engage with the course content.  If unstable internet access is the issue, the instructor may then flex due dates, provide extra downloadable materials, or continually modify assignments as needed throughout the quarter in order to better support the student.

Finally, as mentioned above, in addition to student performance, LMS analytics tools may be used by the instructor to think about the efficacy of their course design.  Canvas’ course analytics tools help instructors see which resources are being viewed/downloaded, which discussion boards are most active (or inactive), what components of the course are most frequented, etc.  Once an online course has been constructed, it can be tempting for instructors to “plug and play” and assume that the course will retain its same effectiveness in every semester it’s used moving forward. Course analytics can help instructors identify redundancies and course elements that are no longer needed/relevant due to lack of student interest.  They can also help instructors think critically about what seems to be working well in their course (i.e. what are students using, where are they spending the most time in the course) why that might be, and how to leverage that for adding other course components or tweaks for the future.

In summary, the information available via an LMS analytics tool should always be considered in concert with all other factors impacting student behavior in online learning, including varying patterns or ‘styles’ in students’ online behaviors and external factors like personal or societal crises that may have impacted the move to online learning in the first place.  Student engagement (as measured by LMS analytics tools) can be helpful tools used for identifying struggling students, providing data for student self-reflection, and providing insight into the effectiveness of the instructors’ course design.  To the extent that analytics tools aren’t considered the “end all be all” when it comes to measuring student success, tools like Canvas Analytics are a worthwhile consideration for instructors teaching online who are invested in student success as well as their own professional development.

References:

Canvas Community(a). (2020). What are Analytics? Canvas. https://community.canvaslms.com/t5/Canvas-Basics-Guide/What-are-Analytics/ta-p/88

Canvas Community(b). (2020). What is New Analytics? Canvas. https://community.canvaslms.com/t5/Canvas-Basics-Guide/What-is-New-Analytics/ta-p/73

Canvas Community(c). (2020). How do I view Course Statistics? Canvas. https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-view-course-statistics/ta-p/1120

Cerezo, R., Sanchez-Santillan, M., Paule-Ruiz, M., & Nunez, J. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education 96, 42-54. https://www.sciencedirect.com/science/article/pii/S0360131516300264

Morris, L.V., Finnegan, C., & Wu, S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education 8, 221-231. https://www.sciencedirect.com/science/article/pii/S1096751605000412 

Wagner, A. (2020, June 6). LMS data and the relationship between student engagement and student success outcomes. Airweb.org. https://www.airweb.org/article/2020/06/17/lms-data-and-the-relationship-between-student-engagement-and-student-success-outcomes 

Student Flourishing in the Virtual Classroom

Image Source, Medium.com

There is no doubt that the COVID-19 pandemic has greatly accelerated the rate at which schools and universities of all shapes and sizes have had to move to online teaching and learning modalities, even if only as a short-term conduit for allowing formal education to continue in these unprecedented times.  There is also no doubt that this emergency shift to online teaching has left many concerned about overall student well-being including screen fatigue, issues of access and equity, teacher readiness, social-emotional support in a digital environment, and the overall efficacy of the educational endeavor for students of all ages in digital mediums.  Is there a light at the end of the tunnel?  Or might there already be some twinkle lights strung up along the tunnel walls guiding the way? 

In this post I’d like to explore some of the evidence that already exists in support of student flourishing—particularly at the postsecondary level—in hybrid or fully online programs, as well as what best practices can be used to support student well-being in all online teaching/learning endeavors, during COVID-19 and beyond.  Thankfully, the pandemic didn’t bring about the dawn of online pedagogy in higher education, and postsecondary educators have places to turn in order to think critically (and perhaps hopefully) about student success and well-being, be it academic or personal, in the digital classroom.

Evidence of Flourishing:

Few would argue that an in-person classroom experience can be identically replicated online.  In fact, those who attempt to do so have probably done so with disappointing results.  But perhaps educators shouldn’t necessarily be trying to replicate a physical classroom experience in an online environment.  Rather, they should think of the virtual classroom as a new endeavor; it is a new context with new possibilities to explore, and online pedagogy may bring new teaching/learning benefits to the table that a physical classroom lacks. 

Indeed, there’s evidence to suggest that a hybrid of in-person and online teaching may be the very best approach to postsecondary learning—with or without a pandemic—as it capitalizes on the “best of both worlds.”  In an extensive, multi-year case study conducted at the University of Central Florida in 2004, research showed that student success in blended programs (success being defined as achieving a C- grade or higher) actually exceeded the success rates of students in either fully online or fully face-to-face programs (Dziuban et al, 2004).  Furthermore, in a meta-analysis of studies on online and hybrid learning conducted by the U.S. Department of Education in 2010, it was reported that students in online and hybrid learning programs had more gain in their learning when compared to face-to-face modalities, and students in hybrid learning courses had the largest gains in their learning among their peers in all delivery formats (Means et al., 2010).  In yet another study (Chen & Chiou, 2014) measuring the learning outcomes, satisfaction, sense of community and learning styles of 140 second-year university students in Taiwan, results showed that students in a hybrid course had significantly higher scores and overall course satisfaction than did students participating in face-to-face courses. The results also indicated that students in hybrid learning classrooms actually felt a stronger sense of community than did students in a traditional classroom setting (Chen & Chiou, 2014).

While one must make many allowances for the various emergency situations brought on by the pandemic (and that there is a distinction between emergency remote instruction and true online teaching/learning), there is plenty of evidence to suggest that well-implemented online teaching/learning can truly enhance student learning beyond what might otherwise be accomplished in a fully face-to-face environment.

Some Best Practices in Online Instruction:

Technology-mediated education is making it possible for students to participate in programs, access content, and connect in ways they were previously unable to.  Rather than viewing the Internet as a necessary evil for distance learning that ultimately begets isolated student learning experiences, digital education should, first and foremost, be connective and communal.  This means a professor accustomed to lecture-based learning in a physical classroom will need to consider a new approach in order to prioritize student voice in the learning process.  In an online context, this means there should be dynamic opportunities for students to engage in debate, reflection, collaboration, and peer review (Weigel, 2002).

If educators are going to seriously account for the rich background experiences, varied motivations, and personal agency of their postsecondary learners, they must also take into account the larger “lifewide” learning that takes place within the lives of their students (Peters & Romero, 2019). Student learning at any age is both formal and informal, and what takes place in a formal classroom environment—digital or otherwise—is influenced by informal learning and daily living that takes place outside of it.  If deep learning takes place, a student’s world and daily life should be altered by the creation of new schemas and the learning that has taken place in a formal classroom environment.  In a multicase and multisite study conducted by Mitchell Peters and Marc Romero in 2019, 13 different fully-online graduate programs in Spain, the US, and the UK were examined in order to analyze learning processes across a continuum of contexts (i.e., to understand to what extent learning was used by the student outside of the formal classroom environment).  In this study, certain common pedagogical strategies arose across programs in support of successful student learning and engagement including:

  1. Developing core skills in information literacy and knowledge management,
  2. Community-building through discussion and debate forums,
  3. Making connections between academic study and professional practice,
  4. Connecting micro-scale tasks (like weekly posts) with macro-scale tasks (like a final project), and
  5. Applying professional interests and experiences into course assignments and interest-driven research.

(Peters & Romero, 2019).

In many regards, each of these pedagogical strategies is ultimately teaching students to “learn how to learn” so that the skills they cultivate in the classroom can be applied over and over again elsewhere. This means that, where digital learning is concerned, the most important learning activities aren’t actually taking place in a large, synchronous Zoom meeting or broadcasted lecture series.

On a practical level, educators can also give attention to some of these simple “tricks of the trade” that have been proven to enhance student learning experiences in a virtual classroom:

  1. Communicate often with students to promote a feeling of connectedness
  2. Create ample space for student voice
  3. Take care that a course set-up in a learning management system is intuitively laid out, action oriented, and adaptable to student needs
  4. Give timely feedback and highlight student strengths
  5. Create opportunities for synchronous activities when possible
  6. Be explicit about expected course outcomes

(Vlachopoulos & Makri, 2019)

At the end of the day, learning and schooling no longer have the same direct relationship they had for most of the 20th century; devices and digital libraries allow anyone to have access to information at any time (Wilen, 2009). Schools, teachers, and printed books no longer hold the “keys to the kingdom” as sources of information.  Online education, then, will not function effectively as a large-scale effort to teach students information through a standardized curriculum.  Rather, education must be a highly relevant venture that enables individual students to do something with the virtually endless information and resources they have access to (Wilen, 2009).

Student Agency & Connection Lead to Student Wellbeing:

When considering how to best support student wellbeing in an online learning environment (at every level), it’s important to remember that the student is not a passive entity.  Indeed, the extent to which students are able to exercise agency in their learning can have a significant impact on their academic success, their attitude towards the learning experience, and their social-emotional wellbeing.  In this case, agency can be interpreted as a student’s ability to exercise choice and be meaningfully present and interactive in the online learning environment.

One of the significant benefits of learning management systems and digital classrooms is the existence of a platform through which resources and learning materials can be shared and posted for any length of time.  Thus, students have the ability to review online course materials at their own pace and engage at a rate that makes sense for their individual needs (Park, 2010).  Allowing students the time and space to persist in completing online learning activities can have significant impact on a students’ success in an academic course (Park, 2019).

Additionally, game-based learning activities, opportunities for collaboration in group projects, participation in threaded discussions, and dedicated spaces for students to freely express their views all assist students in taking ownership of their learning and pursuing their learning interests as those interests materialize in—and overlap with—the course content (Vlachopoulos & Makri, 2019).  These are the activities that directly impact student engagement in a course, as well as the likelihood that a student will have a positive attitude towards the learning experience.

For many traditionally-aged students navigating undergraduate studies during the pandemic, the decreased ability to connect socially with peers, faculty, and support staff has had a direct, negative impact on their academic motivation and overall sense of wellbeing (Burke, 2020).  Thus, creating time and space in the digital learning environment for social interaction, open communication, and for students to gain a sense of identity within the virtual classroom is perhaps more important than ever. 

Finally, it’s very much worth mentioning that the extent to which all spheres of life have been impacted by COVID-19—not just the classroom—is unprecedented.  Helping students think of remote learning as an opportunity for growth, one that will have challenges and limitations as well as potential and new kinds of goals that can be achieved, can help them maintain a sense of purpose and direction amidst the chaos (Burke, 2020).  Growth mindset has already been proven to positively impact student learning at all levels—what better time to remind students (and educators) of the opportunities for growth in the present.

References:

Burke, L. (2020, October 27). Moving into the long term. Inside Higher Ed. https://www.insidehighered.com/digital-learning/article/2020/10/27/long-term-online-learning-pandemic-may-impact-students-well

Chen, B. & Chiou, H. (2014). Learning style, sense of community, and learning effectiveness in hybrid learning environment. Interactive Learning Environments, 22(4), 485-496. https://www.tandfonline.com/doi/abs/10.1080/10494820.2012.680971

Dziuban, C., Hartman, J., Moskal, P., Sorg, S., & Truman, B. (2004). Three ALN modalities: an institutional perspective. In J. R. Bourne, & J. C. Moore (Eds.), Elements of quality online education: Into the mainstream (127–148). Sloan Consortium.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Department of Education, Office of Planning, Evaluation and Policy Development. https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

Park, E., Martin, F., & Lambert, R. (2019). Examining predictive factors for student success in a hybrid learning course. The Quarterly Review of Distance Education 20(2), 11-27.

Peters, M. & Romero, M. (2019) Lifelong learning ecologies in online higher education: Students’ engagement in the continuum between formal and informal learning. British Journal of Educational Technology, 50(4), 1729.

Vlachopoulos, D., & Makri, A. (2019). Online communication and interaction in distance higher education: A framework study of good practice. International Review of Education, 65,605–632. https://doi.org/10.1007/s11159-019-09792-3

Weigel, Van B. (2002) Deep learning for a digital age.  San Francisco, CA: Jossey-Bass.

Wilen, T. (2009). .Edu: Technology and learning environments in higher education. Peter Lang Publishing.

css.php