Diigo as a tool for collaborative learning and research in higher education

There is significant opportunity within higher education environments–indeed, all education environments–to lean into a constructivist educational philosophy and approach knowledge as something co-created by both instructors and students.  Furthermore, as higher education courses and programs are increasingly offered in hybrid and fully online modalities, finding authentic ways for students to increase their social presence and overall engagement-level in coursework is essential (Baran, 2013). Digital tools can greatly assist in the act of socially constructing knowledge by helping to eliminate learning boundaries and extend opportunities for both formal and informal learning in a myriad of ways (Baran, 2013).  

Within higher education, one of the more important realms of knowledge construction between students and instructors, especially at the graduate level, takes place in the academic research process and in the conversations that, quite literally, take place in the margins within that research process (Farber, 2019).  As a graduate student who currently does not use any specific annotation or research collaboration tools for research outside of Microsoft Teams or Google Suite, I am curious to explore the ways in which the social bookmarking tool Diigo (which allows learners to collect, annotate, organize, and share online resources) supports efficient, collaborative research among instructors, students, and their peers in higher education.  Furthermore, I am interested in anecdotally comparing my exploration of this tool with the functionalities of more recently-released (and decidedly more expansive) digital collaboration tools like Teams and Google+.  Does Diigo hold its own in the digital collaboration tool market in 2021?

According to the product website (Diigo Inc., 2021), Diigo supports collaborative learning endeavors in four key ways.  It allows users to:

  • Collect: save and tag online resources to public or private curated libraries for easy access
  • Annotate: annotate web pages, PDFs, and other digital content directly while browsing online
  • Organize: organize links, references and personal input to create a structured research base
  • Share: share research with friends, classmates, colleagues or associates

Originally released in 2011, Diigo (Digest of Internet Information, Groups and Other stuff) quickly found a dedicated user-base and distinguished itself from other types of bookmarking applications (most notably its 2003 bookmarking predecessor, Delicious) due to its user-friendly interface, emphasis on social engagement, and extensions specific to use in education, combined with more traditional bookmarking functionalities (Ruffini, 2011).  The table below shows a helpful comparison between Diigo, its initial competitor and predecessor Delicious (now defunct), and the typical bookmarking capabilities of a web browser.

Table 1. Social Bookmarking Comparison Chart (updated by Ruffini, 2011)

FeatureDiigoDeliciousBrowser
Organize bookmarks automatically with tagsxxx
Popular bookmarksXXX
Anytime, anywhere access to bookmarksXX 
Share bookmarks with othersXX 
Powerful, customizable search toolsXX 
Groups of people with similar interestsXX 
Post automatically to blogXX 
Tools and browser extensions for bookmarkingXX 
Lists of grouped bookmarksXX 
Free iPhone and Android appsXThird party 
iPad Safari browser bookmarkletX  
Add and share sticky notesX  
Capture, mark up, share images and textX  
Collect web pictures into albumsX  
Sync bookmarksX  
Tools for educatorsX  
Original table by: Schmidt, Jason. (2010, July 30). Diigo and Delicious. Interactive Inquiry. https://iisquared.wordpress.com/2010/07/30/diigo-and-delicious/

As the table indicates, Diigo offers much more functionality in several categories, but most notably (pun intended) in the realm of annotation.  When installed as a browser extension, Diigo can be integrated fairly seamlessly into existing research habits, and perhaps most importantly, most Diigo tools can be accessed for free.

Since Delicious left the scene, new social learning/annotation tools have surfaced such as Hypothesis.is and Mendeley, both of which deliver many of the same features as Diigo, and both of which are largely open access.  Though a detailed direct comparison of these tools is beyond the scope of this post, a brief exploration seems to suggest that Hypothesis might be most appealing for educators who would like to incorporate the application into their Learning Management System (LMS).  Hypothesis can integrate nicely into all of the major LMS platforms and it offers many resources and training videos for educators so that they can truly maximize their use of the tool within their planned learning activities (Guhlin, 2020). Mendeley, a tool primarily intended for use in higher education, has a handy “cite as you write” plugin that prioritizes the streamlining of the reference process, automatically capturing author, title, and publisher information as needed (Guhlin, 2020).

Though it’s now been a decade since its release, Diigo seems to maintain its relevance and dedicated user base for several reasons:

  1. It is a bit simpler and more user-friendly than its competitors such that it is more easily adapted for use in a variety of learning environments and contexts, including both K-12 and higher education (Guhlin, 2020).
  2. Diigo seems to be the tool with the strongest integrated support for educators independent of an LMS.  Since its early stages, special accounts are available for educators that empower registered teachers with a variety of extra tools and features, leveraging the tool for use and collaboration with an entire class if desired (Education Technology, 2015).
  3. Individual features like Diigo Outliner, which lets you create and share digital outlines within a document, add sustaining value to the tool; these features are more nuanced than the general commenting features or Track Changes available in so many other types of collaboration tools (Guhlin, 2020).
  4. Because of its longevity, Diigo has had time to collect a large, lasting user base and dynamic Interest Groups (i.e. K-12 teachers, higher education instructors, researchers, etc.) which offer grassroots professional development tips and organic user insights accessible to the whole community (Ruffini, 2011).  
  5. Diigo’s longevity is also a testament to the creators’ ability to update the tool to best fit user needs over time, and there continue to be product and app updates on a regular basis. Diigo has evolved over the years, and today it is used more frequently for its collaboration/annotation capabilities than the social bookmarking services it was originally focused on (Guhlin, 2020).

Having recently worked on a collaborative research publication using Microsoft Teams, and as a frequent user of collaborative Google products for both academic and personal endeavors, I was curious if this exploration would support Diigo as a stand-alone tool to be considered for use in collaborative research endeavors, or whether its offerings were more or less synonymous with tools embedded within these larger platforms.  Anecdotally, I think the answer is yes, Diigo does stand alone, at least for specific use cases.

Both Teams and Google have many strengths when it comes to video conferencing, and cloud-based word processing and document sharing, but I do not find that these tools go quite so far to aid in the initial research phase. According to Educational Technology and Mobile Learning (2015), with Diigo, student and faculty researchers may:

  • Search for online content relevant to their project, bookmark the websites and then add them to a shared ‘class’ or group
  • Organize bookmarks by tags and date to organize content around a particular topic and to make it easy to search for it later
  • Highlight specific segments of a webpage or add sticky notes to annotate them for others to read 
  • Take screenshots of useful online content and annotate them for use as well

In these cases, Diigo essentially cuts out a step (or multiple steps) for an instructor or student trying to share research. In the initial research phase, a researcher using Diigo would not need to save and download a PDF or copy the link for a website of interest, only to reupload it or paste it later to a general repository, not having the same ability to annotate the resource or organize it as efficiently as Diigo would allow.  However, I do think Diigo finds its strongest value in working for a specific research purpose with a specific group of people.  Its value is inherently collaborative and is best used when trying to co-construct knowledge.  Consequently, it isn’t a tool I’ll be using regularly in my daily academic activities for just myself, but it is a tool I’ll be reaching for when it comes time to spearhead my next research project.

Resources:

Baran, E. (2013). Connect, participate and learn: Transforming pedagogies in higher education. Bulletin of the IEEE Technical Committee on Learning Technology, 15(1), p. 9-12. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.681.1177&rep=rep1&type=pdf

Diigo Inc. (2021). Diigo. https://www.diigo.com/

Education Technology and Mobile Learning. (2015, January 14). 7 ways students use Diigo to do research and collaborative project work. https://www.educatorstechnology.com/2015/01/7-ways-students-use-diigo-to-do-research.html

Farber, M. (2019, July 22). Social Annotation in the Digital Age. Edutopia. https://www.edutopia.org/article/social-annotation-digital-age

Guhlin, M. (2020, April 13). Note-taking and outlining: Five digital helpers. TechNotes. https://blog.tcea.org/note-taking-and-outlining/

Ruffini, M. (2011, September 27). Classroom collaboration using social bookmarking service Diigo. Educause Review. https://er.educause.edu/articles/2011/9/classroom-collaboration-using-social-bookmarking-service-diigo

Schmidt, Jason. (2010, July 30). Diigo and Delicious. Interactive Inquiry. https://iisquared.wordpress.com/2010/07/30/diigo-and-delicious/

Leveraging Digital Tools for Instruction Outside the Classroom: Community Engagement Project, EDTC 6102

It’s been a wonderful opportunity to invest time and energy into a project which ultimately helps me do my job better.  Though I am not currently a classroom teacher and do not have typical instructional responsibilities in my day to day work, I do constantly convey important information about the WA State teacher certification process to prospective educators, a process which can often feel overwhelming and convoluted to many would-be career changers. I feel confident that I’ve put together a blueprint for a meaningful, engaging informational session which will help students navigate the first steps of the certification process with confidence and clarity, ultimately helping them discern for themselves if becoming a teacher is the right step for them professionally at this time, and if so, how to make that a reality. Not only that, I’ve been able to think creatively about how to leverage digital tools to make the session interactive, student-centered, and practically useful to those who attend.

Lesson Plan for 60 Minute Information Session

This lesson plan is intended for a Group of 10-20 prospective graduate students using a virtual teleconference platform like Zoom, Teams, Google Meet, etc.

Note: some hyperlinks may not be accessible to all due to permissions settings

Introduction 
10 min.
Students will be notified of my intention to record the session and their associated rights (turning off video, etc.).

Introduce Self and Learning Objectives of Info Session:
Objective 1: learn how someone becomes a teacher in WA State and determine personal readiness to begin the process.
Objective 2: learn what program options are available at SPU and which program is best-fit for personal context
Objective 3: learn what steps to take next with an application to a teacher certification program

10 Signs That You Should Become a Teacher, opening reflection video
Interactive Presentation
20 min.
Google Slide Deck
Students will be provided with access to the Google Slide Deck in advance of the information session so that they may conduct research ahead of time or follow along independently, clicking on hyperlinks, etc. in their own browser window. I will also use it to structure the presentation of information in the session.  For those who do not choose to follow along independently, hyperlinks for interactive elements will be provided directly in the session chat.

Within the slide deck, students will be introduced to SPU’s various graduate teacher certification program options.Students will be given the opportunity to stop and reflect on what feels like their best-fit program halfway through, before new information about the application process is introduced.  They will also be given the opportunity to ask clarifying questions at this point via Jamboard.

Students will then be given detailed information about application requirements and due dates, including specific information about the endorsement verification process.  This is also a time where I intend to help students understand and navigate the various information systems which they will have to engage with throughout the certification and application process (online application, standardized tests, etc), and to what extent they’ll be expected to provide personal data in digital spaces.
Formative Assessment 10 min.Prospective students will then have the opportunity to review and test their understanding of the material via a brief, 15-question Kahoot Quiz.  This will also serve as a formative assessment tool for me, the instructor, and may help point out areas that need further clarification. There will be time to pause and address these areas while going through the Kahoot Quiz.
Performance Task 
10 min.
Students will then have the opportunity to curate a personal To Do List outlining their next steps towards an application (and thus, towards becoming a teacher).  This is a performance task that helps students indicate their understanding of the material covered in the information session, but it is also meant to be a practical, relevant takeaway for each attendee.

Students will be able to make a copy of this Google Doc Template which contains a scaffolded “word bank” of application requirements.  Students will be able to copy/paste from the word bank in order to create their own, personalized To Do List, paying special attention to their specific program needs, endorsement requirements, and chronological order (i.e. which items need attention first). I will also provide the Google Doc Template in an alternate format (i.e. Word document) for any students that need it after the session. 
Self-Assessment & Reflection
10+ min.
Shortly after providing students with the Google Doc Link, I will provide a link to the final Sticky Note Q & A w/ Jamboard so that students may ask any needed questions while they construct without interrupting the thought processes of others in the session.  They may also choose to drop questions privately to me in the chat depending on the immediacy of the need and/or group appeal of their question. 

After the allotted 10 minutes passes for list construction, I’ll also offer a final few minutes for students to review their lists and ask final questions that need resolving on the Jamboard, OR live in the video conference, time permitting. The session will be recorded and the recording will be provided to students after the information session via email. This allows students to go back and review as needed.

Throughout this session, students will have the opportunity to demonstrate their understanding through application and self-knowledge.

  • Application: as students create their own, personally curated “To Do List” they will be able to apply their understanding of the session material by creating a useful tool that will guide them moving forward.  This includes discerning which information is most relevant to their particular context.  Students will be able to effectively navigate the various information systems which they will have to engage with throughout the certification and application process, especially in regards to the information they provide during the application process.  This also brings to mind the fact that students will literally apply to a program as part of their next steps towards becoming a teacher.
  • Self-Knowledge: students will be invited to reflect on their motivations to become a teacher, whether now is the time to take steps towards becoming a teacher, as well as what kind of program would be best suited for their needs. There will also be ample time for students to grapple with what they do not know, or what is confusing for them in this process.  The decisions they make moving forward will be rooted in the self-knowledge acquired from this session.

I do believe that the most helpful reflections on this “lesson plan” will come after I’ve had the opportunity to put it into action for the first time in a professional setting, likely in Fall of 2021. That said, One area for potential improvement that I can already identify is the curated “To Do List” platform.  Though I found some potential tools that I was interested in which might be a bit sleeker/less cumbersome than using a Google Doc, the ones I came across were not open access or would require a full account set-up in order to use them. This wouldn’t translate well to the timing and context of this particular lesson, nor the student audience (i.e. prospective students attending an information session, not students enrolled in a class).  Thus, for now, I have the Google Doc format as a bit of a place-holder.  I’m quite open to tweaking this section of my lesson in favor of a better tool later on.

In summary, this particular project was a wonderful exercise in thinking about learning and instruction on a macro level and the many ways they take place outside of a formal classroom environment. Digital tools may be leveraged in a myriad of ways to help us do our jobs better, and this was an opportunity for me to think creatively about how to bring that home in my own context. I look forward to using this session format in the next recruiting cycle!

Exemplars of Computational Thinking in Higher Education Classrooms

Though the concepts and theory behind computational thinking (CT) have been around for decades in the realms of computer science and engineering, it is widely acknowledged that Jeanette Wing’s 2006 publication on computational thinking laid the groundwork for CT’s popularity and integration in 21st century education theory.  Wing (2006) suggested that CT might be considered essential to all human endeavors as it is a distillation of a way that we naturally approach solving problems, managing our daily lives, and communicating and interacting with people.  It need not be relegated only to the STEM fields and computer science majors, because CT is not about getting humans to think like computers.  Rather, CT harnesses the natural outpouring of human cleverness, creativity, and problem solving that laid the foundations for the field of computer science in the first place (Wing, 2006). CT is about “…solving problems, designing systems, and understanding human behavior” by drawing on, and leveraging, the concepts fundamental to computer science (Wing, 2006, p. 33).

Though academics continue to debate an authoritative definition for CT, certain common themes are generally accepted characterizations of CT across the board.  These characteristics include:

  • Abstraction — thinking through abstract concepts and ill-defined problems, at times breaking them into smaller, digestible pieces, in order to move towards a more concrete, real-world solution (Wing, 2006).
  • Pattern Recognition —  recognizing useful patterns in data, filtering out the characteristics of patterns that aren’t needed, focusing on those that are (Wing, 2006).
  • Algorithmic Thinking — curating a list of steps that can be followed to solve a problem (Lyon & Magana, 2020).
  • Creative Problem Solving — developing a unique, context-based solution that is  considered original, valuable, and useful (Romero et.al., 2017).
  • Evaluating Solutions — considering the efficacy of a proposed solution to a problem, perhaps making considerations for factors like efficiency and resource consumption (Lyon & Magana, 2020).
Image sourced from https://koneilleci201.wordpress.ncsu.edu/2020/01/28/computational-thinking/

These facets of CT and the related skills are all integral parts of a 21st century education at all levels, including K-12 and postsecondary.  Indeed, “…computational sciences have been deemed essential to maintaining national competitiveness in the workplace and national security.” (Lyon & Magana, 2020, p. 1174)  For these reasons, the fundamentals of CT have been championed in education theory over the last decade, and nationally recognized standards like the Common Core State Standards and ISTE Standards for Students have pointedly emphasized the importance of “21st century skills” in K-12 education while simultaneously offering some clear guidance for what CT can look like in action. 

But what about higher education?  The implementation of CT in higher education classrooms is noticeably harder to call out, especially outside of computer science and engineering classrooms.  In my opinion, this is likely due to a number of factors including, but not limited to:

  1. Lack of collegial collaboration:  higher education disciplines are notoriously siloed. Meaningful integration of CT concepts outside of computer science and engineering programs demands intentional professional development for faculty, as well as interdisciplinary cooperation, both of which can be less accessible in higher education.
  2. Lack of resources: there is relatively little literature available which provides ideas for practical application of CT outside of computer science programs (i.e. coding and computer programming) at the postsecondary level. Additionally, published standards often lean more heavily towards K-12 education.
  3. Questions of applicability: the humanities often resist algorithmic ways of knowing because there is so much value placed on interpretation, subjectivity, and open debate about meaning (Czerkawski & Lyman, 2015).
  4. Just getting started: there is growing interest in translating CT pedagogy into a wide variety of disciplines in higher education (and K-12 for that matter), but the research and discussions are just getting started.  There is much yet to be explored.

Knowing that many STEM instructors in higher education automatically incorporate CT in their approach to teaching and learning because of the nature of their field, and knowing that engaging with CT in courses devoted to coding and programming are already integral to computer science and engineering majors, I seek to offer a few alternative examples of CT as it has been used to enhance teaching in learning in other kinds of higher education environments:

  • In a professional writing course taught at the undergraduate level, CT was used to systemize the writing process.  It called for a “deconstructive approach, breaking down the task of structured authoring into multiple layers of abstraction, and teaching each layer independently.” (Lyon & Magana, 2020, p. 1182)
  • In the fine arts, CT can be used as a tool to enhance creativity.  In one example, CT was used to create an organized system for tracing the origins of musical composers, which in turn inspired new creative endeavors based on the organized data. “…Algorithmic composition in music is, effectively, a human-computer collaboration–the computer serving as a tool that extends the composer’s ability to explore new musical ideas” (Edwards, 2011, p. 67)
  • The Stanford Literary Lab famously applied CT via Graph Theory to perform a “network analysis of character relationships and interactions” in a series of Shakespeare’s plays (Czerkawski & Lyman, 2015).
  • In the life sciences, CT has been used to inform systems theory and how to teach and understand biological processes, such as genetics, in an organized, logical fashion (Czerkawski & Lyman, 2015).
  • Utilizing a process dubbed “creative programming,” instructors may engage learners in the process of designing and developing an original work through coding. In this collaborative approach, learners are encouraged to co-construct knowledge in an interdisciplinary way. Examples might be to have students in a history course (co-)create a rendering of a city at a given historical period, or to present a traditional story in a visual programming tool like Scratch. In this kind of activity, learners must use skills and knowledge in mathematics, technology, language arts, and social sciences. (Romero et. al, 2017, para. 3)

Modeling and simulation activities are excellent examples of CT at work, and these types of learning activities can certainly extend themselves to many types of fields and disciplines.  Consider a learning activity where a group of undergraduate philosophy majors create a simulated narrative presentation wherein a human “character” makes a series of daily choices based on their moral philosophy or framework–almost like a “Choose Your Own Adventure” novel meets systems theory within one, or multiple, philosophical frameworks.  The simulation itself could be a computer-based product (or not), but regardless, the learning activity would draw upon many tenets of CT while also demonstrating in-depth knowledge of the discipline-specific subject matter.

All fields and disciplines require problem solving in some form.  Thus, it is reasonable to assume that CT may be useful in expanding the human ability to effectively problem solve in all fields.  In one study comparing the use of CT by an undergraduate computer science student and an art student, the researchers found that both students “…used various CT skills when solving all [italics added] problems, and the application of CT skills was influenced by their background, experiences, and goals.” (Febrian et. al., 2018, para. 1)  Regardless of training, background, or chosen major, CT enables postsecondary students to become more efficient problem solvers in all areas of life, teaching them to recognize computable problems and approach the problem-solving process as skillfully as possible (Czerkawski & Lyman, 2015).

References

Czerkawski, B.C. & Lyman, E.W. (2015). Exploring issues about computational thinking in higher education. TechTrends 59(2), 57–65. https://doi.org/10.1007/s11528-015-0840-3 

Edwards, M. (2011). Algorithmic composition: Computational thinking in music. Communications of the ACM, 54(7), 58-67. doi:10.1145/1965724.1965742  

Febrian, A., Lawanto, O., Peterson-Rucker, K., Melvin, A., & Guymon, S. E. (2018). Does everyone use computational thinking?: A case study of art and computer science majors. Proceedings of the ASEE Annual Conference & Exposition, 1–16.

Lyon, J. & Magana, A. (2020). Computational thinking in higher education: A review of the literature. Computer Applications in Engineering Education 28(5), 1174-1189. https://doi.org/10.1002/cae.22295

Romero, M., Lepage, A., & Lille, B. (2017). Computational thinking development through creative programming in higher education. International Journal of Educational Technology in Higher Education 14(42). https://educationaltechnologyjournal.springeropen.com/articles/10.1186/s41239-017-0080-z

Wing, J. (2006). Viewpoint: Computational thinking. Communications of the Association for Computing Machinery (49)3, 33-35. https://dl.acm.org/doi/fullHtml/10.1145/1118178.1118215?casa_token=DY3JiA-SOKMAAAAA:OYN4CIuf3LvuR1v4IiYsKQ-2J1KQMV6e0k6skWtib9uI02IKHhX9fEEA7rQC459Lk39QGworokaU 

Assessment in higher education during COVID-19 and beyond: Will it ever be the same?

Ridofranz/Getty Images

Perhaps the word “unprecedented” has been overused in recent months, but it consistently seems to be the most fitting word to express the seismic shifts in all areas of life that have occurred during the COVID-19 pandemic. As K-12 and higher education institutions worldwide have grappled with rapid pivots to online teaching and learning (and have continued in these blended and fully remote modalities for much longer than anticipated), academics are now taking a moment to reflect on the past year and its lasting implications for the world of education.  After all, as business theory would posit, disruption leads to innovation.

As an educator interested in online teaching/learning in post-secondary education, I would specifically like to explore how the last year of remote learning has impacted assessment strategies in higher education. How have widespread shifts to online teaching/learning impacted college students’ abilities to demonstrate competency in varied and student-driven ways?  Higher education is notoriously “old school,” and post-secondary classes are most frequently  lecture-based, led by instructors who are slower to adapt to more progressive, student-driven pedagogies.  And yet, as universities across the U.S. have made college and graduate school entrance exams like the SAT, ACT, and GRE optional for applicants in 2020 and 2021, a world beyond high stakes standardized testing can perhaps be imagined now more than ever.

Higher education instructors worldwide are already engaging in this issue and offering recommendations for ongoing change.  Perhaps surprisingly, some of the first publications I encountered came from educators in the graduate medical school community in both Australia and Pakistan. This was particularly striking since the medical sciences require lab work and clinical assessments which are particularly challenging to address in remote situations, as well as the fact that the medical sciences have long required high stakes testing at many stages of a medical student’s training.

According to Torda (2020), many medical school instructors in Australia have moved to lower the stakes of traditional written or multiple choice exams delivered online during the pandemic.  At the same time, a shift has been made to put more weight on multi-sourced feedback and student portfolios.  Where clinicals are concerned, simulation platforms such as the OSPIA (Online Simulated Patient Interaction and Assessment) system have been leveraged to bridge the gap until in-person clinicals may safely resume.  Another significant shift has been an emphasis on measuring a student’s ability to exhibit key professional skills, known in the medical community as “Entrustable Professional Activities,” over and above written examinations (Torda, 2020)..  In other words, students are being assessed on their ability to apply their learning in professionally relevant contexts.  Some of these skills include, but are not limited to, recommending and interpreting common diagnostic and screening tests, providing a (virtual) oral presentation of a clinical encounter, forming clinical questions and retrieving evidence to advance patient care, and collaborating with professional colleagues (Torda, 2020).  It was noted that, taken as a whole, these measures go a long way toward easing test anxiety and motivations to cheat in an otherwise high stakes, demanding field of study (Torda, 2020).

Additional examples of altered assessment strategies in the medical community have been reported in Pakistan.  Similar to Torda (2020), Khan and Jawaid (2020) posit that the pandemic has necessitated lowering the stakes of online-proctored, traditional exams.  The authors advocate for the use of student portfolios and video evidence of professional tasks completed, as well as synchronous open book exams.  The authors note that the aim of synchronous open book exams “…is to assess the ability of students to analyze and solve a problem, [and to] assess critical thinking and creativity. With open book exams taken in real time, the issues of cheating can be minimized.” (Khan & Jawaid, 2020, p. 109)

Changes in higher education assessment have also been reported in the United States. In June of 2020, Natasha Jankowski, in partnership with the National Institute for Learning Outcomes Assessment (NILOA), spearheaded a higher education survey meant to capture “a snapshot of assessment-related changes made during Spring 2020 in response to the sudden shift to remote instruction…” (Jankowski, 2020, p. 3). The survey included responses from faculty and staff at 624 different institutions, both public and private, with representation from all 50 states. The survey sought to record learning changes that higher education instructors were making, the impacts of those changes on assessment culture, and the role of student voice in the decisions (Jankowski, 2020).  The survey results showed that 97% of respondents made learning, instructional, and assessment changes of some kind during Spring 2020. Changes included modifying assignments and assessments, allowing flexibility in assignment deadlines, shifting to a pass/fail grading model, and modifying assessment reporting deadlines.  Though some respondents made changes that included accepting alternative assignments, this was a less often made change (Jankowski, 2020).  The survey also showed “…that assessment-related changes were undertaken to address student needs” (p. 3).  However, these changes may have had more to do with faculty/staff perception of student needs as opposed to action taken in direct response to student reports: “Information gathered from students was less likely to influence decisions on what to change, and students were less likely to be asked to identify their needs prior to decisions being made.” (p. 3) Consequently, it might be hard to define many of these changes in assessment as authentically “student-driven.” 

Nevertheless, it seems that the pandemic has disrupted “business as usual” in higher education such that many of the changes reported above may in fact have lasting impact with increasing opportunity for student voice to take a front seat in decision-making.  Dr. Funmi Amobi of Oregon State University’s Center for Teaching and Learning puts forth compelling arguments in favor of  “reimagining” assessment in higher education in light of the lessons we’ve learned in the pandemic (Amobi, 2020).  Amobi asserts that the radical move to remote instruction has “refocused attention on improving assessment practices to alleviate student stress and anxiety, emphasize learning, and redress inequities in student success.” (par. 2)  The author goes further and provides seven practical strategies for reimagining assessment in higher education.  Though these strategies can certainly be used effectively in remote learning environments, they are not only meant to solve problems related to online teaching and learning.  The strategies presented by Amobi (2020) should be taken seriously by all higher education instructors wanting to diversify their approach to assessments and create more student-centered learning experiences: 

  1. Use short, weekly quizzes to assess students formatively, and consider making the quizzes cumulative so that they may contribute to a summative assessment score.
  2. Ask for justification on multiple choice tests and grade the response instead of the answer.
  3. Create opportunities for collaborative, group tests.
  4. Have students construct exam questions themselves as a way of reviewing and exercising higher order thinking skills; then, include many of the student questions on the exam.
  5. Allow for notes or a study card and have students submit the prepared materials for credit along with the actual exam.
  6. Utilize practice tests.
  7. Spend time reviewing exams to address misunderstandings and improve future performance; consider giving credit for thoughtfully corrected exams where learning is evident.

In each of the reviewed publications, certain recurring themes were readily apparent: 1) it may be high time for colleges and universities to rethink the value of high stakes testing 2) varied assessment strategies allow for a more effective presentation of student learning 3) assessment is part of the overall learning process and should not be divorced from student voice 4) varied assessment strategies reduce test anxiety and the motivation to cheat (the ladder being oft-cited as a obstacle in online assessment). 

We must avoid the underlying assumption that more technology is needed in order to solve the problems that technology introduces.  In other words, as the pandemic continues to require extended engagement in remote teaching, higher education instructors must not assume that the only way to assess online is to find a way to virtually proctor the same exam that would normally be given in a physical classroom (Kumar, 2020).  Instead, educators at all levels may take this opportunity to make meaningful changes to their use of assessments, both now and into the future, thinking critically and creatively about how to best meet students where they’re at.  

References:

Amobi, F. (2020, November 12). Reimagining assessment in the pandemic era: Comprehensive assessment of student learning. OSU Center for Teaching and Learning. https://blogs.oregonstate.edu/osuteaching/2020/11/12/reimagining-assessment-in-the-pandemic-era-comprehensive-assessment-of-student-learning/

Jankowski, N. A. (2020). Assessment during a crisis: Responding to a global pandemic. National Institute for Learning Outcomes Assessment. https://public.uhcl.edu/education/centers-initiatives/planning-assessment/documents/niloa-covid-assessment-report.pdf

Khan, R. A. & Jawaid, M. (2020). Technology Enhanced Assessment (TEA) in COVID 19 Pandemic. Pakistan Journal of Medical Sciences 36, 108-110. 10.12669/pjms.36.COVID19-S4.2795

Kumar, R. (2020). Assessing higher education in the COVID-19 era.  Brock Education Journal 29(2), 37-4. https://journals.library.brocku.ca/brocked

Torda, A, (2020). How COVID‐19 has pushed us into a medical education revolution. Internal Medicine Journal 15(9), (1150-1153).  https://doi.org/10.1111/imj.14882

Student Flourishing in the Virtual Classroom

Image Source, Medium.com

There is no doubt that the COVID-19 pandemic has greatly accelerated the rate at which schools and universities of all shapes and sizes have had to move to online teaching and learning modalities, even if only as a short-term conduit for allowing formal education to continue in these unprecedented times.  There is also no doubt that this emergency shift to online teaching has left many concerned about overall student well-being including screen fatigue, issues of access and equity, teacher readiness, social-emotional support in a digital environment, and the overall efficacy of the educational endeavor for students of all ages in digital mediums.  Is there a light at the end of the tunnel?  Or might there already be some twinkle lights strung up along the tunnel walls guiding the way? 

In this post I’d like to explore some of the evidence that already exists in support of student flourishing—particularly at the postsecondary level—in hybrid or fully online programs, as well as what best practices can be used to support student well-being in all online teaching/learning endeavors, during COVID-19 and beyond.  Thankfully, the pandemic didn’t bring about the dawn of online pedagogy in higher education, and postsecondary educators have places to turn in order to think critically (and perhaps hopefully) about student success and well-being, be it academic or personal, in the digital classroom.

Evidence of Flourishing:

Few would argue that an in-person classroom experience can be identically replicated online.  In fact, those who attempt to do so have probably done so with disappointing results.  But perhaps educators shouldn’t necessarily be trying to replicate a physical classroom experience in an online environment.  Rather, they should think of the virtual classroom as a new endeavor; it is a new context with new possibilities to explore, and online pedagogy may bring new teaching/learning benefits to the table that a physical classroom lacks. 

Indeed, there’s evidence to suggest that a hybrid of in-person and online teaching may be the very best approach to postsecondary learning—with or without a pandemic—as it capitalizes on the “best of both worlds.”  In an extensive, multi-year case study conducted at the University of Central Florida in 2004, research showed that student success in blended programs (success being defined as achieving a C- grade or higher) actually exceeded the success rates of students in either fully online or fully face-to-face programs (Dziuban et al, 2004).  Furthermore, in a meta-analysis of studies on online and hybrid learning conducted by the U.S. Department of Education in 2010, it was reported that students in online and hybrid learning programs had more gain in their learning when compared to face-to-face modalities, and students in hybrid learning courses had the largest gains in their learning among their peers in all delivery formats (Means et al., 2010).  In yet another study (Chen & Chiou, 2014) measuring the learning outcomes, satisfaction, sense of community and learning styles of 140 second-year university students in Taiwan, results showed that students in a hybrid course had significantly higher scores and overall course satisfaction than did students participating in face-to-face courses. The results also indicated that students in hybrid learning classrooms actually felt a stronger sense of community than did students in a traditional classroom setting (Chen & Chiou, 2014).

While one must make many allowances for the various emergency situations brought on by the pandemic (and that there is a distinction between emergency remote instruction and true online teaching/learning), there is plenty of evidence to suggest that well-implemented online teaching/learning can truly enhance student learning beyond what might otherwise be accomplished in a fully face-to-face environment.

Some Best Practices in Online Instruction:

Technology-mediated education is making it possible for students to participate in programs, access content, and connect in ways they were previously unable to.  Rather than viewing the Internet as a necessary evil for distance learning that ultimately begets isolated student learning experiences, digital education should, first and foremost, be connective and communal.  This means a professor accustomed to lecture-based learning in a physical classroom will need to consider a new approach in order to prioritize student voice in the learning process.  In an online context, this means there should be dynamic opportunities for students to engage in debate, reflection, collaboration, and peer review (Weigel, 2002).

If educators are going to seriously account for the rich background experiences, varied motivations, and personal agency of their postsecondary learners, they must also take into account the larger “lifewide” learning that takes place within the lives of their students (Peters & Romero, 2019). Student learning at any age is both formal and informal, and what takes place in a formal classroom environment—digital or otherwise—is influenced by informal learning and daily living that takes place outside of it.  If deep learning takes place, a student’s world and daily life should be altered by the creation of new schemas and the learning that has taken place in a formal classroom environment.  In a multicase and multisite study conducted by Mitchell Peters and Marc Romero in 2019, 13 different fully-online graduate programs in Spain, the US, and the UK were examined in order to analyze learning processes across a continuum of contexts (i.e., to understand to what extent learning was used by the student outside of the formal classroom environment).  In this study, certain common pedagogical strategies arose across programs in support of successful student learning and engagement including:

  1. Developing core skills in information literacy and knowledge management,
  2. Community-building through discussion and debate forums,
  3. Making connections between academic study and professional practice,
  4. Connecting micro-scale tasks (like weekly posts) with macro-scale tasks (like a final project), and
  5. Applying professional interests and experiences into course assignments and interest-driven research.

(Peters & Romero, 2019).

In many regards, each of these pedagogical strategies is ultimately teaching students to “learn how to learn” so that the skills they cultivate in the classroom can be applied over and over again elsewhere. This means that, where digital learning is concerned, the most important learning activities aren’t actually taking place in a large, synchronous Zoom meeting or broadcasted lecture series.

On a practical level, educators can also give attention to some of these simple “tricks of the trade” that have been proven to enhance student learning experiences in a virtual classroom:

  1. Communicate often with students to promote a feeling of connectedness
  2. Create ample space for student voice
  3. Take care that a course set-up in a learning management system is intuitively laid out, action oriented, and adaptable to student needs
  4. Give timely feedback and highlight student strengths
  5. Create opportunities for synchronous activities when possible
  6. Be explicit about expected course outcomes

(Vlachopoulos & Makri, 2019)

At the end of the day, learning and schooling no longer have the same direct relationship they had for most of the 20th century; devices and digital libraries allow anyone to have access to information at any time (Wilen, 2009). Schools, teachers, and printed books no longer hold the “keys to the kingdom” as sources of information.  Online education, then, will not function effectively as a large-scale effort to teach students information through a standardized curriculum.  Rather, education must be a highly relevant venture that enables individual students to do something with the virtually endless information and resources they have access to (Wilen, 2009).

Student Agency & Connection Lead to Student Wellbeing:

When considering how to best support student wellbeing in an online learning environment (at every level), it’s important to remember that the student is not a passive entity.  Indeed, the extent to which students are able to exercise agency in their learning can have a significant impact on their academic success, their attitude towards the learning experience, and their social-emotional wellbeing.  In this case, agency can be interpreted as a student’s ability to exercise choice and be meaningfully present and interactive in the online learning environment.

One of the significant benefits of learning management systems and digital classrooms is the existence of a platform through which resources and learning materials can be shared and posted for any length of time.  Thus, students have the ability to review online course materials at their own pace and engage at a rate that makes sense for their individual needs (Park, 2010).  Allowing students the time and space to persist in completing online learning activities can have significant impact on a students’ success in an academic course (Park, 2019).

Additionally, game-based learning activities, opportunities for collaboration in group projects, participation in threaded discussions, and dedicated spaces for students to freely express their views all assist students in taking ownership of their learning and pursuing their learning interests as those interests materialize in—and overlap with—the course content (Vlachopoulos & Makri, 2019).  These are the activities that directly impact student engagement in a course, as well as the likelihood that a student will have a positive attitude towards the learning experience.

For many traditionally-aged students navigating undergraduate studies during the pandemic, the decreased ability to connect socially with peers, faculty, and support staff has had a direct, negative impact on their academic motivation and overall sense of wellbeing (Burke, 2020).  Thus, creating time and space in the digital learning environment for social interaction, open communication, and for students to gain a sense of identity within the virtual classroom is perhaps more important than ever. 

Finally, it’s very much worth mentioning that the extent to which all spheres of life have been impacted by COVID-19—not just the classroom—is unprecedented.  Helping students think of remote learning as an opportunity for growth, one that will have challenges and limitations as well as potential and new kinds of goals that can be achieved, can help them maintain a sense of purpose and direction amidst the chaos (Burke, 2020).  Growth mindset has already been proven to positively impact student learning at all levels—what better time to remind students (and educators) of the opportunities for growth in the present.

References:

Burke, L. (2020, October 27). Moving into the long term. Inside Higher Ed. https://www.insidehighered.com/digital-learning/article/2020/10/27/long-term-online-learning-pandemic-may-impact-students-well

Chen, B. & Chiou, H. (2014). Learning style, sense of community, and learning effectiveness in hybrid learning environment. Interactive Learning Environments, 22(4), 485-496. https://www.tandfonline.com/doi/abs/10.1080/10494820.2012.680971

Dziuban, C., Hartman, J., Moskal, P., Sorg, S., & Truman, B. (2004). Three ALN modalities: an institutional perspective. In J. R. Bourne, & J. C. Moore (Eds.), Elements of quality online education: Into the mainstream (127–148). Sloan Consortium.

Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Department of Education, Office of Planning, Evaluation and Policy Development. https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

Park, E., Martin, F., & Lambert, R. (2019). Examining predictive factors for student success in a hybrid learning course. The Quarterly Review of Distance Education 20(2), 11-27.

Peters, M. & Romero, M. (2019) Lifelong learning ecologies in online higher education: Students’ engagement in the continuum between formal and informal learning. British Journal of Educational Technology, 50(4), 1729.

Vlachopoulos, D., & Makri, A. (2019). Online communication and interaction in distance higher education: A framework study of good practice. International Review of Education, 65,605–632. https://doi.org/10.1007/s11159-019-09792-3

Weigel, Van B. (2002) Deep learning for a digital age.  San Francisco, CA: Jossey-Bass.

Wilen, T. (2009). .Edu: Technology and learning environments in higher education. Peter Lang Publishing.

A Few Best Practices for Online Learning & Adoption in Higher Education

Though the digital age may not actually be changing a student’s capacity to learn, it’s certainly changing how students access content and participate in learning environments. Digital technology thoroughly transforms the way in which we create, manage, transfer, and apply knowledge (Duderstadt, Atkins, & Van Houweling, 2002). Unsurprisingly, it’s also changing how educators teach, particularly with technology-mediated instruction in higher education. The demand for online instruction is on the rise.  In the United States alone, the number of higher education students enrolled in online courses increased by 21% between fall 2008 and fall 2009, and the rate of increase has only grown in recent years, both nationally and globally (Bolliger & Inan, 2012).  Of course, the COVID-19 pandemic of 2020 has also necessitated a radical—though in some cases temporary—shift to online learning modalities at all educational levels across the globe.

Fortunately, there’s evidence to support that digital education incorporation can enhance pedagogy and improve overall student performance at the college level.  An extensive, multi-year case study conducted at the University of Central Florida showed that student success in blended programs (success being defined as achieving a C- grade or higher) actually exceeded the success rates of students in either fully online or fully face-to-face programs (Dziuban, C., Hartman, J., Moskal, P., Sorg, S., & Truman, B., 2004).

In the switch to online teaching and learning, a clear challenge is presented: teaching faculty are faced with a need to move their programs and classes into online/flexible learning formats, regardless of their discipline or their expertise/ability to do so.  It is not uncommon for teachers, no matter the level at which they teach, to be asked to implement something new in their classroom without sufficient support, professional development, or resources to make the implementation successful.  The need for appropriate training becomes that much more pressing when educators are asked to engage with an entirely different instruction medium from that which they are accustomed to.  In the case of blended or online learning, many faculty will need to develop completely new technological and/or pedagogical skills.  While a number of scholars have conducted investigations into the effectiveness of blended or online learning, very few have provided guidance for adoption at the institutional level (Porter, Graham, Spring, & Welch, 2014). 

Far from being a comprehensive guide, this post seeks to explore a few major themes and best practices for online learning in postsecondary education which may prove helpful for teaching professionals and higher education institutions heading into an otherwise unfamiliar world of digital education.

Create a Learning Community:

Digital education is made possible by computers and the internet.  In the age of the Internet, the computer is ultimately used most to provide connection, whether that be through social media, e-commerce, gaming, publications, or education (Weigel, 2002).  Technology-mediated education is making it possible for students to participate in programs, access content, and connect in ways they were previously unable to.  Rather than viewing the Internet as a necessary evil for distance learning that ultimately begets isolated student learning experiences, digital education should, first and foremost, be connective and communal.  This means a professor accustomed to lecture-based learning in a physical classroom may need to consider a new approach in order to make space for student voice in the learning process.  In an online context, this means there should be dynamic opportunities for students to engage in debate, reflection, collaboration, and peer review (Weigel, 2002).

Beyond Information Transfer:

Learning and schooling no longer have the same direct relationship they had for most of the 20th century; devices and digital libraries allow anyone to have access to information at any time (Wilen, 2009). Schools, teachers, and even books no longer hold the “keys to the kingdom” as sources of information.  Higher education, then, will not function effectively as a large-scale effort to teach students information through a standardized curriculum.  Rather, education must be a highly relevant venture that enables individual students to do something with the virtually endless information and resources they have access to (Wilen, 2009).

Relevance:

If university instructors are going to seriously account for the rich background experiences, varied motivations, and personal agency of their postsecondary students, they must also take into account the larger “lifewide” learning that takes place within the life of most college students (Peters & Romero, 2019). Student learning at any age is both formal and informal, and what takes place in a formal classroom environment is influenced by informal learning and daily living that takes place outside of it.  Likewise, if deep learning takes place, a student’s world and daily life should be altered by the creation of new schemas and the learning that has taken place in a formal classroom environment. 

In a multicase and multisite study conducted by Mitchell Peters and Marc Romero in 2019, 13 different fully-online graduate programs in Spain, the US, and the UK were examined in order to analyze learning processes across a continuum of contexts (i.e., to understand to what extent learning was used by the student outside of the formal classroom environment).  Certain common pedagogical strategies arose across programs in support of successful student learning and engagement including: developing core skills in information literacy and knowledge management, community-building through discussion and debate forums, making connections between academic study and professional practice, connecting micro-scale tasks (like weekly posts) with macro-scale tasks (like a final project), and applying professional interests and experiences into course assignments and interest-driven research (Peters & Romero, 2019).  In many regards, each of these pedagogical strategies is ultimately teaching students to “learn how to learn” so that the skills they cultivate in the classroom can be applied over and over again elsewhere.

Professional Development:

Still there remains the question of implementation.  In order for the mature adoption of digital education to take place, faculty need to be given time and training to help them develop new technological and pedagogical skills.  If an institution fails to provide sufficient opportunities for professional development, many faculty members will likely fail to fully embrace the shift to an online format, and will instead replicate their conventional teaching methods in a manner that isn’t compatible with effective online instruction (Porter, et al., 2014).  If higher education institutions are committed to delivering high quality instruction in all contexts, it will be important for administrators to retain qualified instructors who are motivated to teach online and who are satisfied with teaching online (Bolliger, Inan, & Wasilik, 2014).

 In a 2012-2013 survey of 11 higher education institutions reporting on their implementation of blended learning programs, Wendy Porter et al found that every university surveyed provided at least some measure of professional development to support faculty in the transition.  Each university had their own customized approach, but the fact that developmental support was prioritized in some regard remained consistent across all of the institutions in the survey.  Strategies used for professional development in digital learning included presentations, seminars, webinars, live workshops, orientations, boot camps, instructor certification programs for online teaching, course redesign classes, and self-paced training programs (Porter et al., 2014).

Digital Literacy:

Digital literacy among higher education faculty can’t be taken for granted.  A recent Action Research study aimed at exploring the digital capacity and capability of higher education practitioners found that, though the self-reported digital capability of an individual may be relatively high, it did not necessarily relate to the quality of their technical skills in relation to their jobs (Podorova et al., 2019).  Survey results from the study also showed that the majority of practitioners (41 higher education professors in Australia) were self-taught in the skills they did possess, receiving very little formal training or support from their employer, even with technology devices and tools directly pertaining to teaching and assessment (Podorova et al., 2019).  Though this data relates to a specific case study, it is not difficult to imagine that higher education faculty in institutions all over the world might report similar experiences.  If faculty aren’t given sufficient technological support and training, they will be less satisfied in their work and, ultimately, the student experience will suffer (Bolliger, et al., 2014).

Institutional Adoption:

In addition to providing sufficient technological or pedagogical resources, it is important for university administrators to communicate the purpose for online course adaptation.  In a later study conducted by Wendy Porter and Charles Graham in 2016, research indicated that higher education faculty more readily pursued effective adoption strategies when they were in alignment with the institution’s administrators and the stated purpose for doing so (Porter & Graham, 2016). If faculty members are, in essence, adult learners being asked to acquire new skills, it is essential to take their own motivations for learning into account.  Additionally, sharing data and course feedback internally from early-adopters to online instruction can go a long way in helping reticent faculty feel ready to approach online learning (Porter & Graham, 2016).  Institutional support is cited frequently in literature pertaining to faculty satisfaction in higher education. In the domain of online learning, institutional support looks like: providing adequate release time to prepare for online courses, fair compensation, and giving faculty sufficient tools, training, and reliable technical support (Bolliger et al., 2014).

One effective approach to professional development for online learning places professors in the seat of the student.  At Hawaii’s Kapi’olani Community College on the island of Oahu, Instructional Designer Helen Torigoe was charged with training faculty in the process of converting courses for online delivery.   In response, Torigoe created the Teaching Online Prep Program (TOPP) (Schauffhauser, 2019). In TOPP, faculty participate in an online course model as a student, using their own first-hand experience to inform their course creation.  As they participate in the course, faculty are able to use the technology that they will be in charge of as an instructor (programs like Zoom, Padlet, Flipgrid, Adobe Spark, Loom, and Screencast-O-Matic), gaining comfort and ease with the tools and increasing their overall digital literacy.  Faculty also get a comprehensive sense for the student experience while concurrently creating an actual course template and receiving guidance and support from the TOPP course coordinator.  Such training is mandatory for anybody teaching online for the first time at Kapi’olani Community College. A “Recharge” workshop has also been created to help faculty engage in continued learning for best practice in digital education, ensuring that faculty do not become static in their teaching methods and are consistently exposed to new tools and strategies for digital education (Schauffhauser, 2019).  Institutions that participate in online education need to provide adequate training in both pedagogical issues and technology-related skills for their faculty, not only when developing and teaching online courses for the first time, but as an ongoing priority in faculty professional development (Bolliger et al., 2014).

Summary:     

The number of graduate courses and programs that must be offered in an online format is increasing in many higher education environments.  Effective online educators will acknowledge the unique needs of their postsecondary learners: that their students need to have their background experiences and context utilized in the learning process, that their learning needs to be relevant to their life and work, and that their learning needs to be providing them with actionable skills and learning strategies that ultimately change how they interact with their world.  Effective online learning will also provide ample space for student connection and active participation.  This means there should be dynamic opportunities for students to engage in debate, reflection, collaboration, and peer review (Weigel, 2002).  Additionally, online learning ought to be a highly relevant venture that enables individual students to do something with the virtually endless information and resources they have access to (Wilen, 2009).  Yet in order for the mature adoption of digital education to take place, faculty need to be given time and training to help them develop new technological and pedagogical skills.  This training needs to happen with initial adoption and as an ongoing venture.  One example of highly effective faculty professional development can be found in Instructional Designer Helen Torigoe’s Teaching Online Prep Program (TOPP) (Schaffhauser, 2019).  In this program the instructors become the students as they familiarize themselves with a new learning system, create a customized course template, and get feedback and support from knowledgeable online educators.  In short, well-equipped, well-trained, and well-supported graduate faculty are fertile ground for effective online education.

References

Bolliger, D. U., Inan, F. A., & Wasilik, O. (2014). Development and validation of the online instructor satisfaction measure (OISM). Educational Technology Society, 17(2), 183–195.

Duderstadt, J., Atkins, D., Van Houweling, D. (2002). Higher education in the digital age: Technology issues and strategies for American colleges and universities. Praeger Publishers.

Dziuban, C., Hartman, J., Moskal, P., Sorg, S., & Truman, B. (2004). Three ALN modalities: An institutional perspective. In J. R. Bourne, & J. C. Moore (Eds.), Elements of quality online education: Into the mainstream (127–148). Sloan Consortium.

Peters, M. & Romero, M. (2019) Lifelong learning ecologies in online higher education: Students’ engagement in the continuum between formal and informal learning. British Journal of Educational Technology, 50(4), 1729.

Podorova, A., Irvine, S., Kilmister, M., Hewison, R., Janssen, A., Speziali, A., …McAlinden, M. (2019). An important, but neglected aspect of learning assistance in higher education: Exploring the digital learning capacity of academic language and learning practitioners. Journal of University Teaching & Learning Practice, 16(4), 1-21.

Porter, W., & Graham, C. (2016). Institutional drivers and barriers to faculty adoption of blended learning in higher education. British Journal of Educational Technology, 47(4), 748-762.

Porter, W., Graham, C., Spring, K., & Welch, K. (2014). Blended learning in higher education: Institutional adoption and implementation. Computers & Education, 75, 185-195.

Schaffhauser, D.  (2019). Improving online teaching through training and support. Campus Technology. https://campustechnology.com/articles/2019/10/30/improving-online-teaching-through-training-and-support.aspx

Weigel, V.B. (2002) Deep learning for a digital age. Jossey-Bass.

Wilen, T. (2009). .Edu: Technology and learning environments in higher education. Peter Lang Publishing.