Blog 165

Welcome to Part II of this blog post looking at my personal experience with re-thinking assessments in an online environment.  Last week I began this post by providing some initial information on the planning and rationale behind the changes I made and ended with specific examples of how I scaffolded the student learning process.  You can read Part I here!

In this second part of the post, I will further highlight some of the assessment strategies used for this course, including the use of rubrics, experiential education components, peer-review, and a gamified strategy.

 Rubrics

I spent a considerable amount of time studying a wide sample of rubrics. Some were specific to eClass templates and simplifying while not losing nuance and rhetorical advantage on discussion forums, short writing and speaking assignments submitted online in textual and video/audio formats, as well as creative assignments (through kritik.io) and tests online using eClass quiz functions. For example, please see two of these rubrics highlighted below: one for a “Speaking Activity” that was assigned early in the first weeks of classes, to stimulate discussion, begin to engage the students in “flipped classroom” style activities to show how important their questions are for guiding our focus in the course and Zoom class sessions, and also to model and refine their crafting of superior questions that move more deeply into grasping and engaging with the materials.

 

Figure 1: Rubric for “Creative Assignment 1”

The above rubric was created on the Kritik.io platform; one of its key adaptive and useful features for instructors is that the rubrics are fully customizable. There are six pre-loaded basic templates for rubrics (“Creative Communication”, “Essay”, “Question”, “Teach Peers”, “Lab Reports”, “STEM”, “Presentation Lab” and “Programming”) some of which (the last few) were recently added by Kritik during the Fall 2020 term. When I programmed these into my course, I used the ones available as a basis (“Creative Communication”, “Essay” and “Question”) and added or modified the criteria according to my course’s objectives and needs.

Figure 2: Rubric for “Speaking Activity 1”

EE components

Of utmost consideration was to facilitate the engagement of students with the course materials and to have them apply their work with those to a format that could be shared with each other, expanded upon through commentary and questioning, as well as make some of those creative products of their thinking available to other audiences such as the Spanish Program’s and student’s own social media sites. Creative application of material in real time, and in asynchronous formats were integrated. Connecting while also protecting privacy and rights was also a priority. A key component for the latter aspect in the Winter term connects the study of the texts and the group projects with a speaker series virtually organized by other cultural and education centres in Toronto, Uruguay and Argentina and expanded to invite attendees from other cultural communities across Canada. The series titled “Inspirad@s” are on the third Monday of each month from February to May, 2021) and will connect to certain readings we complete in our course those months.

Peer Review

Finally I would like to describe a novel approach and system for peer review that I integrated into this course this year using the program called Kritik (pronounced like ‘critique’). At first this took some time to adjust, given that students and I all needed some time to become familiarized with the way the cyclical patterns and algorithms work on the site, while also dealing with the overwhelm that most of us experienced during the first phase of the 2020 pandemic measures effects. The following is an excerpt from page 4 of my AP SP 2200 syllabus; it is an overall description of the application of the Kritik components and was based on a sample basic text sent to me by tha site’s support representative (one is assigned to each instructor):

Kritik.io

In this course, we will be using Kritik.io, a peer-to-peer learning platform. It is an engaging and gamified web technology that helps you develop your higher-order-thinking skills according to Bloom’s taxonomy of cognitive thinking. It includes Creation and Evaluation skills.

When you participate in Kritik activities, you will receive 3 scores: “Creation” score, “Evaluation” score and “Participation” score. Together, these will add up to 20% of your final course mark. To understand what these scores are and how they are calculated, please read the section “how scoring works” on Kritik’s help center here.

An email invitation will be sent to your school email account that contains the link to register an ac- count on Kritik online and enrol in the course. You MUST use your university email to sign up in or- der to access the course. If you did not receive any email yet, please contact Kritik using the live chat on kritik.io.

How to get help: If you have any questions about Kritik, please use the live chat in the app. A human agent will respond promptly in a few minutes during business hours. You can also visit Kritik Help Cen- ter at https://www.kritik.io/student-user-guidance-page, which outlines an overview to get you up and running on the system.

* Creative Projects and the Oral Presentation will be submitted through Kritik.

My aim is to integrate additional technology tools that will enhance the eLearning environment of my courses, while creating an engaging, motivational and safe space for my students. That space is now virtual, so as the tools I integrate must:

• add to the relationship building of community in the class,
• optimize the clarity, alignment and purpose of each assignment with the stated course outcomes,
• and develop higher order thinking aims of the course.

I had often used many multimedia platforms for my courses such as for my fourth year seminar “Literature and Music in Spanish America,” courses at various levels of Spanish as a second language and culture courses, world literature, etc., including videos, YouTube, Vimeo, social media, films, sound recognition software, interactive learning platforms, Kahoot, Moodle, etc. These multimodal approaches not only energize the spaces for higher learning but also add dimensions that would be lost in text only based delivery and assessment.

When I transferred these aspects to the post-Covid remote teaching and learning environment, I was enthusiastic about expanding my educational toolkit, as long as these tools could enhance the pedagogies and engagement methods I seek to achieve in my courses.

Applying KRITIK for assessment, developing critical thinking and peer connection/class community

One of the first new technologies I have learned about since May, while I have been preparing a second year Hispanic literary readings course, is Kritik. This is a gamified approach for building in peer review and feedback cycles into assessment of assignments. I have set up several activities on that platform so far, and am planning to use it for 20% of the course work. I may adjust this for future offerings of this and other course, after I see how it goes this year. While it’s an external tool, Kritik is not a replacement for Moodle or online resources associated with our text (or platforms such as Zoom, Adobe Connect, etc.) or content delivery. It is an assignment based tool, which offers both templates for and opportunities to disseminate assignments to students. (Assignments can be uploaded as any type of file, including video and audio.) Its goal is not to overlap with other platforms; rather, its innovation is that all tasks are peer evaluated, teaching and building evaluation, analytical and communication skills in students as they 1) evaluate each other’s work, and 2) consider and rate the accuracy of classmates’ evaluation of their own work. Its main aim is to develop critical thinking (following Bloom’s Taxonomy) and a very real-world skill that is to learn to evaluate others’ work. Students are offered algorithm based feedback to help them improve on their evaluations of others. As their ability to evaluate others improves, so, presumably do their skills relevant to successful realization of own assignments, including their critical thinking, etc. They then are motivated to achieve higher scores on that platform. It can be helpful for peer evaluation of video presentations, website creation projects (using Wix or Weebly for e.g.) or writing assignments. It can also be used to upload smaller assignments, and it handles many other software other than those of word processing. What I like about it is the gamification of the approach, as well as the clear yet colourful interface, that is an appealing complement to eClass.

Most students immediately took to the platform, and appreciated its attractive interface and ease of use, while others took longer to achieve this appreciation. In one case, the intense turnover of the review process, from evaluation stage and feedback following the submission of an assignment was intimidating and caused some shyness regarding the ability of the student to dedicate adequate time and focus on the peer review. Nevertheless, this tends to occur even in courses where no such software is integrated, and I have seen it when I have used other forms of peer review for classmates’ individual and/or group presentations. As we know in our experienced capacity as instructors, and as students find out when we engage them in these types of activities, grading others’ work can be the most challenging part of any academic journey. For that reason I began with low stakes activities in the first semester (two “Question” assignments, one as a “Speaking Activity” on video or audio, and the other in textual format, as well as one small creative assignment). As we got to know each other, the course format and the Kritik platform, while reflecting on the pros and cons of the process, our comfort level increased, and I gained insights on how best to time each of the three parts of each assignment (Creation, Evaluation, Feedback); the latter is best completed in a 1-2 day timeline, while the evaluation stage needs a wider time window. For the second semester we have the opportunity to apply the process to larger assignments (a second creative assignment — of flash fiction— and the group project, in two parts). I continue to learn how best to approach the training of students in this more advanced skill of peer reviewing for quality as well as for stimulating each other’s thinking processes and creativity, while also linking this to the scaffolded and summative work as we evolved towards the culmination of the course, to prepare them for upper-level work in: 1) the study and interpretation of cultural products such as novels, poems, plays and essays; 2) the understanding of the social, cultural, historical and political frameworks that shaped the Hispanic world, as evident through their cultural productions; 3) the familiarity, application and higher ability to navigate close readings of text and the terminology applied to it; and 4) engagement with the larger Hispanic world in general, and using the Spanish language in cultural context.

Much of the criteria and many of the characteristics I describe above are transferable to other courses in humanities, STEM and other language studies. I hope it can be useful to you as well. Please let me know if you’d like to discuss any of the above further: mfiguere@yorku.ca.

 Resources you may like to explore

• Resource 1: How to Make Smart Choices About Tech for Your Course (Chronicle of Higher Ed)
• Resource 2: Gamification in Online Learning (Purdue)
• Resource 3: A Rubric for E-Learning Tool Evaluation (via Western University)
• Resource 4: Engineering Education: Is Problem-Based or Project-Based Learning the Answer?
• Resource 5: “Teaching the Novel in the Audio Age” by J. English, PMLA 135:2 (2020): 419-426)
• Resource 6: “Absorb, Do, Connect activities” by Teaching Commons leaders Yelin Su and Lisa Endersby, resource provided in the TC course “Taking it Online – July” by Robin Sutherland-Harris and Eliana El Khoury.
• Resource 7: Virtual Learning: 33 Ways to Build Relationships
• Resource 8: Engaging the online learner: Activities and resources for creative instruction, by R. Conrad & J.A. Donaldson.
• Resource 9: “Authentic Assessment Toolbox” by J. Mueller
• Resource 10: A Visual Guide to Innovative Assessments from Queen’s University, provided in the TC course “Assessing Learning Online – Accelerated” by Linda Carozza.
• Resource 11: Favourite Poem Project (R. Prinsky)
• Resource 12: “Online Assessment in Higher Education,” by A. Weleschuk, P. Dyjur & P. Kelly, Taylor Institute for Teaching and Learning Guide Series, University of Calgary.
• Case studies to see how other instructors have been using Kritik in their courses (provided by David Liptrot, Kritik Education Group):
  • Steven Forsey – University of Waterloo – Organic Chemistry Kritik for STEM Courses
  • Dr. Jeff Boggs – Brock University – Geography Using Kritik to Save Time and Address Dwindling TA Hours
  • Nada Basir – University of Waterloo – Entrepreneurship and Business Kritik’s Group Dynamics for Peer Assessment
  • Kaston Anderson-Carpenter – Michigan State University – Psychology Building Term Papers Through Kritik
  • Alexander Gainer – University of Alberta – Economics Kritik vs LMS
  • Elliott Currie – University of Guelph – Accounting and Financial Management Case Analysis of Kritik

About the Author

Maria Figueredo is an Associate Professor with the Department of Languages, Literatures and Linguistics (LA&PS) at York University, where she teaches courses in Spanish language, Hispanic cultures and literature. She received the President’s University-wide Teaching Award in 2016. She previously served as President of the Ontario Chapter of the American Association of Teachers of Spanish and Portuguese (AATSP), and as Delegate of Region 1 of the Modern Language Association (MLA). At York University she has served for many years as Coordinator of the Spanish and Portuguese Studies Section and was Director of Undergraduate Program Director–Languages and Literatures from 2011 to 2013. Her research on the relationship of literature and music in Latin America, contemporary innovations in Spanish American literature, digital and e-poetry, and teaching has been published in books, specialized journals and cultural magazines. She was a York-Massey Fellow from 2008-2009 and since then is Senior Fellow at Massey College of the University of Toronto. At York University, Maria established a trilingual journal titled Entre Voces, together with students, to publish student work from across the university and of the greater GTA community, in Spanish or Portuguese with English translation. She also created a Pan Am Games “Poet-Tree 2015” community Ignite project, which included the publication of an edited anthology The Poetry of Sports & the Sport of Poetry: POET-TREE 2015. She is also a member of the League of Canadian Poets.