How Much Do You Really Know About Student Self Assessment?
Self-assessment is an undersung hero in the online instructor’s toolbox. It can not only help students develop skills in critical analysis, research by Sharma, et al. (2016) found it can increase their interest and motivation level, leading to enhanced learning and better academic performance. How ‘bout them apples?! Watch on to learn more.
The Most Effective Way to Elimate Barriers to Students' Learning
Universal Design for Learning (UDL) is a framework for thinking about teaching and learning that offers flexibility in the ways students access course material, engage with it, and show what they know. UDL principles benefit all learners by building in responsiveness that can be adjusted for every learner’s strengths and needs.
At the end of the semester, each student receives a single letter grade that summarizes how well they learned the content of our course. But, how much of that grade is really a measure of what they learned? How much of a student’s final grade is based on:
How complicated is their life?
Students with many responsibilities (such as unpredictable work schedules or family members who face emergencies) may lose points on late work because they need to choose between helping their child and helping themselves.
One of the concerns faculty have with removing late penalties is that students will abuse this allowance. This was not my experience when I went from very harsh late penalties to none at all. Most students still completed the assignments on time. The main difference I saw was that students who would likely have dropped due to missing early assignments stayed in the class and learned the content.
How much extra time do they have?
When we give students extra points for activities that are time intensive (such as watching a movie and connecting it to course content), we may be grading students on how much free time they have. Faculty often use extra credit as a “slush fund” to make up for points deducted due to things like late work. If we remove those penalties, students can focus on the learning rather than “making up points”.
When we make assumptions about what a “good” essay looks like or the test-taking skills students bring, we may be measuring which high school students attended rather than their knowledge of our content.
When we have harsh late policies but bend them for students who request an extension, we are actually measuring a student’s willingness to tell us about their challenges, their feelings of being worthy of special treatment, and their cultural background. It’s much more fair to let all students know up front that late work is acceptable. That is a move towards a more equitable learning environment.
A concern I often hear about removing late penalties is that this makes the class less fair to students who complete the work on time. Treating all students equally can seem like the fairest approach but, in actuality, we are creating a playing field that benefits students who come to our class from backgrounds most similar to our own. Clearly outlining the flexibility in our course policies helps us build a course that responds to individual student needs. This lets us and the students focus on the course content rather than navigating course logistics.
Let’s revisit the points marketplace
In the table above, a student may choose to lose points for a late assignment over losing hours at work. After all, they can make up these points with extra credit but can’t make up the lost wages. And, a student with limited time may skip the reading. This reading takes the most time and results in the fewest points/hour. Unfortunately, this is also the single most useful item on the list. I highly encourage you to read Grading for Equity (for zero points)! It brings a fascinating perspective that completely changed the way I look at grading.
Disinvestment from the points marketplace
Instead, what if we focus our grading directly on what we want students to learn and remove all the confounding variables that add inaccuracies to our measures.
For example: I want to measure a student’s understanding of the respiratory system. I could ask students to write an essay - but this measures writing skill AND their understanding of the respiratory system. The writing skills are a confounding variable because they mask the true measure of a student’s understanding of the respiratory system. Similarly, using a timed multiple choice test adds the confounding variables of how fast students recall information and their ability to parse multiple choice test questions.
An alternative approach to this assignment is to tell students exactly what you intend to measure and let them choose how best to demonstrate this knowledge. For example:
Trace the movement of a molecule of oxygen from outside the body until it reaches a red blood cell. This may be easiest to answer using a numbered list, but you are welcomed to approach this however works best for you.
Include the following structures: alveoli, bronchus, bronchioles, epiglottis, larynx, pharynx, trachea. Briefly describe each of these structures.
Also include the terms: diffusion, oxygen and carbon dioxide.
The assignments in my biology class give students options in how they demonstrate their knowledge. Students choose to describe this process using a wide variety of strategies: an essay, labeled diagram, flowchart, video, and many more. I’ve been amazed at all the creative and engaging strategies students find for explaining concepts. Grading becomes fun! (yes, really…) The key to giving students options in how they demonstrate their knowledge is to be clear about what I am assessing and grading. Good rubrics (like the one below) are essential.
Criterion
Exemplary
Accomplished
Developing
Structures are described correctly
3
2
1
Flow of oxygen molecule is accurate described using the listed structures
3
2
1
Diffusion, oxygen, and carbon dioxide are accurately identified
3
2
1
Compare this rubric with the table at the very beginning of this article. What are students asked to focus on? Which gives us more accurate information about what a student is learning? I invite you to share your thoughts and equitable grading approaches in a comment below!
During the summer of 2020, as the pandemic made it obvious we weren’t going back to “normal life” any time soon, my curriculum needed to reflect the massive changes that were happening in our society. Using the traditional STEM assessment style of short answer or multiple choice exams would not function well within this online learning environment. I could continue to give traditional exams, but would they actually be a valid measure of student knowledge? And, more broadly, would these exams serve my students in helping them to develop skills necessary to be successful beyond my class?
Additionally, if I were to continue to use traditional assessments I would need to employ an online proctoring tool. This tool would help me maintain academic integrity, to an extent. However, these proctoring tools have significant implications for student equity. Knowing this, I could not, in good conscience, use one.
In 2019, I made the decision to leave my exams and not use an online proctoring tool. But this left me in a tough spot. How would I measure a student's knowledge without using exams?
Switching Assessment Styles
As an undergraduate STEM student and a STEM instructor, exams are the only type of assessment I have ever known. As I made this change, I began to realize that it would be important to switch to assessments that enabled my students to demonstrate their knowledge and develop new skills that could be used in other classes, as well as in life beyond higher education. With access to an abundance of human knowledge at our fingertips via the internet, the ability to research, synthesize, and communicate ideas is of more value to my students’ future than memorizing all the steps of photosynthesis for an exam. With this shift, I was able to move from assessing rote memorization to critical thinking skills – isn’t that what we all should be striving to do? I was also able to connect abstract concepts to current events or students’ daily lives, making them more meaningful and memorable.
Through this process, I developed a set of projects that draw on the principles of authentic assessments to assess student learning. I provide the basic structure of what needs to be included in the project so I can assess my students’ comprehension of the concepts, but the format of the project is generally open-ended, and multimedia projects are encouraged.
One example is a role-play scenario where students step into the role of interns for a state government committee on health and human safety. Their goal is to brief the state representative for whom they work about the rise of antibiotic resistant bacteria. This project was inspired by the Performance Assessment Resource Bank. In the brief, students must include the following:
A discussion of what makes bacteria different from other forms of life
An explanation of how bacteria become resistant to antibiotics, in evolutionary terms
A description of environmental conditions that select for antibiotic resistant mutations
A discussion of potential state-wide solutions that can be implemented to slow the development of antibiotic resistant bacteria
Within this one project, I was able to assess students’ comprehension of several learning goals: their ability to distinguish between prokaryotic and eukaryotic cells, as well as natural selection and evolution. This project also required students to demonstrate their ability to apply their knowledge of the evolutionary process to evaluate large-scale solutions to combat this issue. The form of the final product was entirely up to the students. One student, who was studying digital marketing, built a website. Another held a mock webinar. Some typed their project into a traditional research essay. Even though their final projects took many forms, grading and assessing their work was not as challenging as I expected because I provided a clearly defined rubric.
Here is a 3-minute video explanation I provide for my students about this project:
Reactions to a New Assessment Style in a STEM Course
During the week leading up to the start of the semester as students were exploring our syllabus and Canvas course, I had several inquiries about exams. Students asked, “When are the exams?” and “Will we need to use [proctoring service] to take exams in this class?” After fielding several variations of these questions, I explicitly explained to my students my philosophy for adopting this new assessment strategy and why we would not have any exams. The idea of being able to show their knowledge outside of an exam in a science class was, at first, mysterious to students. However, they quickly acclimated to this new style of assessment as I promptly answered their questions.
Student feedback about this new assessment strategy was very positive. In an anonymous course evaluation, 97% of students rated the class as “always or almost always having assessments that are related to course material.” In another metric, 100% of students rated the class as “always or almost always having activities and projects which are useful for learning and understanding.” Students reported the projects as “fun and interesting” and said they “helped [to]… understand this subject better.” One student stated these projects helped them “gain a better understanding of the topic when applying it to real life,” which was my intent when making this shift.
In making this change to my assessments, I was met with some skepticism and backlash from colleagues, which resulted in me being reluctant to speak out about equity and assessments in online learning. When I did speak out, I received push back from colleagues saying “Students will have to get used to exams,” as well as, “There’s just no other way to assess learning in my class” except through exams. I even had a colleague claim I was calling anyone who used proctoring tools and exams “racist.” I see now that this reaction is tied to a larger, systemic issue about power and privilege in White dominant culture but I also know it made me hesitate to discuss the topic of assessments and proctoring tools again.
In Fall of 2020 I was due to be evaluated, and as a part-time faculty member I was incredibly nervous that this different assessment style would be seen as inferior, and thus my employment status and income would be impacted as a result. Luckily, despite the backlash I had received, I had many other colleagues, including my evaluators, who were curious and encouraged by these efforts to adopt different assessment styles. Out of this discussion about assessment and proctoring tools that was met with backlash, I was able to open a conversation about rethinking how we assess learning in STEM. Yes, this is currently an uncommon way to approach assessment for many STEM classes, and can be a challenging pivot to make. But, if we’re truly dedicated to closing opportunity gaps then we must make STEM courses more equitable for diverse learners.
Brookhart, S. M. (2018). Appropriate criteria: Key to effective rubrics. Frontiers in Education, (3)10. doi:https://www.frontiersin.org/article/10.3389/feduc.2018.00022 .
A fundamental aspect of instruction is the assessment of student learning. The rapid response to move classes online in a pandemic has exposed concerns surrounding the practice of online proctoring. There are many online proctoring features offered by companies such as Proctorio, Examity, Honorlock, and Respondus. The methods that do not require a webcam include locking down the students’ browser so they cannot perform functions such as open another application or tab, use the toolbar, copy/paste, or print screen while taking an exam. The intrusive methods include requesting a photo ID, activating facial recognition, and a live proctor monitoring for sounds and motions. Sessions are typically recorded from the exam start to finish and a live proctor can monitor potential testing infractions as they occur. Proctoring services say exam videos and other data are securely stored. Some store videos in a certified data center server, and then archive them after a defined period of time in line with Family Educational Rights and Privacy Act (FERPA) guidelines.
According to a 2017 study, it is suggested instructors familiarize themselves with how the services work so they can anticipate students’ concerns. Instructors should identify students’ technical difficulties and try to address them by spending time familiarizing students with how to get ready for and ultimately take their exams. In this pandemic, we know many students lack access to computers and wifi, and the newly issued Chromebooks challenge students to operate another new device and establish wifi access.
Online testing may seem to make things easier but it’s possible the transition to new technology, or the lack of access using current technology that doesn’t include a webcam, may complicate matters and lead to a significant level of discomfort with online proctoring. A survey of 748 students about technology and achievement gaps found about one in five struggled to use the technology at their disposal because of issues such as broken hardware and connectivity problems. Students of color or lower socioeconomic status encountered these difficulties more often.
My colleague, Aloha Sargent, Technology Services Librarian, shared with me an article from Hybrid Pedagogy that asserts "algorithmic test proctoring’s settings have discriminatory consequences across multiple identities and serious privacy implications." When Texas Tech rolled out online proctoring, they recognized students often take exams in their dorm or bedrooms, and students noted in a campus survey “They thought it was big brother invading their computers.” Some test takers were asked by live proctors to remove pictures from their surroundings and some students of color were told to shine more light on themselves. That’s a disturbing request in my opinion. Many of our community college students occupy multi-family or multi-person residences that include children. These proctoring settings will "disproportionately impact women who typically take on the majority of childcare, breast feeding, lactation, and care-taking roles for their family. Students who are parents may not be able to afford childcare, be able to leave the house, or set aside quiet, uninterrupted blocks of time to take a test."
At the University of California, Davis, they are discouraging faculty members from using online proctoring this semester unless they have previous experience with such services. “It suggests faculty consider alternatives that will lower students' anxiety levels during an already stressful time, such as requiring them to reflect on what they learned in the course.” The following article highlights a University of Washington story about adopting Proctorio because of the COVID-19 rapid transition to online. Read the experience of one University of Washington student, Paranoia about cheating is making online education terrible for everyone. The students’ experiences “are another sign that, amid the pandemic, the hurried move to re-create in-person classes online has been far from smooth, especially when it comes to testing.” Live online proctoring is a way to preemptively communicate to students, we don't trust you. It is a pedagogy of punishment and exclusion.
In higher education, traditional exams represent the most appropriate assessment tool. There are ways to cheat on exams no matter what method is used to deploy them. Even a major “NSA-style” proctoring software is not “cheat-proof.” Their sales representative was very candid in showing me how it’s done. There are alternatives to typical exam questions—often referred to as authentic assessment. According to Oxford Research Encyclopedia, “authentic assessment is an effective measure of intellectual achievement or ability because it requires students to demonstrate their deep understanding, higher-order thinking, and complex problem solving through the performance of exemplary tasks.”
Given the limited timeframe, there will be limits to what you can use now. That’s OK. Consider using Canvas question pools and randomizing questions, or even different versions of the final. For example, replacing six multiple-choice or true-and-false questions with two short-answer items may better indicate how well a question differentiates between students who know the subject matter and those who do not. Or ask students to record a brief spoken-word explanation for the question using the Canvas media tool. Just keep in mind, there are a dozen or more ways to assess learning without “biometric-lockdown-retinal scan-saliva-sample-genetic-mapping-fingerprint-analysis.”
Woldeab, Daniel, et al. “Under the Watchful Eye of Online Proctoring.” Innovative Learning and Teaching: Experiments Across the Disciplines, University of Minnesota Libraries Publishing Services’ open book and open textbook initiative, 2017.
The Learning Never Stops: Preparing to Teach in an Emergency
The Learning Never Stops: Preparing to Teach in an Emergency
Emergency power shut-offs. Fire evacuations. Pandemic quarantines.
The list of emergencies that could require you to temporarily move your course online seems to be growing every day. CVC-OEI provides a robust set of instructional resources for Canvas and online course design that can aid faculty, administrators, and instructional support staff with the transition of on-campus courses into the distance education modality.
Don't wait for disaster to strike - check out these resources and prepare now.
CVC-OEI Emergency Preparedness Resources: Planning for uninterrupted instruction and student support in the event on-campus courses need to temporarily move online.
Course Design Rubric: The CVC-OEI Course Design Rubric can be used as a checklist for instructors who are moving instruction from the classroom to the Canvas common course management system. The Rubric guides instructors on setting up a course with clear navigation, regular effective contact, effective assessments, and accessibility compliance.
Canvas Support Portal: This one-stop resource includes links to the CCC Canvas Home Base, a Canvas Community where users can access text and video user guides and post questions to the Canvas Community.
Byte Sized Canvas Videos: These short, desktop video tutorials provide pedagogical and technological tips on the use of Canvas.
Student to Student Interactions This Pocket PD Guide shows instructors how to meet the student-student interaction requirement that is now part of the Title 5 Education Code for California Community College Distance Education courses (Instructor Contact, Section 55204).
Adoptable Courses: Colleges needing to quickly prepare faculty to teach online in the case of an emergency can download courses from CVC-OEI/@ONE and adapt them to use locally. Fourteen “CVC-OEI Adoptable” courses can be found in Canvas Commons, including Introduction to Online Teaching & Learning, Introduction to Course Design, and Introduction to Designing for Accessibility.
When We Empower Students to Become Experts
Join Chelsea on a tour of this assignment in the 4-minute video above.
How might you blend research, group work, video creation, and friends and family into an empowering and equitable learning experience for your students? In the 4-minute video below, Chelsea Cohen from Laney College, will show you!
Chelsea’s students, who are English for Speakers of Other Languages (ESOL) learners, engage in a multi-stepped project, beautifully scaffolded into managing meaningful chunks. Each step of the way, students collaborate and increase their knowledge of a particular topic. Chelsea will demonstrate how extending discussions beyond the classroom or Canvas and into a students’ circle of family and friends can foster more diverse dialogue that situates a student as an expert. Can learning get more meaningful than this?
Share and discuss your video with friends and family (Extension: share the videos with your Twitter communities).
Reflect upon the experience with your classmates in our class discussion. Summarize the ideas that came up with your friends and families and how it felt for you to facilitate the conversation.
Accessibility tips! If you have a student in your class that uses a screen reader to navigate the web, you will need to provide an alternative to Adobe Spark Video. Also, if you have a student with a hearing impairment, have at least a few students caption their videos before sharing them with the class. To caption an Adobe Spark Video, download it from Spark, upload it into YouTube, and edit the auto-captions.
We suggest surveying your students in week one to let them know about your multimedia project plans and ask if they will need any accommodations. They'll appreciate your efforts to support them!
Learning from Students Who Use #EdTech
In November, a group of five college students representing the California Community Colleges and California State University systems participated in a virtual panel at the annual Directors of Educational Technology in California Higher Education (DET/CHE) conference. Projected on a screen in front of hundreds of educators, students shared their candid reflections and experiences with technology in teaching and learning.
I had the honor of moderating the panel with support from J.P. Bayard, Director for System-Wide Learning Technologies and Program Services at the CSU Chancellor's Office. As always, listening to student experiences inspired me and reconnected me with the reasons I do what I do. As technology plays a more expansive role in teaching and learning, we must make efforts to center what we do around the real experiences of the humans at the other end of the screen. I also find myself reflecting on the courage it took these students to volunteer to participate and be candid about their experiences. And that is also something all of us can learn from.
I hope you listen to the 30-minute recording and let the students' messages inform your practices as you start the new term ahead. Leave us a comment below and share a takeaway -- we'd love to hear from you!
https://youtu.be/tjEf6SDtvqk
30-Minute Archive of a student panel from the 2018 DET/CHE Conference.
Quick Links
Don't have 30 minutes to listen? Here are the 5 questions the students were asked and a video quick link to their responses.
3:28 Reflect on your experiences as a college student and answer this question, "I wish my teachers would _____________. "
8:08 If your instructor gave you the choice for a test to: write a paper, create a video, or create a verbal presentation, how would you feel about having that choice? Which option would you choose and why?
18:03 How are you using mobile devices to access the resources and services provided by your instructors and college?
20:25 Why did you choose to take an online class? What makes an online class a good learning experience?"
How to Handle That Pesky Little Dash in the Canvas Gradebook
Ever been confused by all the options and symbols in the Canvas Gradebook? Well, you’re not alone! In this episode, we’ll zero in on the little dash that shows up in ungraded assignments. Knowing how to deal with it properly will mean you’re headache-free when it’s time to turn in your grades at semester’s end.
Empower Me! An Online Student’s Perspective
Empowering students is a critical part of education and there is room for improvement in the United States. Students need self-worth, motivation, determination, and persistence to thrive in a course, and often times those traits come out of student empowerment. Giving students the opportunity to shape their education, develop their lessons, and apply it to their own path is essential because it encourages critical thinking, and gives lessons that can be applied outside of the textbook and the classroom, and eventually applied to their career. Being a passionate student about this topic, I have much to share and a few suggestions too.
In 2017, I graduated from College of the Canyons (COC), a California Community College, with associates degrees in Mathematics, Computer Science, and Physics. After that, I transferred to Cal Polytechnic, San Luis Obispo where I am now majoring in Computer Science and minoring in Entrepreneurship. I have been taking online courses since I was in high school and I was also employed at COC as a member of their Open Educational Resources (OER) program. From these experiences, I have gained a great deal of insight about how to engage and empower students in the online environment.
When students feel valued in a class, they will see it as a meaningful experience. But where do you start with this lofty goal? Start by reflecting on a few questions about your class.
“After the class is over, what can students share with future employers?”
“How might this class affect students’ career paths?”
“Am I making assignments that nurture the learning abilities of the students?”
If you struggle to answer these questions, ask your students. They love to give input and opinions, and just want to feel important to the class. Asking students for input is a great way to empower them!
9 Tips for Empowering Your Online Students
Here are some additional suggestions.
Have your online students:
Create their own prompt for an assignment
Provide a rubric you will use for grading, and give students the freedom to craft an assignment around the rubric after you have approved the plan.
Create their own test questions
This is a great way to see if students understand the material and get an idea of what they think is important in the class.
Give input on the course at the beginning and the end.
Asking for student feedback immediately is a great way to establish a trust and convey that you are here to support your learners. Some suggestions for week one include: What do you expect from this class? What do you hope to learn? .
Teach some of the lessons in the class (using Canvas Groups )
One of the best classes I took had groups of four students teach the class every day for the last half of the quarter. This was great because each group had to know their material well to teach it.
Find resources for the class
Students can find great resources, and having additional relevant resources is always a good idea, especially for students who struggle.
Run a socratic seminar
When students facilitate discussions about topics they are more engaged. Act as the guide of the conversation and requires each student to speak at least once.
Seek out scholarships, competitions, grants, and more.
For every topic in school, there is always a way to get students more involved. Encouraging them to apply for an opportunity and let them know you believe in them! . If they are successful, they will have something to show off for a lifetime.
Help write materials for future courses
When students know their assignments are not ‘throw away’, they are usually more willing to invest time in the topic and deliver something that will last. Plus, they may be able to deliver it in a way future students will understand a bit better
Connect with their community
Local companies are always looking for help. Connecting students with local businesses gives students a chance to understand a career and be considered for a potential employment opportunity.
How do you empowering your online students? Leave a comment below so we can keep this list going!
The One Thing You’ve Been Missing to Keep Students Focused on Your Content
Many students are brand new to Canvas or, even if they’ve been using it awhile, just don’t know it very well. You can decrease their floundering and frustration (and make your life easier) by giving students some basic guidance as part of your course design. In this episode, we look at four simple things you can do to keep students from getting sidelined by inexperience with Canvas.
BONUS: I created a "Canvas Tips for Students" cheatsheet you can share with your students. It's currently in a barebones, accessible Word format--I suggest you download and customize it for your class and then add it to a page in your course.
Visually-Oriented Discussions with Consistent Instructions
Discussions are an important part of fostering student-student interactions in an online course. After completing the OEI Online Course Review process, Shannon Van Zant, who teaches History at Columbia College, recognized the importance of designing her discussions with clear, consistent instructions. Having the same instructions available to students each week scaffolds their learning and supports them to become confident and independent learners. Shannon, also incorporates images into the prompts of her discussion topics. By integrating images into the textual content of her discussion, she more fully engages the way people learn.
In the 7-minute video below, Shannon shares her experiences with the OEI Online Course Review process and takes us on a tour of her online visual discussions.
