Time for a Change: Authentic Assessment in STEM

""
Photo by Elena Mozhvilo on Unsplash

Leaving Exams in 2019

During the summer of 2020, as the pandemic made it obvious we weren’t going back to “normal life” any time soon, my curriculum needed to reflect the massive changes that were happening in our society. Using the traditional STEM assessment style of short answer or multiple choice exams would not function well within this online learning environment. I could continue to give traditional exams, but would they actually be a valid measure of student knowledge? And, more broadly, would these exams serve my students in helping them to develop skills necessary to be successful beyond my class? 

Additionally, if I were to continue to use traditional assessments I would need to employ an online proctoring tool. This tool would help me maintain academic integrity, to an extent. However, these proctoring tools have significant implications for student equity. Knowing this, I could not, in good conscience, use one. 

In 2019, I made the decision to leave my exams and not use an online proctoring tool. But this left me in a tough spot. How would I measure a student's knowledge without using exams? 

Switching Assessment Styles

As an undergraduate STEM student and a STEM instructor, exams are the only type of assessment I have ever known. As I made this change, I began to realize that it would be  important to switch to assessments that enabled my students to demonstrate their knowledge and develop new skills that could be used in other classes, as well as in life beyond higher education. With access to an abundance of human knowledge at our fingertips via the internet, the ability to research, synthesize, and communicate ideas is of more value to my students’ future than memorizing all the steps of photosynthesis for an exam. With this shift, I was able to move from assessing rote memorization to critical thinking skills – isn’t that what we all should be striving to do? I was also able to connect abstract concepts to current events or students’ daily lives, making them more meaningful and memorable. 

Through this process, I developed a set of projects that draw on the principles of authentic assessments to assess student learning. I provide the basic structure of what needs to be included in the project so I can assess my students’ comprehension of the concepts, but the format of the project is generally open-ended, and multimedia projects are encouraged. 

One example is a role-play scenario where students step into the role of interns for a state government committee on health and human safety. Their goal is to brief the state representative for whom they work about the rise of antibiotic resistant bacteria. This project was inspired by the Performance Assessment Resource Bank. In the brief, students must include the following:

Within this one project, I was able to assess students’ comprehension of several learning goals: their ability to distinguish between prokaryotic and eukaryotic cells, as well as natural selection and evolution. This project also required students to demonstrate their ability to apply their knowledge of the evolutionary process to evaluate large-scale solutions to combat this issue. The form of the final product was entirely up to the students. One student, who was studying digital marketing, built a website. Another held a mock webinar. Some typed their project into a traditional research essay. Even though their final projects took many forms, grading and assessing their work was not as challenging as I expected because I provided a clearly defined rubric

Here is a 3-minute video explanation I provide for my students about this project:

Reactions to a New Assessment Style in a STEM Course

During the week leading up to the start of the semester as students were exploring our syllabus and Canvas course, I had several inquiries about exams. Students asked,  “When are the exams?” and  “Will we need to use [proctoring service] to take exams in this class?” After fielding several variations of these questions, I explicitly explained to my students my philosophy for adopting this new assessment strategy and why we would not have any exams. The idea of being able to show their knowledge outside of an exam in a science class was, at first, mysterious to students. However, they quickly acclimated to this new style of assessment as I promptly answered their questions.

Student feedback about this new assessment strategy was very positive. In an anonymous course evaluation, 97% of students rated the class as “always or almost always having assessments that are related to course material.” In another metric, 100% of students rated the class as “always or almost always having activities and projects which are useful for learning and understanding.” Students reported the projects as “fun and interesting” and said they “helped [to]… understand this subject better.” One student stated these projects helped them “gain a better understanding of the topic when applying it to real life,” which was my intent when making this shift.

In making this change to my assessments, I was met with some skepticism and backlash from colleagues, which resulted in me being reluctant to speak out about equity and assessments in online learning. When I did speak out, I received push back from colleagues saying “Students will have to get used to exams,” as well as, “There’s just no other way to assess learning in my class” except through exams. I even had a colleague claim I was calling anyone who used proctoring tools and exams “racist.” I see now that this reaction is tied to a larger, systemic issue about power and privilege in White dominant culture but I also know it made me hesitate to discuss the topic of assessments and proctoring tools again. 

In Fall of 2020 I was due to be evaluated, and as a part-time faculty member I was incredibly nervous that this different assessment style would be seen as inferior, and thus my employment status and income would be impacted as a result. Luckily, despite the backlash I had received, I had many other colleagues, including my evaluators, who were curious and encouraged by these efforts to adopt different assessment styles. Out of this discussion about assessment and proctoring tools that was met with backlash, I was able to open a conversation about rethinking how we assess learning in STEM. Yes, this is currently an uncommon way to approach assessment for many STEM classes, and can be a challenging pivot to make. But, if we’re truly dedicated to closing opportunity gaps then we must make STEM courses more equitable for diverse learners. 

References:

Van Meter, F. (2020, Sept 11). “Online Proctoring - Impact on Student Equity.Online Network of Educators.

Authentic Assessment PocketPD Guide. (2020, June 17). Online Network of Educators 

Brookhart, S. M. (2018). Appropriate criteria: Key to effective rubrics. Frontiers in Education, (3)10. doi:https://www.frontiersin.org/article/10.3389/feduc.2018.00022 . 

Dialing Up the Quality & Inclusivity of Live Online Classes

I already knew long lecture videos are a bad way to fill an hour of synchronous instruction, but this course showed very specifically how to break things up into manageable segments of interaction, instruction, and active *participatory* learning regardless of camera-on status!
– Cynthia Hamlett, Distance Education Faculty Lead, Crafton Hills College

For decades, asynchronous online courses have been the gold standard for increasing access to higher education for students who have traditionally been left out. Prior to COVID, 28% of California Community College's 2.2 million students took at least one online course (CCC Chancellor's Office, 2017). But we know that access does not equal quality. In fact, known equity gaps in face-to-face courses have been exacerbated by online courses. To improve this problem, a system and institutional commitment to providing high-quality professional development in online course design and teaching is paramount.

Every teaching modality presents both opportunities and challenges for faculty and students. When COVID surfaced one year ago, faculty across the nation scanned their digital toolkits and recognized that videoconferencing could provide an option for a digital classroom environment. As classrooms were shutdown, Zooming became part of teaching vernacular and the inequities that synchronous online instruction created for students quickly began to be recognized. Karen Costa's powerful piece, Cameras Be Damned, opened our eyes to how enabling a webcam can be a trauma trigger for students, as well as faculty. Megan Corieri, a California community college student, shared this advice with faculty in Spring 2020 article, "There have been many times that I have had to shut my camera off in class, have a good cry, and come back. We are living through unprecedented times, and now more than ever it is important to have empathy, kindness, and patience." While video conference serves as a helpful scaffold for many, it comes with a high price for others. Zoom can place an extra tax on one's already strained mental health, not to mention the inequities associated with device ownership and network access. Today, community college educators recognize the need to support the whole student more than ever.

New Course: Introduction to Live Online Teaching & Learning

an tablet showing the homepage of Intro to Live Online Teaching & Learning

This is precisely why professional development is critical to guide faculty through the twists and turns of our dynamic instructional landscape. This month, in response to this emerging need, CVC/@ONE introduced its newest professional development course, Introduction to Live Online Teaching & Learning, a 2-week, facilitated course available to California Community College faculty and staff at the low cost of $45 with the option to receive one continuing education unit for an additional fee.

In the course, participants are immersed in the role of a student as they experience instruction that models the effective use of Canvas and Zoom to support learning at a distance. Participants work independently and in small group Zoom "huddles" to learn how to design and teach effective, inclusive live sessions with an ice breaker to check the emotional pulse of students or help them make connections with peers, direct instruction using screensharing, and active learning techniques using chat, polling, breakout rooms, and external tools like Google Docs, Slides, and Jamboard. Participants also engage in live class sessions in the role of "students" that model inclusive instructional practices. In one session, they participate in a lesson in empathy that requires them to complete a collaborative activity in a Zoom breakout room while attending from a smartphone. In the United States, Black and Hispanic adults are more likely to be smartphone dependent. When teaching at an open-access institution, teaching practices must be mobile-friendly to avoid perpetuating equity gaps.

Upon completion of the course, participants receive a digital badge that verifies their new skills. The two sections we scheduled for the spring are full ... but don't worry. We'll be offering more sections of the course this summer!

Adopt the Course!

Like all of CVC/@ONE's other professional development courses, we have shared our new Intro to Live Online Teaching & Learning course in the Canvas Commons with a CC-BY license, making it simple for your college to adopt, adapt, and offer the course locally for your faculty. To ensure the quality of your own local PD, we strongly advise that the local facilitator of an adopted course complete the course with us first. In your re-use of the course materials, please attribute California Community Colleges | California Virtual College. Sharing really is caring!

To adopt the new Live Online Teaching & Learning Course, go to the Canvas Commons and search for "CVC Adoptable." That will lead you to all of our adoptable courses.

If there's one thing we've learned from COVID, it is that effective, inclusive design and teaching are central to serving the needs of all students ... regardless of the modality of instruction.

We'd like to extend a special thank you to Francine Van Meter of Cabrillo College who contributed her expertise to the development of this new course. Francine will be retiring from our system in May and has left a tremendous legacy in distance education. You will be missed, Francine!

Conducting an Online Course Cultural Curriculum Audit: Steps Toward Student Equity and Success

What is a Cultural Curriculum Audit?

Long Beach City College (LBCC) facilitates numerous student equity initiatives for basic needs, student services, and access, just to name a few. Academically, however, we were not student-ready based on multiple measures. Namely, in Spring 2018, LBCC was ranked 113th out of 114 California Community Colleges for course-level student success. Our students of color experienced the greatest obligation gaps in terms of course-level success, retention, and transfer; these gaps were exacerbated in our online classes. 

Clearly a change was in order.

In response, LBCC began a collective effort between Student Equity, Curriculum, Faculty Professional Development, Academic Senate, Administration, Institutional Effectiveness, and Guided Pathways. Our Vice President of Academic Affairs has also been involved since the beginning, and continues to support the faculty-led project. We developed a training program that would invite full-time and part-time faculty to redesign their courses for student equity and success. This includes an examination of course-level student success data, revisions of syllabi, reviewing the Course Outline of Record through an equity lens, and creating culturally relevant curricula.

By Summer 2019, 30 faculty gathered on campus for the inaugural Cultural Curriculum Audit (CCA). This was a three-day intensive workshop that included guest speakers, presentations, discussions, and a 100+ page workbook. The program was so successful, it was repeated in Winter 2020 with a larger cohort.

LBCC faculty seated at round desks, participating in a check-in session.

With the pandemic-induced campus closure in March 2020, we modified our CCA so as to not lose momentum. We devoted Spring 2020 to building a robust online program for equitable online teaching. In Summer 2020, we offered a three-week audit using a blended model of synchronous Zoom meetings and asynchronous Canvas learning. Feedback was so positive that we facilitated the fourth iteration of the CCA in Winter 2021, which welcomed faculty leaders from other California Community Colleges and Long Beach Unified School District, our K-12 partner. 

What Do We Do in the Audit?

The three-week Online Course CCA is a pragmatic experience. We merge elements of the CVC-OEI Course Design Rubric with our dynamic set of equity precepts (e.g. Welcome our students, build Partnerships in our classes, Demystify college processes, etc.). Content draws from the growing body of research, literature, and presentations specific to student equity and online teaching. 

Each week has two 90-minute Zoom meetings featuring peer presentations, discussions, and breakout activities. Our asynchronous content includes Pages with multimedia presentations, Discussions, Assignments, and feedback surveys. The CCA is divided into three broad modules of teaching:

Module 1: Student Equity in the Online Context

This first week includes an overview of student equity, confidential distribution of course-level success data to participants, and the application of welcoming practices to our online classrooms. The three main content areas include:

  1. Student Equity Online: An overview of equity, distinction of equity from equality, student success data, and recent survey results that inform the student experience at LBCC.
  2. Equity Precepts and Protocols: The introduction of equity-minded course redesign practices, and how they can be applied to our syllabi, Canvas Home Pages, and other parts of our classes.
  3. The Welcoming Online Classroom: A walkthrough of practices that humanize our web classrooms, and concrete steps we can take to make students feel more welcome in the online learning space.

The week 1 assignment tasks participants to create an equity-minded class orientation module which employs the protocols in part 2 above. And finally, the discussion forum has participants peruse a mock Canvas course and apply a campus space review that we have adopted from the USC Center for Urban Education and modified for online courses. 

Module 2: Equitable Content and Communication

Week 2 features explorations of short and long-term steps toward student equity online. The three content areas are as follows:

  1. Active Online Learning: We present a curated list of 85 Online Active Learning Strategies, emphasizing that such activities engage our students more effectively than traditional summative assessments. 
  2. Effective and Intrusive Communication: Drawing from Dr. Frank Harris III and Dr. Luke Wood, we blend principles from the CVC-OEI Rubric Part B: Interaction, with the practice of intrusive communication--a proactive approach to reach our students.
  3. Culturally Relevant Curriculum: We encourage a critical examination of the Course Outline of Record in addition to rethinking class content, examples, images, language use, and others to be more culturally relevant. 

The week 2 assignment asks participants to respond to a mock email from a struggling student, while identifying their application of principles above. The discussion forum requires participants to locate an accessible and culturally relevant video clip, embed it in the forum, and describe how it fits in their class.

Module 3: Equitable Assessment and Accessibility

The final week explores our assignments, assessments, and accessibility, leaving participants with some next steps once the CCA has concluded.

  1. Equitable Assignments/Assessments: Participants take self-assessments that inform their teaching style, read literature that speaks to this style (e.g. Becoming the Warm Demander), and apply the Transparent Assignment Template to their online classes.
  2. Accessibility and Universal Design: Drawing heavily from the CVC-OEI Rubric, Part D, we affirm that Universal Design is good design. Participants learn about the Ally Accessibility Checker as well as LBCC-specific services for serving our students with disabilities.
  3. Moving forward: A summation of practices for additional implementation and reflection, as well as references to encourage continued learning.

The last assignment has participants read, “The Wise Feedback Model,” and write a self-reflective piece on how they employ feedback in online classes. Our final discussion forum is a space for participants to share some next steps for redesigning their online courses for student equity based on what they have learned.

Final Module: Deliverables

Once the Online CCA is complete, participants have additional time to put their skills and practices to work in their online courses. In order to receive a stipend for the audit, we ask them to submit the following deliverables as evidence of their applications:

  1. Equitized Syllabus: A revised course syllabus with substantive changes to some or all of the following aspects of the course: course content (readings, topics, etc), classroom activities, assignments, grading structure, language, class policies.
  2. Welcoming HomePage: A “front page” in your Canvas course that students see as soon as they log in, which will contain redesigned formatting, content, and images.
  3. Canvas Content Page: A new or updated “Page” in Canvas with changes to formatting, content, images, hyperlinks, etc. This page may be: Learning content, Course Success/Resource Page, or Weekly Introductions.
  4. Transparent Assignments: One or more new or revised assignments using the transparent assignment template.
  5. Active Learning Activities: An explanation of two or more newly designed active learning activities (synchronous or asynchronous)
  6. Highlights Powerpoint: A short powerpoint presentation highlighting the changes you made to your class.
    1. Sharing with the Campus: Participants should be willing to share their curriculum audit work at other venues such as the Curriculum Committee, Academic Senate, or Flex events.

Outcomes and Next Steps

Our CCA Leadership Team solicits participant feedback to continue improving the program. In addition, we monitor the initial cohort’s course-level student success data over time. Early findings show a 9% course-level success rate increase for LBCC Black, LatinX, and Pacific Islander students. However, equity gaps still persist, so we endeavor to revise, innovate, and train.​

We will continue offering the CCA. Many of our participants have expressed a desire for an “Audit 2.0” so they can continue this type of professional development. While there are many things that need to happen to make the CCA successful, there are two priorities that are paramount:

  1. Build a strong and cohesive leadership team. Every audit has featured at least two Lead Facilitators with an advisory group to support from start to finish. 
  2. Equity work is dynamic, and a readiness to adjust our methods, check ourselves, own our mistakes and work to ameliorate them is essential. 

Lastly, we hope to invite faculty leaders from other colleges to join us in this effort, so they can create comparable programs for their institutions. You are welcome to complete our interest form here to learn more: Long Beach City College Cultural Curriculum Audit Interest Form