SWF Primer: Evaluation Factors (Part One)

[I note that this post concerns SWFs, and is therefore of interest primarily to Full-Time faculty. I’ll try to follow it up with a post on a related issue that might be of some relevance to Partial-Load faculty.]

Okay, so… let’s talk SWFs for a bit.

It’s worth introducing the topic at this point, since SWFs for the Winter semester are due six weeks before the start of the first scheduled day of teaching, not including holidays.

So, if the teaching period of the Winter semester happens to start on Monday, January 4, then the SWF would be due on… Friday, November 13, if my calendar and math are correct.

The SWF’s delivery is, according to the Collective Agreement (Art. 11.02 A1[a]), supposed to be preceded by a discussion of the workload proposed by your manager.

In addition, Article 11.01 E3 reads:

And it’s precisely the methods of evaluation that I want to review, because the shift to online teaching may impact them in ways that faculty don’t realize.

So, let’s start by covering the three categories of evaluation / feedback recognized in the Collective Agreement. In my next post on this topic, I’ll talk about how to read a SWF to understand how evaluation factors translate to specific amounts of time that faculty are given, to evaluate their students in each of their classes and provide feedback.

Let me preface with a couple of points:

  1. Full-time faculty are given a finite amount of time to complete the evaluation and feedback of the students in their classes; that time is clearly recorded on the SWF.
  2. The Collective Agreement provides minimum attributions of time for the grading of each student in each class. The attribution of time depends on the type of evaluation that is required for each class.

There are three different types of Evaluation. I’ll outline them here, and then in my next post, I’ll demonstrate how they are put into action on a SWF.

Essay Type: Article 11.02 E2 of the Collective Agreement specifies that the Essay evaluation factor is used for grading “essays; essay type assignments or tests; projects; or student performance based on behavioral (sic) assessments compiled by the teacher outside teaching hours”. Given that list, the term “Essay” type is a bit misleading.

Broadly speaking, the litmus test for determining whether a test or assignment is “essay type” depends upon the level of interpretation required. If you’re grading according to the presence/absence of keywords, or if the work could possibly be put into a format where it could be graded by a computer or a well-trained chimpanzee, then it’s probably doesn’t qualify as “Essay” type. If, on the other hand, the evaluations depend on students working with or applying abstract concepts, or writing extended passages, then it probably does.

The other point that I would make concerns the role of student presentations, which I interpret to qualify as “student performance” that is graded “based on behavioral assessments”, and that therefore count as “Essay” type evaluation if the teacher has to compile the assessment and feedback at home.

By the same token, I understand all portfolios of students’ work (including, say, student journals or any assessment representing a compilation of student work throughout the semester) to count as “Essay-type” assessment.

If a course’s evaluation/feedback is rated as 100% “Essay”-type, then a professor is attributed a minimum of 108 seconds per week, per teaching hour of the course. In other words, a prof could be granted as little as 5.4 minutes of time each week to complete the assessments for each student of a course that has three teaching hours weekly.

Routine or Assisted Type: The same article defines the second category of evaluation — “Routine or Assisted” — as “grading by the teacher outside teaching contact hours of short answer tests or other evaluative tools where mechanical marking assistance or marking assistants are provided”.

There has been some debate about which noun was modified by the “where” clause, but let’s let that slide for the moment. I’ve always interpreted this type to include the grading that could be done by Scantron or could be graded according to an answer key that could be provided to a marking assistant (e.g., “Connect the terms in Column A to the definitions in Column B”).

It’s significant that this is a kind of assessment that isn’t evaluating students’ ability to apply theoretical concepts, or demonstrate critical thinking or apply principles to their own experiences.

The minimum attribution for Routine/Assisted-type grading in the Collective Agreement is half that attributed to Essay-type grading. This translates to 54 seconds per teaching hour per week, for a course that has only Routine/Assisted-type assessments.

In-Process: The final category of assessment for the purpose of measuring workload is “in-process”, which is defined as “evaluation performed within the teaching contact hour”.

When asked for examples of “in-process”-type assessments, I often offer an example of nursing students demonstrating their ability to use a blood pressure cuff. The assessment takes place during the class time; the student knows their grade within the class period; and there is nothing about the evaluation or feedback that the teacher (i.e., professor or instructor) needs to complete at home.

Since the evaluation takes place (and feedback is provided) during the class hour — which is already counted on the SWF — in-process evaluation/feedback is attributed very little time to complete: approximately half the time attributed for Routine/Assisted grading. So if a 3-hour course were to be graded entirely using in-process grading, the minimum attribution would be 1 minute and 40 seconds for evaluation and feedback to each student, weekly.

Some points worth considering about the evaluation types:

  • In-Process assessments (those done “in real-time” during the class teaching hour) are virtually non-existent in online teaching. I suppose that if a professor were teaching a class on American Sign Language, then it might be possible to assess every student’s skill (and return their grade) within scheduled class hours, over a screen. Beyond that, I’m having a hard time imagining what in-process grading might look like in an online (and particularly an asychronous) context.
  • Changing evaluation factors is the easiest way for managers to assign more students to faculty without increasing the mathematical measurement of their workload. Faculty are attributed as much time to grade essays for 50 students as they are to grade multiple choice tests for 100 students, or to grade in-class assessments for 163 students.
  • The minimum attribution for “Essay” type evaluation is simply inadequate for providing meaningful feedback to students. Teachers are attributed only twice as much time to grade a class with all essays as they are to grade a class that only has multiple-choice tests all semester. And they’re only given 3.3x more time to grade essays at home than faculty are given to grade assessments during the class time. And, in fact, I can read and grade an essay in a couple of minutes. What I can’t do in a couple of minutes is explain the basis for that grade to the student, and give the student feedback to help them improve on future work.
  • The time attributed to faculty for evaluation/feedback is based upon the teaching hours of a class. If a class were reduced from 3 weekly teaching hours weekly down to 2, then the time attributed for evaluation would similarly be reduced by 1/3, although it’s not clear that the assessment obligations of the faculty would be reduced appropriately.

That’s it for now. Next post will be looking at how these numbers get plugged into a SWF, and how FT faculty members can determine whether their SWFs accurately reflect the time-demands presented by their classes’ actual evaluation and feedback needs.

As ever, I invite faculty — particularly contract faculty — in Ontario Colleges to share their questions, stories and opinions. You can do this by e-mailing ontariocollegeprof@yahoo.com. Confidentiality will be strictly guarded.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s