As promised: Reflections on my teaching evaluations

First of all, I want to express a sincere thanks to all of the students that took the time to complete the teaching evaluations for my courses. I very much appreciate the feedback that I get from students, and I genuinely value that students in my courses are helping me become a better teacher.

Click to enlarge

I've presented the quantitative results from my course evaluations above, and to provide some context I've also presented previous evals for the same courses when I have taught them before. If you squint, here is what you might see:

1. My evals are improving! Yay! On my job performance review, which the University of Toronto calls "Progress Through the Ranks" (not at all intimidating) my department indicated they want to see my evals improve, but I'm pretty pleased with the progress I've made since coming to U of T.
2. Students don't love my projects, assignments and exams. This one is tough, because I work really hard on my assignments but I'm also constrained on the amount of support I get to assess coursework. However, I can see that in Psy372 my projects, assignments and exams are rated very well. Taking my cue from this course, I am trying something new in Psy270 this year, incorporating several reflective writing assignments throughout the semester instead of one big assignment. It will provide more opportunity to practice writing, and it takes the pressure off of one big writing assignment that doesn't make room for improvement. In Psy280, I will also include a scaffolding assignment this year. This is a smaller, lower-weighted assignment that will prepare students for the essay they write near the end of the semester. I will also continue with the in-class worksheets that I started using last semester. The comments from the evals indicated students really liked them... However, I'm not sure students see these as projects or assignments. I think they are, but it's tough to know how students are reading the course eval questions. Thoughts?
3. I am enthusiastic for the material I teach. Totes bro! I love the courses I have been assigned, all for different reasons, but I really am very lucky to have found this career. When I was growing up, I liked sharing facts. "Did you know...." was always coming out of my mouth. It is serendipity that I've turned this into a paying gig!
4. Something went slightly awry in winter 2014 section of Psy270 (when you compare it with the fall section).

From the comments across my courses, I see that I talk too fast. I eternally endeavor to  slow down my rate of speech, both in life and in lectures. I think I will ask for help from students by telling my classes that asking questions slows me down, and simply pointing out that I'm talking really fast is also helpful.

I revised the slides in all of my courses this year, and the feedback was mixed. I had some feedback last year regarding my slides, suggesting they were not effective. I spent part of the summer investigating best practices for slide design based on empirical evidence from the cognitive and education literatures (c.f. Kalyuga, Chandler & Sweller, 1999; Mayer, 2008; Tangen, Constable, Durrant, Teeter, Beston & Kim, 2011). Some students loved the slides, other students hated them. In particular, students who disliked the slides felt like the lack of text meant they had to write too many notes in class. However, research has consistently shown that students who take notes in class perform better on assessments (Knight & McKelvie, 1985; Peper, 1986). As such, this is one of those areas where I am satisfied to take slight lower ratings in order to increase student learning. However, I think I can do a better job of explaining to students what I'm doing with the slides. I can also gentle indicate that my approach to slides is not something I am going to change just to appease some students' preferences.

There is a theme that students, especially in Psy270 don't think my exams are fair. I actually disagree. I use a lot of objective metrics to evaluate my exams, and I think they are fair. I will be writing a dedicated blog post on this in the future (closer to midterms), so I will come back to explaining those metrics then. However, one again I think I can do a better job of explaining that to students. I have be re-writing my lectures for Psy270, and that includes defining what critical thinking is, and then pointing out to students where it appears in lecture and on the exams. In using the reflective writing assignments, I have freed up some time for the TAs so that I can re-introduce a short-answer section on the exams. I think explaining that I take a lot of time and care in creating my exams, coupled with some opportunity for students to be creative on exams, will help with this.

But the winter section of Psy270 went awry for other reasons. I think there were two factors: (1) the room was terrible - it was hot, loud, and the seats were horribly uncomfortable, and (2) I put lecture recordings on line, and more than half the class stopped coming to lectures. I have indicated to room bookings that I will not take the same lecture theater again, and I'm not going to put up lecture recordings. I'm also going to use participation points to motivate students to come to lecture, so this should be less of a problem. It was clear from the comments that students who used the recordings as a substitute for coming to lecture had difficulty covering the material on their own. This also showed up in the course grade distribution - it was hugely bimodal. One group had a mean in the high 70s, while the other group had a mean in the mid-60s.

Finally, the comments and ratings show that most students recognize my enthusiasm for the material, my enthusiasm for teaching, and the fact that I genuinely care about my students in terms of both their success and well-being. The positive comments far outweighed the negative or critical comments I received, and that is really meaningful to me.

My only serious concern about my teaching evaluations is the level of participation. In all of my courses, the average evaluation completion rate is around 50%. That means that half of my students aren't telling me what they liked and didn't like about my courses, and there is a serious possibility that the above themes are obscuring the the components of my courses that could really use improving. I am going to try some things this year to encourage participation, and I would really love it if I could get the completion rate closer to 80%. That's a huge difference, but I'm up for the challenge!

how would you rate me as a person?

I've had students ask me if profs really read their teaching evaluations. The truth is, it depends, but yes, some profs don't read their teaching evaluations. As my friend Dr. Don Boyes puts it,

"I can only speculate as to why someone would not want to read them: either they are so confident in their teaching abilities that they think feedback is unnecessary, or they are worried about what they will read because they are not at all confident in their abilities."

For the instructors who are not at all confident in their abilities, I think at least half of that has to do with how cruel a small proportion of students can be. When I was at STLHE recently, I had a conversation with several educators who all commiserated in how hard it is to read student comments, "I wait a few weeks to brace myself", "I wait until I've had a really great day, and then open a bottle of wine", "I still haven't done it". It's understandable.

I get a lot of truly great comments from students. They can be really positive. Some are critical. The vast majority of comments are written with consideration and kindness, whether they're suggesting improvements or expressing approval. But there are always a few comments that really sting. These comments aren't about my teaching or how to improve the course, but are a personal critique like "She's not a good role model for women" or "She's ruining the university's reputation" (seriously, I've had both of those). They are always like a punch in the gut.

But teaching evaluations are important, and I do read them – every frequency distribution, every mean and mode, and every comment. They help me identify what students need more of, and they can give me a really clear idea of how to delivery on those needs. They also help me figure out what to keep doing. Sometimes I try something like worksheets, and it's hard to tell if students think they're helpful or if they think they're silly, but I will definitely figure that out once I get the teaching evaluations back.

So I read every comment – the good, the bad and the ugly. I try very hard to keep the personal attacks for taking up too much of my time or attention because it's the comments from students that make an effort to tell me something sincere and useful that really deserve my attention. I want to pay tribute to those students who have taken time out of their "spare time" during the last weeks of a semester to help me improve myself... what a kind and generous gift.

This is all a preamble to my next blog post in which I will reflect on my teaching evaluations from the 2013/2014 academic year in anticipation of the new semester. I'm really looking forward to sharing my insights from those evals, and putting those insights into action in September!

It's less about content, more about process

I'm back today from the Society for Teaching & Learning in Higher Education (STLHE) annual meeting. Last year the event was in Cape Breton, but this year the conference was in the neighborhood in Kingston, Ontario. FYI - Kingston is beautiful! I want to go back for a visit with my husband to actually see more of the attractions. My impression, though, is that Kingston is small (as my friend put it, "Rush hour is from 4-4:15"... perfect), walkable, and has the most gorgeous architecture. In addition, there were so many great restaurants to choose from! As a person who lives to eat, great restaurants within walking distance are actually an important travel variable for me ;)

But obviously it was the conference that brought me to Kingston this year. I started going last year on the advice of my friend at UBC, Dr. Catherine Rawn. Catherine is a recently-tenured teaching stream lecturer at UBC, and has been a great mentor and friend to me since my days as a teaching post-doctoral fellow in Vancouver. She passed on some wise words from one of her mentors, "If you only go to one teaching conference, go to STLHE."

Fireworks at Fort Henry during the STLHE 2014 banquet

There are a lot of teaching & learning conferences to choose from, & many of them are discipline-specific. There are three psychology-specific teaching conferences that I know of. However, I have deliberately chosen to go to STLHE because it's not discipline-specific. There are so many great innovations in teaching at the university/college level across the disciplines, & my experience is that we're all trying to do the same thing: teach critical thinking skills in a way that engages students in the learning process. 

One of the themes that I see emerging from these kinds of conferences is that it's less about the content, it's more about the process. I don't expect that my students will remember the specific details for the experiments I talk about in lecture 2 months after the course has ended. If they can recall some general effects (can you remember what transfer appropriate processing is?), that would be awesome. But besides that, I want students to walk out of my courses with the confidence to ask the question, "How do you know that?" when they hear a claim on the news. Writing notes by long-hand is better for learning compared to writing notes by computer. How do we know that? Did the researchers rule out the possibility that it's because students on computers are often doing anything but taking notes? How did they explain this advantage for hand-written notes? Does that explanation need some work? Does it warrant another experiment to test the validity of the explanation? Then I want them to be motivated to go find the answers to their questions.

In one of the keynote addresses, Dr. Eric Mazur made the point that as an independent academic, no one in my discipline expects me to be able to provide an exact definition of transfer appropriate processing. I probably can, but if I forget, I can look up the answer.  I'm not being judged by my colleagues about whether I know the information, but rather about how I use it to build arguments about cognition & perception. Mazur also made the point that assessment in higher education is often at odds with this truth. It has really motivated me to work harder at introducing critical thinking exercises in class & on my methods of assessments (exams & assignments, alike). This is something I try to do anyway, but I think it will also be really helpful to make this motivation clear to students in September.

I always find conferences so inspiring. It's really wonderful to connect with other people who are so interested & dedicated to giving students a wonderful education experience. I'm incorporating a few of the ideas I gathered from the conference into my courses for the fall, & I'm looking forward to next year already :D

Introspections on introspection (whoa. meta.)

I decided to call my blog on teaching & learning in higher education "Introspection" as an homage to Wundt, Titcher, & all the other introspectionists from the early days of psychology. Introspection is the examination of one's own thoughts & mental processes, & has been around since well before Plato & Aristotle's discussions of the act.

While introspection served as one of the methods on which modern psychology was grounded, the idea of introspection as a method of studying human behavior has been poo-pooed for almost a century. This was largely due to the influence of John B. Watson, who argued that the discipline of psychology should concern itself with the prediction & control of behavior, stating that "Introspection forms no essential part of its methods, nor is the scientific value of its data dependent upon the readiness with which they lend themselves to interpretation in terms of consciousness." Ouch.

But introspection isn't in & of itself a useless endeavor. In several of my courses I include introspection-like assignments in which I ask students to engage in reflective writing around what they're learning. I ask students to examine their own learning process, & try to make a personal connection between academic concepts & methods, & their own lives. If introspection is so terrible for studying human behavior, what's the point of these assignments? It turns out that introspection – self-reflection,  bringing the self into the learning process, whatever you want to call it – is phenomenal for making information stick! Seriously guys, if you want to learn something, write about it in your own words, & try to drawn in new connections. The benefit of introspection is due to the fact that the knowledge that is being acquired is also being transformed, and in many cases, new information is being generated by the writer.

I do a lot of introspection about my own process as a teacher. I really consider every event that happens in my teaching practice, and I use those events & introspections to inform my future practice. I've also found that students tend to ask me about a lot of the decisions I've made in organizing my courses, often with the assumption that I haven't really thought about it. The reality is, I've definitely thought about it. A lot. In fact, sometimes I agonize about what seems like an small decision, like how will students submit an assignment? How many levels should I use in my marking rubrics? Cardigan or no cardigan? (ok, that last one was an easy decision - always yes to a cardy! I get cold easily. )

So, I'm going to share! I'm not going to lie to you - putting it in writing is also going to be a big help for me. Pausing to reflect on why I'm doing something has in the past made me realize this is not the way to achieve your goal, Kristie. I read a paper recently extolling the virtues of reflective writing assignments in which a student reported, "...if I don't understand it when reading, when I write it, it just so happens I do understand it for some reason." True dat :D

Just testing things out.

Imagine I had something to say here.