The week before

The week before lectures begin is easily my busiest week of the year. I have a kabillion things to do (that's a real value, right?).

I have all of the content for my courses that needs to be loaded onto Blackboard, the course management software that we use at UofT, but it needs to be perfect before I do. The last thing I want is to notice a typo on a document, have to edit the document, save as pdf, and re-upload to Blackboard. Instead, I hold everything back until I'm sure it's in the best condition possible, and then I try to let go of any of the little typos. Because no matter how hard you look, there are always tyops.

But committing to a decision on things like exams scheduling, assignment weight, and the overall course calendar of events is nerve-wracking. What if I accidentally schedule a term test for the same week I'm expected to present at a conference? Or the week before my final exams are due in to the Faculty of Arts & Science? Where will I find the time to create a really great final exam, when I'm preparing an answer key for the term test & training the TAs on how to use it?

My first semester at UofT, I accidentally scheduled all three of my courses to have a term test in the same week. That meant that the week before, when I was still preparing & delivering lectures, I also was creating 3 exams that needed to be to the Photocopy Centre before Friday. And the next week, I was madly writing the answer keys for the TAs just as fast as I could to make sure the TAs had them before the exams were actually done. And then I spent the rest of the week and the entire weekend writing 9 hours of new lecture material for the following week (which takes about 5-12hrs to prep per hour of lecture, depending on the topic). It was something of a nightmare. Of course, I had less than a month to prepare for 3 courses,  over 1000 students  and 14 TAs that semester, so I still marvel at how I survived that semester.

I'm really trying to make two points: (1) In writing the syllabus, there are a lot of variables to consider. If you see something on a syllabus that you think doesn't make sense or seems unfair (for example, a term test the day after Thanksgiving), try to put yourself in your instructor's shoes. Suddenly, the decision might make sense; and (2) this is why my Blackboard pages aren't available yet. I want to make the content perfect and anticipate all possible dangers before I release the Blackboard pages into the world. Please be patient ;)

Education & Reseach

I recently attended the McMaster Symposium on Education & Cognition. One of the organizers, Dr. Joe Kim, told a very interesting story. He was with colleagues who were discussing the over-haul of a medical program at another Ontario university from its current incarnation to a more evidence-based program. Everyone joked, "What were they basing their medical training on before??!"

Hahahaha...right?! Seriously, what are you teaching medical students, if not what research has to say about the practice of medicine?!

After the jokes were over, Dr. Kim asked the faculty members he was with, "How many of you use evidence-based teaching practices?" No one laughed. There was mostly an uncomfortable silence.

This very much echos my own experience. When I started teaching, it was trial-and-error. I tried to emulate the instructors that I admired when I was an undergraduate, and I threw in a few things that I thought were interesting and innovative but had no reason to believe would work. I did not consult the literature.

There is a whole world of research on best practices for instructional design in high education, but far too few university-level instructors take advantage of it. It comes naturally to most researchers to go to the literature when they have a discipline-specific question, but for some reason, it doesn't occur to us to do the same thing when we have a teaching question. My own teaching practice has evolved to rely quite heavily on relevant educational and cognitive research, and I think I can genuinely call my practice evidence-based. I also think there is a move among instructors and educational developers at universities to make educational research more prominent in professional development.

I get a few students every year who challenge me on my teaching practice. They question my slides, or my evaluation rubric, or my exam questions, etc. I appreciate that students want to understand my motivation (it's part and parcel why I started this blog). I think students should challenge more of their instructors to produce evidence to support their teaching practice. At the very least, it will keep us on our toes! And it could very well motivate more instructors to migrate to evidence-based practice.

I wrote more about my experiences at the EdCog Symposium for the Centre for Teaching Innovation & Support (CTSI) blog. If you're interested in finding out about some of the research presented and some ideas about how I might implement practical applications of the research, click here to read on.

what is critical thinking?

I was working on my course learning objectives for Psy270 Introduction to Cognition in May, and I realized that I wasn't exactly sure what critical thinking was... I mean, I couldn't explicitly define it off the top of my head. Getting students to think critically about cognition is a major learning objective for the course. More generally, I think acquiring "critical thinking skills" are a major component of the undergraduate experience. Every instructor I know, in every discipline at U of T (and beyond), wants to teach their students critical thinking skills. But would I know it when I saw it? More than that, would students?

I actually have an intuitive sense of critical thinking skills, and I work really hard at creating exams that will target critical thinking skills. Even in a multiple choice format, you can write questions that challenge students to think critically about the material. These questions are more about novel applications of the material, and they challenge students to question the validity of their implicit assumptions. And so begins the definition of critical thinking.

I started my quest with the Wikipedia page on this – which is surprisingly thorough! I continued reading various sources on critical thinking, and while there is a lot of theory and several frameworks, I found myself identifying with several important characteristics of critical thinking:

1. Critical thinking is informed by evidence. If the evidence changes, so should the conclusions. The discipline of psychology has the scientific method at it's core, training each psychology student to be a scientific thinker. So make a claim, but I want to see some data to back it up.
2. It involves reflective skepticism. Students (and teachers!) should examine their own beliefs and how they interact with the interpretation of evidence.
3. As a result, it is easier to recognize errors and biases in one's own thinking. 
4. Sometimes, just recognizing the problem can be a major hurtle... Do you really know what the question is about? You might think it's about one thing, but it's really about something else. Be on guard!
5. There might be several solutions, but critical thinking involves prioritization in problem-solving. Maybe there are both top-down and bottom-up contributions to cognition, but in a particular circumstance, which one is more critical to the outcome?
6. Logical relationships are important - does the conclusion follow from the premise? Are you sure?
7. Out of this, it's easy to see that drawing conclusions and making generalizations that are warranted is a central characteristic.
8. Similarly, there are sometimes unstated assumptions that are important for drawing warranted conclusions. Take a second to see if you're considering all unstated assumptions!!

So let's do it! Let's think critically y'all! :D

In the fall, and particularly in my Introduction to Cognition class, I am going to work really hard at helping students see when I am engaging them in critical thinking. Recognizing that these are some skills we are practicing can go a long way in developing those skills! So if you're taking Psy270 with me, be prepared to be reminded of these characteristics over and over again. And be prepared for exams that test these skills!

As promised: Reflections on my teaching evaluations

First of all, I want to express a sincere thanks to all of the students that took the time to complete the teaching evaluations for my courses. I very much appreciate the feedback that I get from students, and I genuinely value that students in my courses are helping me become a better teacher.

Click to enlarge

I've presented the quantitative results from my course evaluations above, and to provide some context I've also presented previous evals for the same courses when I have taught them before. If you squint, here is what you might see:

1. My evals are improving! Yay! On my job performance review, which the University of Toronto calls "Progress Through the Ranks" (not at all intimidating) my department indicated they want to see my evals improve, but I'm pretty pleased with the progress I've made since coming to U of T.
2. Students don't love my projects, assignments and exams. This one is tough, because I work really hard on my assignments but I'm also constrained on the amount of support I get to assess coursework. However, I can see that in Psy372 my projects, assignments and exams are rated very well. Taking my cue from this course, I am trying something new in Psy270 this year, incorporating several reflective writing assignments throughout the semester instead of one big assignment. It will provide more opportunity to practice writing, and it takes the pressure off of one big writing assignment that doesn't make room for improvement. In Psy280, I will also include a scaffolding assignment this year. This is a smaller, lower-weighted assignment that will prepare students for the essay they write near the end of the semester. I will also continue with the in-class worksheets that I started using last semester. The comments from the evals indicated students really liked them... However, I'm not sure students see these as projects or assignments. I think they are, but it's tough to know how students are reading the course eval questions. Thoughts?
3. I am enthusiastic for the material I teach. Totes bro! I love the courses I have been assigned, all for different reasons, but I really am very lucky to have found this career. When I was growing up, I liked sharing facts. "Did you know...." was always coming out of my mouth. It is serendipity that I've turned this into a paying gig!
4. Something went slightly awry in winter 2014 section of Psy270 (when you compare it with the fall section).

From the comments across my courses, I see that I talk too fast. I eternally endeavor to  slow down my rate of speech, both in life and in lectures. I think I will ask for help from students by telling my classes that asking questions slows me down, and simply pointing out that I'm talking really fast is also helpful.

I revised the slides in all of my courses this year, and the feedback was mixed. I had some feedback last year regarding my slides, suggesting they were not effective. I spent part of the summer investigating best practices for slide design based on empirical evidence from the cognitive and education literatures (c.f. Kalyuga, Chandler & Sweller, 1999; Mayer, 2008; Tangen, Constable, Durrant, Teeter, Beston & Kim, 2011). Some students loved the slides, other students hated them. In particular, students who disliked the slides felt like the lack of text meant they had to write too many notes in class. However, research has consistently shown that students who take notes in class perform better on assessments (Knight & McKelvie, 1985; Peper, 1986). As such, this is one of those areas where I am satisfied to take slight lower ratings in order to increase student learning. However, I think I can do a better job of explaining to students what I'm doing with the slides. I can also gentle indicate that my approach to slides is not something I am going to change just to appease some students' preferences.

There is a theme that students, especially in Psy270 don't think my exams are fair. I actually disagree. I use a lot of objective metrics to evaluate my exams, and I think they are fair. I will be writing a dedicated blog post on this in the future (closer to midterms), so I will come back to explaining those metrics then. However, one again I think I can do a better job of explaining that to students. I have be re-writing my lectures for Psy270, and that includes defining what critical thinking is, and then pointing out to students where it appears in lecture and on the exams. In using the reflective writing assignments, I have freed up some time for the TAs so that I can re-introduce a short-answer section on the exams. I think explaining that I take a lot of time and care in creating my exams, coupled with some opportunity for students to be creative on exams, will help with this.

But the winter section of Psy270 went awry for other reasons. I think there were two factors: (1) the room was terrible - it was hot, loud, and the seats were horribly uncomfortable, and (2) I put lecture recordings on line, and more than half the class stopped coming to lectures. I have indicated to room bookings that I will not take the same lecture theater again, and I'm not going to put up lecture recordings. I'm also going to use participation points to motivate students to come to lecture, so this should be less of a problem. It was clear from the comments that students who used the recordings as a substitute for coming to lecture had difficulty covering the material on their own. This also showed up in the course grade distribution - it was hugely bimodal. One group had a mean in the high 70s, while the other group had a mean in the mid-60s.

Finally, the comments and ratings show that most students recognize my enthusiasm for the material, my enthusiasm for teaching, and the fact that I genuinely care about my students in terms of both their success and well-being. The positive comments far outweighed the negative or critical comments I received, and that is really meaningful to me.

My only serious concern about my teaching evaluations is the level of participation. In all of my courses, the average evaluation completion rate is around 50%. That means that half of my students aren't telling me what they liked and didn't like about my courses, and there is a serious possibility that the above themes are obscuring the the components of my courses that could really use improving. I am going to try some things this year to encourage participation, and I would really love it if I could get the completion rate closer to 80%. That's a huge difference, but I'm up for the challenge!

how would you rate me as a person?

I've had students ask me if profs really read their teaching evaluations. The truth is, it depends, but yes, some profs don't read their teaching evaluations. As my friend Dr. Don Boyes puts it,

"I can only speculate as to why someone would not want to read them: either they are so confident in their teaching abilities that they think feedback is unnecessary, or they are worried about what they will read because they are not at all confident in their abilities."

For the instructors who are not at all confident in their abilities, I think at least half of that has to do with how cruel a small proportion of students can be. When I was at STLHE recently, I had a conversation with several educators who all commiserated in how hard it is to read student comments, "I wait a few weeks to brace myself", "I wait until I've had a really great day, and then open a bottle of wine", "I still haven't done it". It's understandable.

I get a lot of truly great comments from students. They can be really positive. Some are critical. The vast majority of comments are written with consideration and kindness, whether they're suggesting improvements or expressing approval. But there are always a few comments that really sting. These comments aren't about my teaching or how to improve the course, but are a personal critique like "She's not a good role model for women" or "She's ruining the university's reputation" (seriously, I've had both of those). They are always like a punch in the gut.

But teaching evaluations are important, and I do read them – every frequency distribution, every mean and mode, and every comment. They help me identify what students need more of, and they can give me a really clear idea of how to delivery on those needs. They also help me figure out what to keep doing. Sometimes I try something like worksheets, and it's hard to tell if students think they're helpful or if they think they're silly, but I will definitely figure that out once I get the teaching evaluations back.

So I read every comment – the good, the bad and the ugly. I try very hard to keep the personal attacks for taking up too much of my time or attention because it's the comments from students that make an effort to tell me something sincere and useful that really deserve my attention. I want to pay tribute to those students who have taken time out of their "spare time" during the last weeks of a semester to help me improve myself... what a kind and generous gift.

This is all a preamble to my next blog post in which I will reflect on my teaching evaluations from the 2013/2014 academic year in anticipation of the new semester. I'm really looking forward to sharing my insights from those evals, and putting those insights into action in September!

It's less about content, more about process

I'm back today from the Society for Teaching & Learning in Higher Education (STLHE) annual meeting. Last year the event was in Cape Breton, but this year the conference was in the neighborhood in Kingston, Ontario. FYI - Kingston is beautiful! I want to go back for a visit with my husband to actually see more of the attractions. My impression, though, is that Kingston is small (as my friend put it, "Rush hour is from 4-4:15"... perfect), walkable, and has the most gorgeous architecture. In addition, there were so many great restaurants to choose from! As a person who lives to eat, great restaurants within walking distance are actually an important travel variable for me ;)

But obviously it was the conference that brought me to Kingston this year. I started going last year on the advice of my friend at UBC, Dr. Catherine Rawn. Catherine is a recently-tenured teaching stream lecturer at UBC, and has been a great mentor and friend to me since my days as a teaching post-doctoral fellow in Vancouver. She passed on some wise words from one of her mentors, "If you only go to one teaching conference, go to STLHE."

Fireworks at Fort Henry during the STLHE 2014 banquet

There are a lot of teaching & learning conferences to choose from, & many of them are discipline-specific. There are three psychology-specific teaching conferences that I know of. However, I have deliberately chosen to go to STLHE because it's not discipline-specific. There are so many great innovations in teaching at the university/college level across the disciplines, & my experience is that we're all trying to do the same thing: teach critical thinking skills in a way that engages students in the learning process. 

One of the themes that I see emerging from these kinds of conferences is that it's less about the content, it's more about the process. I don't expect that my students will remember the specific details for the experiments I talk about in lecture 2 months after the course has ended. If they can recall some general effects (can you remember what transfer appropriate processing is?), that would be awesome. But besides that, I want students to walk out of my courses with the confidence to ask the question, "How do you know that?" when they hear a claim on the news. Writing notes by long-hand is better for learning compared to writing notes by computer. How do we know that? Did the researchers rule out the possibility that it's because students on computers are often doing anything but taking notes? How did they explain this advantage for hand-written notes? Does that explanation need some work? Does it warrant another experiment to test the validity of the explanation? Then I want them to be motivated to go find the answers to their questions.

In one of the keynote addresses, Dr. Eric Mazur made the point that as an independent academic, no one in my discipline expects me to be able to provide an exact definition of transfer appropriate processing. I probably can, but if I forget, I can look up the answer.  I'm not being judged by my colleagues about whether I know the information, but rather about how I use it to build arguments about cognition & perception. Mazur also made the point that assessment in higher education is often at odds with this truth. It has really motivated me to work harder at introducing critical thinking exercises in class & on my methods of assessments (exams & assignments, alike). This is something I try to do anyway, but I think it will also be really helpful to make this motivation clear to students in September.

I always find conferences so inspiring. It's really wonderful to connect with other people who are so interested & dedicated to giving students a wonderful education experience. I'm incorporating a few of the ideas I gathered from the conference into my courses for the fall, & I'm looking forward to next year already :D

Introspections on introspection (whoa. meta.)

I decided to call my blog on teaching & learning in higher education "Introspection" as an homage to Wundt, Titcher, & all the other introspectionists from the early days of psychology. Introspection is the examination of one's own thoughts & mental processes, & has been around since well before Plato & Aristotle's discussions of the act.

While introspection served as one of the methods on which modern psychology was grounded, the idea of introspection as a method of studying human behavior has been poo-pooed for almost a century. This was largely due to the influence of John B. Watson, who argued that the discipline of psychology should concern itself with the prediction & control of behavior, stating that "Introspection forms no essential part of its methods, nor is the scientific value of its data dependent upon the readiness with which they lend themselves to interpretation in terms of consciousness." Ouch.

But introspection isn't in & of itself a useless endeavor. In several of my courses I include introspection-like assignments in which I ask students to engage in reflective writing around what they're learning. I ask students to examine their own learning process, & try to make a personal connection between academic concepts & methods, & their own lives. If introspection is so terrible for studying human behavior, what's the point of these assignments? It turns out that introspection – self-reflection,  bringing the self into the learning process, whatever you want to call it – is phenomenal for making information stick! Seriously guys, if you want to learn something, write about it in your own words, & try to drawn in new connections. The benefit of introspection is due to the fact that the knowledge that is being acquired is also being transformed, and in many cases, new information is being generated by the writer.

I do a lot of introspection about my own process as a teacher. I really consider every event that happens in my teaching practice, and I use those events & introspections to inform my future practice. I've also found that students tend to ask me about a lot of the decisions I've made in organizing my courses, often with the assumption that I haven't really thought about it. The reality is, I've definitely thought about it. A lot. In fact, sometimes I agonize about what seems like an small decision, like how will students submit an assignment? How many levels should I use in my marking rubrics? Cardigan or no cardigan? (ok, that last one was an easy decision - always yes to a cardy! I get cold easily. )

So, I'm going to share! I'm not going to lie to you - putting it in writing is also going to be a big help for me. Pausing to reflect on why I'm doing something has in the past made me realize this is not the way to achieve your goal, Kristie. I read a paper recently extolling the virtues of reflective writing assignments in which a student reported, "...if I don't understand it when reading, when I write it, it just so happens I do understand it for some reason." True dat :D