Student Assessment – Page 6 – Centre for Teaching Excellence Blog

Students are just [not] like they used to be — Mary Louise McCallister (Faculty of Environment Teaching Fellow)

A few years ago when I was an undergraduate student (OK, it was the 1970s) I heard the following from another student in my class. I don’t recall if it was an eye-witness account or just a good story:

A professor was lecturing in a sonorous monotone at the front of a huge lecture hall. He was becoming increasingly frustrated by the apparent lack of interest exhibited by the meager group of students scattered throughout the room. One member of the class was slumped in a chair fast asleep right in front of the lecture podium. In an aggravated tone, the professor asked the student sitting next to his slumbering classmate to rouse his peer. Said student retorted, “YOU wake him up; you put him to sleep.”

I remember this story every time I read or hear about university instructors in North America and elsewhere worrying about how students these days are too easily distracted (often by mobile devices), seem to have poor listening skills and don’t have due regard for pursuing serious knowledge—not like when THEY were in school. Really? If so, these instructors were either much better students than I was (which might very well be true) or they were educated throughout university by star performers. I have a somewhat different perspective: while we did not have the smartphones and iPods, we did have other ways of tuning out uninspiring lecturers.

I discussed this perspective with one of my colleagues at the Centre for Teaching Excellence, Mark Morton, a renaissance kind of person, who has expertise in new educational technologies, but is also a scholar of Shakespeare, etymology, food culture, and arcane types of knowledge. It could persuasively be argued that the sentiment held by ‘learned professors’ about their students’ listening skills and lack of attention to scholarly pursuits has been going on throughout human history. As a case in point, Mark sent me the following quotation penned in the medieval ages. It’s anonymous, and found in a collection known as Carmina Burana 6, translated by George F. Whicher:

Learning that flowered in days of yore

In these our times is thought a bore.

Once knowledge was a well to drink of;

Now having fun is all men think of.

Today mere striplings grow astute

Before their beards begin to shoot –

Striplings whose truant dispositions

Are deaf to wisdom’s admonitions.

Yet it was true in ages past

No scholar paused from toil at last

Nor shrunk from studies the most weighty

Till his years numbered more than eighty.

Florebat olim studium,

nunc vertitur in tedium;

iam scire diu viguit,

sed ludere prevaluit.

iam pueris astutia

contingit ante tempora,

qui per malivolentiam

excludunt sapientiam.

sed retro actis seculis

vix licuit discipulis

tandem nonagenarium

quiescere post studium.

Students have tuned out lectures tens of decades, or centuries, before Twitter was invented—they were just possibly somewhat less obvious about it. Nevertheless, today, academics and members of the media (as noted in this BBC piece) are increasingly sounding the alarm about how the conventional form of university education—the class lecture—is threatened by a combination of technological advances, mobile devices and MOOCs (massive open on-line courses). This situation is now stimulating a conversation around university teaching and how to ensure that it is both effective and relevant; how to foster ‘deep’ vs. ‘shallow’ learning; and how much ‘content-delivery’ is to be traded for ‘student-centred’ learning. The fact that such conversations are now taking place could certainly be seen as a positive development. I’ll be sharing some diverse perspectives about how university education is perceived, and is evolving (or not), as well as some innovative approaches to teaching in the context of this rapidly-changing learning environment.

In the meantime, here is a simple, but important teaching practice employed one of our colleagues, Prof. Kevin Markle in Accounting and Finance: memorize your students’ names in order to effectively engage their attention. Kevin does so by the first week of classes and he has 270 students. Click here to see how he manages it.

[This post originally appeared on the Green TEA blog, and has been republished here with permission of the author]

D.I.Y. Exam Questions as a Tool for Deep Learning

I couldn’t think of anything other than the OND conference presentations for my blog posting today. It was extremely hard for me to choose which presentation to talk about, but because of my Engineering background I decided to talk about an Engineering presentation that captured my attention.
The presentation was by Andrea Prier, Bill Owen, David Wang, Paula Smith, and Mary Robinson. The presenters focused on a process that they used in their courses (three first year courses and one graduate course) to encourage their students to think more deeply about the material they are learning.
They started their process by demonstrating to the students how their level of learning influences their motivation to learn the material, their retention of the information, and ultimately, their mastery of the course material. They designed a set of assignments in which the students were asked to create their own DIY test questions. This allowed the students find a deeper level of understanding of the content.
The DIY assignments were composed of the following steps:

Identify 5 different concepts from one of your courses.
Choose an example problem from each of the different areas.
Manipulate your problems so that you are solving for a different variable.
Solve your problems.
Trade with a peer and critique / solve the peers problems.
Create 20 questions for a ‘practice final’.
Submit the problems for review by course instructors.
Write your DIY practice final; Mark your own work!
Write your Final Exam; it May include specific questions you created.

By the end of the term, the instructors found that the students were more comfortable working with the course concepts, more engaged with the course material and created some interesting questions. Furthermore, the students’ comments indicated that the majority enjoyed the ability to create these questions; however, the instructors needed to edit some of the created questions.
The most valuable outcomes from the presenters’ perspective were increased success rate of the students and the increase in student motivation to work with the course material. They felt that, since the students valued what they were doing and believed in their ability to do well in the course.
Finally, I would like to thank the presenters for a very useful presentation and would advise myself and other instructors of experimenting with this process.

Mistakes– Keemo Delos Santos, CTE Co-op Student

Mistakes are one of life’s great inevitabilities (along with death and taxes). From small blunders to monumental life-altering ones, mistakes are destined to terrorize our numbered days. In fact, the capacity to err comes so naturally to us that it might as well be a bodily function. However, in spite of (or maybe because of) this certainty, we have grown scared and indifferent to failure.

A side-effect of my continuous years of test-taking is a tendency of assigning a great deal of weight on grades rather than mastery of the material. The upshot: a fear of failure shackling me to the comfortable and routine. Often times, this fear has prevented me from speaking up in class or visiting office hours. But, as humbling and humiliating as they are, mistakes should not be avoided or ignored as they are critical to the learning process. Our attempts at ignoring them serve as barriers to learning. By way of example, the questions that I’ve gotten wrong on tests, and the topics I’ve failed to fully comprehend on the first try are the ones that have stayed with me long past the two-week statute of limitations after exam period ends.

While it’s a struggle to be deal with our insufficiencies, it is vital that we change our attitudes on erring as it is from them that we can grow as learners and citizens of this world. Ultimately, our errors, the inconsequential to the significant, are our becoming. The grace to admit our mistakes and the willingness to learn from them is not only a useful skill to have in university, but is also imperative to our survival in the “real world”.

This is Real Life – Katherine Lithgow

“For the things we have to learn before we can do them, we learn by doing them” –Aristotle

We know that students learn best when learning is personally meaningful and when they are able to use knowledge and concepts to solve real-world, complex problems. They retain that knowledge longer when they learn-by-doing. Authentic learning experiences have the following ten design elements (Lombardi, 2007)

Real-world relevance
Ill-defined problem
Sustained investigation
Multiple sources and perspectives
Collaboration
Reflection (metacognition)
Interdisciplinary perspective
Integrated assessment
Polished products
Multiple interpretations and outcomes

Authentic learning experiences are effective because they help learners make connections between new concepts and existing knowledge structures. When learners can see how new knowledge is personally meaningful, they are better able to retain and assimilate that knowledge into their existing knowledge structures. Working with the concepts regularly and repeatedly in different contexts with others helps with retention and understanding. Including a cycle of reflection on action gives students the time and space they need to consider why they acted as they did, consider the group dynamics and begin to develop the habit of questioning their actions and ideas to help inform future action. Finally, authentic learning experiences help students connect the concepts to the ‘big picture’ which includes the richness of the social setting- the people, the environment, and the activity. This helps the learner explore the concepts in different contexts.

Qualtars (2010) contends that “experiential education needs to be viewed as a unique form of pedagogy involving deep reflection, collaboration and assessment” (p.95). There are a number of courses on campus that offer authentic learning experiences; some of these have been presented at the Integrative and Experiential Learning Series. Examples include the following. Students in Knowledge Integration complete an undergraduate senior research project and present their findings at a poster session- (See the abstracts for the 2013 class projects). Mary Louise McAllister (Environment) offers an integrative, blended course which combines lectures with field trips, peer teaching and tutorial–based project work. The students present their qualitative research findings in a multi-media journal format. Troy Glover (Rec & Leisure Studies) offers a course on program management where students work with a community partner to offer a program. During the course, the students work in small groups with the community partner to conduct a needs assessment and design, implement and evaluate a program. The course is designed so that the students have a number of opportunities to reflect on the experience. In addition, the students participate in a weekend retreat which serves as a team building exercise as well as providing them with a program to critique and use to inform their own program planning. Students in Kelly Anthony’s (SPHHS) course can opt to work on a project with a community partner rather than complete traditional forms of assessment. These students enrich the class readings and discussions by sharing their experiences with their classmates throughout the term.

Initially, students may experience frustration as they deal with the ill-defined problems but they are motivated to carry on because the activity connects the course to the ‘big picture’. Participating in an authentic learning experience helps students relate to the concepts and processes on a personal level and better appreciate nuances that cannot be adequately captured by reading or listening to a lecture. They begin to immerse themselves in the practices of the discipline- both the social structures and the culture of the discipline; they begin to envision themselves as members of the discipline’s community.

There are challenges associated with implementing authentic learning experiences. Risk-taking for both the learner and the instructor is involved; as with any real-life ill-defined problem, neither the student nor the instructor can accurately predict how the experience will unfold. Authentic learning is a collaborative effort. Taking the time to develop teams…well it takes time, effort and requires support. But, the advantage of placing such an experience in an early course is that students can use these skills in later courses as well as outside the academic environment. And it helps students get to know people in their class and program.

Qualtars ( 2010) raises a point worth considering- “unless experiences outside the classroom are brought into the classroom and integrated with the goals and objectives of the discipline theory, students will continue to have amazing outside experiences but will not readily connect them to their in-class learning….Without a careful curriculum involving structured, reflective skill building, students may never learn what we hope they will outside the four walls of the classroom” (p. 95-96). This raises a number of questions and challenges – How can we ensure that students have the opportunity to experience authentic learning at least once during their time at university? What kind of support structures have to be in place to support authentic learning experiences?
How can these courses be identified so students can take advantage of the opportunity? [Some university websites provide a list of courses that offer experiential learning components. These can be further categorized according to faculty or department. See for example:
o Elon University , Kent State, DePaul University Catalogue
o Simon Fraser University – a place where students can find the curricular and co-curricular EE opportunities

If we agree that authentic learning is beneficial to students, is it worth leaving to chance?

Resources:

Qualtars, D.M. (2010). Making the most of learning outside the classroom. In D.M. Qualtars, (Ed), Experiential Education: Making the Most of Learning Outside the Classroom. New Directions for Teaching and Learning, Number 124, (pp. 95-99). San Francisco, CA: Jossey-Bass. ISBN: 978-0-470-94505-6.

The hidden classroom

Public School, 1892. (Boston Public Library)

As you stand at the front of your class, about to start the session, what do you see? What do you notice about your students? Who sits at the front? Who sits at the back? Who is chronically late? Who rarely makes it to class? Who always sits alone?

While we see our students at class, what do we really know about them? There was once a student in my class who consistently fell asleep. As a very new teacher, I was really surprised and, at times, wondered if there was something I was doing wrong. Was my class really that bad? I also wondered about the student and the choice to come to class just to sleep through it. The reality is that I don’t know why the student was falling asleep. Perhaps it was from the fatigue of working two or three part-time jobs in order to pay for school. Possibly, the student’s role outside class was the main caregiver for a sick relative. Maybe 3:30pm on Monday in a darkened 250-seat lecture hall just didn’t work.

In course design workshops, we ask participants to explore the context of their classroom. As part of that reflection, we discuss factors such as the class size, required/elective course, the distribution of majors and non-majors, TA support, etc. But what about the hidden context, the part we rarely hear about.

The reality is that we don’t know what is impacting students’ participation or engagement in our classes. I remember speaking with a colleague several years ago who spoke about how she never attended class. Having never pondered the idea of missing class when I was a student, and knowing how successful my colleague was at school, I was surprised. In essence, the lecture hall didn’t work for her. It was much better for her to learn from the textbook and the assignments. She wasn’t “skipping” class, she was making a choice that made sense for her as a learner.

Given that there is no way for us to know about all the contextual factors that are impacting our students’ success in our class (and, of course, online), what can we do? One option is to consider universal design, which focuses on design for all. Although its roots are in designing accessible places and spaces for persons with disabilities, anyone can benefit from this design modality. Consider curb cuts. Although originally designed to support people with mobility limitations, as a mom, I can tell you they are a blessing when you are pushing a stroller, your toddler is on their tricycle, or your pre-teen is pulling a wagon full of flyers to deliver. There are examples all around campus of design choices that originally served one purpose but benefited many others.

The same holds true for our courses. Thoughtful design that focuses on engaging a broader spectrum of learners, or one that was made consciously to support a specific group, can have far-reaching benefits. Consider transcriptions of video content. While this text helps students with hearing impairments, it can also be really helpful for students whose first language is not English. I would have loved podcasts and video content as a student simply because I am not the best note-taker in the world and am sure I lost a lot by scrambling to take notes. With a podcast, I would have the chance to go back to catch what I’d missed.

There are countless examples of how we can make small changes in our course’s design, our teaching methods, learning activities, and assessments, that can provide deep learning experiences for a broader spectrum of students. If you would like to explore this idea, why not join Jay Dolmage and I next Friday, March 8, as we explore universal design. For information about the workshop, please go to CTE’s events listing.

Using “Transit Questions” in place-based pedagogy – Trevor Holmes

I love being in the classroom, whether it’s large or small, whether I’m officially the teacher or the learner. But I also love getting out of the classroom. Some of the most powerful experiences in my own learning and my own teaching have been observing, interacting, and reflecting in spaces other than lecture halls and seminar rooms. Some time ago, I wrote about place-based pedagogy (with some suggested reading) and gave the example of a workshop for the Educational Developers Caucus (EDC) conference at Thompson Rivers University. Since then, I have continued to use what previously I hadn’t a name for in my own cultural studies course — the field observations and intellectual response papers, the spontaneous “field trips” out into parts of campus to apply concepts, the incorporation of people’s experiences into the framework of the course.

Today’s post is about a small piece of the place-based learning experience I had at the EDC conference, a piece that I’m considering using with my own learners when they do their field observations. To date, I’ve supplied them with reflection questions and notetaking guides for the site visits. I’ve used the online quiz tool in the learning management system to ask “prime the pump” journal questions. But I’ve never yet tried the “transit question” approach. Transit questions were thought-triggering questions handed out just before traveling to the field sites in Kamloops. There were, to my recollection, four different cue cards and each pair of people received one or two cue cards. The idea was that the question on the front (and maybe there was one on the back) would ready us for what we were about to see by asking us about related prior experience with X, or what we expect to find when we get to X, or how is X usually structured. The idea was to talk to our partners about the questions and answer them informally as we made our way to the sites (which took 10-20 minutes to get to).

Photograph of two people in Iceland — Photo of two people in Iceland. Source: Karlbark’s Fotothing stream (shared under CC license)

I can imagine transit questions for pairs that would be suitable for my course too. However, we don’t always have pairs (sometimes small groups, sometimes solitary learners going to a space in their hometown, and so on). I can easily adapt the idea for solo use, though clearly I wouldn’t want someone to be taking notes in response to the prompt while, say, driving!

If we do the field trip to Laurel Creek Conservation area again to test ideas found in Jody Baker’s article about Algonquin Park and the Canadian imaginary, I’ll be using transit questions for the bus ride for sure. With other observations I will have to think about how to adapt the idea. Choosing the right question or questions seems to be important, and offering space to jot notes for those who don’t want to start talking immediately. I’d strongly encourage this approach when you know people will be traveling somewhere for the course by bus, or by foot/assistive device. I can imagine that there are lots of opportunities to do this (and it’s likely already done) in disciplines as varied as geography, planning, fine art, architecture, biology, geosciences, accounting, anthropology, and many others. I’m thinking it would be great if they could pull questions from a question bank to their phones or other devices en route as well… the possibilities!

Transit questions on the way to field sites helped to ready me and my partner for what we’d be looking at, to reflect on the implications of our mini-field trip, and to connect our histories to the present task. I recommend them wholeheartedly.

Engage your students: A SYDE Example — Samar Mohamed

Group 2 working with their TA, Justin Eitchel

What is student engagement and how can we achieve it? These questions are always in my mind. Heller et al state that:

“Faculty stimulate engagement by providing students with active learning experiences, conveying excitement and enthusiasm for their subject, and providing opportunities for student-faculty interactions. Students show their engagement by participating in class discussions, doing research projects, and interacting with their professors and peers.”

An example of an engaging engineering course was discussed in a previous blog in which the course instructors used several blended activities to engage the students with their course material.

Another example on engaging engineering courses is SYDE 411 “Optimization and Numerical Methods”, which is the focus of this blog. I have been working with Prof. Paul Calamai and his teaching team to design and implement engaging blended activities for their students. The designed activities satisfy the previously mentioned criteria by giving the students the opportunity to:

interact actively with both their peers and their teaching team
do research projects
participate in group discussion
provide constructive feedback to their peers
reflect on their own work

SyDe 411 is a new fourth year core Engineering course with an emphasis on understanding and applying numerical methods and optimization techniques as tools for problem solving and systems design. Students’ engagement with the course material is an important aspect of their learning. In order for them to be actively engaged with the course material, Professor Paul Calamai and his teaching team implemented several blended activities that were designed to keep the students engaged with each week’s topics and eager to learn more about these topics. Group Projects and Group Assignments are two main blended activities in this course:

Group Projects:

Prof. Calamai took the group project beyond the regular boundaries and created an enjoyable learning experience for everyone. The group project activity is summarized as follows:

Each group is responsible for a project topic that is worth 25% of the course total grade.
Each group researches a specific topic and submits:

o Lecture notes on the topic including examples of application and/or demonstration.
o One project topic Problem per group member with their solutions.

Groups are paired and dry run presentations between paired groups are conducted to provide the presenting group with feedback and recommendations for improvements.
The presenting group’s project is then posted to a discussion board and another group (reviewing group) reviews it and provides the presenting group with questions and feedback through the discussion board. The presenting group is expected to respond to these questions during it’s presentation.
The presenting group delivers a 30 minutes presentation/lecture on it’s specific topic followed by 10 minutes for questions and answers.
Peer evaluation is conducted twice during the term among each group’s members using the “Comprehensive Assessment for Team-Member Effectiveness” CATME online tool. Peer evaluation provides the students with feedback regarding their effectiveness as team members throughout the academic term.

Group Assignments:

Prof. Calamai presented an interesting scenario for the group assignments in which the students engage with the material and come to the tutorial prepared and ready for the learning experience. The group assignment activity is summarized as follows:

The class is divided into groups in which each group, under the supervision of the TA, is responsible for solving and presenting their specific group assignment problems. Students are encouraged to prepare excellent solutions because a subset of these questions will contribute to parts of the Midterm and Final exams. Each student in the group prepares a solution to a specific assignment problem according to the following schedule:

Individual questions are sent by email to each student in Group X.
Each student submits the answer to his specific question/s to a dropbox.
Professor Calamai grades and gives personal independent feedback to the students.
The students submit a revised version of their answers to a dropbox.
After Professor Calamai approves the answers, the TA posts them to a discussion board so that the rest of the class can see them and ask for clarification.
Group X will run the tutorial and facilitate a discussion around their assignment problems.

I think that SYDE 411 teaching team puts a lot of time and effort in providing an exciting and enjoyable learning experience to their students.

1- R. S. Heller, C. Beil, K. Dam, and B. Haerum “Student and Faculty Perceptions of Engagement in Engineering”, Journal of Engineering Education, July 2010.