Having an opportunity to reflect on my brief time with CTE is a most welcomed development. Not only for those interested in CTE, but for myself, this chance to pause and consider all that has transpired within my introductory entry into the world of CTE has been, quite frankly, remarkable. I’ve only just wrapped up my first term as a GID (Graduate Instructional Developer), though the wealth of experiences makes it feel as though I’ve been here much longer (and I mean that in the best way possible). Firstly, I suppose a bit of preamble is in order before we get ahead of ourselves… Continue reading Teaching teaching to (future) teachers – Joseph Buscemi
We spend a fair bit of time in the CTE talking about assessment. Whether it be in workshops, one-on-one consultations about teaching, curriculum meetings, teaching observations, or just in passing conversation among colleagues. In these instances, discussion often focuses on the function of the assessment – is it being used for diagnostic (to ascertain prior knowledge), formative (to provide feedback and improve performance during the learning process), or summative (to evaluate the learner’s knowledge, skills, or values) purposes. Discussion of course revolves around the type of assessment being used, as well as questions regarding grading, but there are aspects of assessment that go beyond this and begin to explore the role of the student in the assessment process.
When thinking about assessment and how we position it in our teaching, it can be helpful to think about it conceptually while considering what purpose we as instructors assign to it. To do so, we might turn to three approaches to assessment: assessment of learning, assessment for learning, and assessment as learning.
Assessment of learning suggests that assessment is being done for the primary purpose of determining what students have learnt in the class; it is typically done in a summative manner so that students can receive a grade.
Assessment for learning views assessment more as a process, whereby students learn from the assessment; feedback is of great importance as it assists the student in the learning process.
Assessment as learning takes this even further, situating the student him- or herself as an assessor and therefore takes greater responsibility in the learning and assessment process.
None of these approaches to assessment are inherently better than the other, but I would encourage you to reflect on your assessments from a different perspective, adapting what I call Advanced Assessment Considerations. These considerations are intended to allow you to reflect and evaluate your assessment during the design, facilitation, and grading & feedback stages. They do not suggest distinct or varied methods of assessment, but rather, attempt to provide options as to how to modify an existing assessment to make it even better. And by better, I mean better for the students – assessment should motivate and empower students to demonstrate what they have learnt, and the more we can have students become invested in the assessment process, the more meaningful it will be to them.
So what are these advanced assessment considerations? I’ll run through each briefly, providing some initial insight into what each consideration entails and leave you with some questions to consider.
Design considerations apply to how the assessment itself is constructed before the students begin to actively work on the assessment. They are intended to provide learners with agency to be contributors to the assessment and determine what their role will be in the assessment.
To incorporate these into our assessments, we should look at things such as:
- Student choice
- Develop student responsibility for learning with controlled options for assessment; students can choose the type of assessment they feel most suits their learning needs, but must reflect on why they chose it/didn’t choose others (Weimer, 2011)
- Low-stakes and high-stakes assessments
- Implement a variety of low-stakes and high-stakes assessments that allow for students to build confidence and motivation and alleviate stress or anxiety; create a culture of routine assessment, and don’t be afraid to allow strategies (such as cheat sheets or open-book resources) to alleviate the stress of high-stakes assessments such as exams; good students do well, and poor students do poorly, regardless of the exam type (Gharib & Phillips, 2013)
Facilitation considerations apply to the process whereby students complete the assessment, and the ways in which the instructor can help manage this process. They are intended to ensure that assessments are structured logistically and rationally so as to promote student learning and alleviate difficulties that are external to the objectives of the assignment.
These can be implemented when considering the process underlying the assessment by thinking about aspects such as:
- Spacing effect
- Ensure sufficient space between assessments so as to continually reinforce understanding before information is forgotten; the repetition of course content and the process of retrieving content as students begin to forget it helps to reinforce and solidify understanding (Cepeda, Pashler, Vul, Wixted & Rohrer, 2006; Kornell, Castel, Eich & Bjork, 2010)
- Testing effect
- Ensure that formative assessment in course matches type of summative assessment; students learn better by testing course content as opposed to simply studying from a textbook, and they learn especially well if testing is accompanied by some form of feedback (Agarwal et al., 2007)
Finally, grading & feedback considerations apply to the means by which feedback is provided and received by students as a means to close the learning cycle. These ensure that sufficient and meaningful feedback is provided to each student multiple times throughout the course, and that the provided feedback is reflected upon by the student.
In some ways, these are most important as they directly encourage the learner to be involved as the assessor to some extent, the degree to which can vary depending on the assessment. To involve the learner in this capacity, consider the following:
- Peer feedback
- Peer feedback allows for rich and meaningful insight that can directly support and scaffold learning (Dochy, Segers & Sluijsmans, 1999)
- Students need the opportunity to examine feedback and determine how to improve through structured means; this can be especially powerful if part of the peer review process (Reinholz, 2015)
- Student-led feedback
- Support individual student development by encouraging reflection on their own learning goals before receiving feedback; have students take ownership of what they want to receive feedback on
- Assessment guideline creation
- Involving students directly in the creation of assessment guidelines can result in sustained motivation; students can create their own rubrics for the assessments they are completing in consultation with the instructor
Ultimately, depending on your assessment, some of these considerations may simply not be applicable, and attempting to adapt many of these to a single assessment may prove equally challenging. I would however encourage you to think about how some of these considerations could find their way into your own assessments to make the experience all the more meaningful to students and instructors alike.
Agarwal, P.K., Karpicke, J.D., Kang, S.H.K., Roediger III, H.L. & McDermott, K.B. (2007). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22, 861-876.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132, 354–380.
Dochy F., Segers, M. & Sluijsmans, D. (1999) The use of self-, peer and coassessment in higher education: A review. Studies in Higher Education, 24(3), 331-350.
Gharib, A. & Phillips, W. (2013). Test anxiety, student preferences and performance on different exam types in introductory psychology. International Journal of e-Education, e-Business, e-Management and e-Learning, 3(1), 1-6.
Kornell, N., Castel, A.D., Eich, T.S., & Bjork, R.A. (2010). Spacing as the friend of both memory and induction in young and older adults. Psychology and Aging, 25(2), 498-503.
Reinholz, D. (2015). The assessment cycle: A model for learning through peer assessment. Assessment & Evaluation in Higher Education, 41(2), 37-41, 1-15.
Weimer, M. (2011, June, 21). A rose for student choice in assessment? Retrieved from http://www.facultyfocus.com/articles/teaching-professor-blog/a-role-for-student-choice-in-assessment/
The amount of paper handling to administer the potential 33,000 final exam papers from these large courses will be monumental. (For fun, guestimate the volume of paper this amounts to.)
The Mathematics Faculty has been successfully experimenting for a year with a online grading system called Crowdmark, a company founded by Professor James Colliander of the Mathematics Department of the University of Toronto.
Professor Colliander was faced with a similar problem: grading 5000 Canadian Open Mathematics Competition (COMC) papers each year with 100 volunteers. As with final exams, each paper is typically graded by a number of markers so keeping track of which questions are graded on which papers and when the papers are free to be passed to another marker is a time consuming and error prone business.
Crowdmark (CM) attempts to eliminate some of the time and trouble spent managing the grading process. We are not talking about a quiz system with automatic grading. Crowdmark is hand-marking done online. Skilled people still grade, and tests and assignments are still created for printing on paper so there is nothing new in this part of an instructor’s routine.
So, what is it that makes the marking process more efficient when done online?
- Markers are able to grade the same paper at the same time. No more locating and waiting for a paper that someone else is grading. Or waiting for a batch of papers to arrive at your location to begin your stage of grading. Grading can be done concurrently at multiple locations and times.
- Grades can be automatically summed, collected, summarized, distributed and recorded in a Learning Management System without needing to check for arithmetic or transcription errors.
- No time needs to be spent returning piles of exam papers.
There is a time and money cost to using online grading. The physical papers have to be scanned into digital format (PDF file) before grading can start. High speed scanners (500 pages per minute) can process 1000 10-page exams in 20-30 minutes once delivered to the scanning machine.
Here I’ll briefly discuss how instructors and students use CM.
Steps for an instructor:
- upload one test or exam pdf file into CM (leave 2 inches blank on the top of each page for CM ID info and set 1 question per page)
- CM duplicates the test pdf for each student and adds a paper and page ID to each page
- download from CM the pdf file of student tests and print it
- after the test scan all written test papers into a pdf file and upload the file into CM
- CM arranges the pdf file pages into a grid pattern: each row holds a student’s test pages
- each marker clicks on a page in the grid to read, comment, and grade it
- when grading is complete page grades are summed for each test paper by CM
- match each test paper cover page student ID with a student name in your CM course (assigned seating at UW can eliminate this step)
- you choose whether CM sends each student their grade and a CM link to their graded test paper or to keep the grades and graded papers private and just download the grades for inclusion into a course grade
Steps for a student:
- write the test paper by hand as usual
- may receive an email from CM with a link to a CM page showing their test results
The links at the end of this post provide further details about Crowdmark. In addition, 2 live sessions demonstrating Crowdmark are coming up at the end of August and the beginning of September. The first is an introduction to Crowdmark on Wednesday August 31 and the second follows up a week later on Wednesday September 7 (1:30-3 PM) with details about a University of Waterloo system named Odyssey that works with Crowdmark. Odyssey organizes test papers, students and exam room seating providing relief from some time-consuming management overhead.
Crowdmark is not a free service, but the University of Waterloo has a licence so there is no charge to individuals (instructors or students) at the university.
If you are interested in learning more about online grading for your course please get in touch with me.
Mathematics Faculty CTE Liaison
firstname.lastname@example.org, x37047, MC 6473
Goals. Aims. Objectives. Outcomes. Metrics. Performance Indicators. Ideal Graduate Attributes.
Last week, I spent some time with colleagues debating the meaning of these various terms. They are often used interchangeably but, depending who you ask, they don’t mean the same thing. I tend to lump goals, aims and objectives together because they represent our intentions – what we will work towards during a given learning experience. I see outcomes and attributes as what students are actually able to do by the end of that experience (specific behaviours, knowledge, skills, attitudes they have developed). Finally, I place metrics and performance indicators into a category of measurements of those outcomes. Our discussion last week verified that while we use these terms in the similar ways, it’s worth taking the time to clarify our shared understanding of these key terms.
Now, it’s time to expand that conversation across campus. I’m excited to announce a new learning community at CTE – program outcomes assessment. Many departments across campus are engaged in program assessment through academic program review, accreditation, and curriculum design and renewal. Bob Sproule (a member of the School of Accounting and Finance’s Learning Outcomes Committee) and I will be leading this group as we explore various aspects of program outcomes assessment.
The first session, on May 12, 2016 12:00-1:15pm in EV 241, is a brainstorming session to explore topic ideas for the coming year. Our goal is to meet twice per term, starting in Fall 2016, and we want to ensure the sessions reflect areas of interest for you, our community members. If you are unable to attend the session but are interested in joining the community, please email me, Veronica Brown (email@example.com), Sr. Instructional Developer, Curriculum and Quality Enhancement, Centre for Teaching Excellence.
Before you read this blog post, let me assure you that I believe every student should have access to a positive learning environment. I am in no way advocating that we should start making students fight to survive in a distressing domain. However, there is a movement spreading wildly through post-secondary campuses to offer students increasing protection from ideas they do not like and words that make them uncomfortable. To date, there is no scientific evidence that coddling students is having a positive impact on society or on their future; the widespread use of trigger warnings could be dangerous for mental health. In an article written for The Atlantic, Greg Lukianoff and Jonathan Haidt pose the question: “What exactly are students learning when they spend four or more years in a community that polices unintentional slights, places warning labels on works of classic literature, and in many other ways conveys the sense that words can be forms of violence…?” (1). Using warnings to protect students from potentially harmful ideas is setting them up for larger issues once they leave the “safe space” campuses that have been created for them. Instead, let’s help teach students how to cope with these potentially triggering situations.
Assisting people’s efforts to avoid their fears is misguided. Cognitive behavioural therapy (CBT) is a non-pharmaceutical treatment of mental illness that would be more beneficial to students than using trigger warnings. CBT involves the patient working with a mental health counselor to help him or her “become aware of inaccurate or negative thinking so [he/she] can view challenging situations more clearly and respond to them in a more effective way” (2). The basis of the process is simple: notice that you’re being affected by a stressor, name it, describe the facts of the situation, and consider alternative interpretations. After treatment, people are less likely to suffer from anxiety, depression, and anger. Over time, this process becomes more automatic and also helps enhance critical thinking skills. This method can further be combined with exposure therapy, where a patient’s cognitive distortions are diluted with a gradual increase of exposure to the offending scenario. Over time, using either approach, the triggered reaction would deflated. You can’t always control the situation, but you can control how your thoughts, actions and feelings affect the situation.
Keep in mind that I am not an expert. I am not prescribing treatment, nor am I making false promises for results I cannot guarantee (which in this case – I can’t). I’m also not naïve enough to think that cognitive behavioural therapy or exposure therapy are one-stop solutions that will work for everyone. Every person and situation is different. Furthermore, not every reaction to trauma will require the use of one of these coping mechanisms.
Where is all this coming from? Straight out of high school, I attended college. I earned my diploma, worked in my field for a couple years, and then decided it wasn’t the right fit for me. As someone who has recently made the transition back to academia from the “real world,” I’m rather shocked at how hard it is to say the right thing anymore. It can be very tricky to live in a world where “I’m offended” could be used at any given time as an unbeatable trump card. Constantly having to worry about whether an answer you give in class will elicit an upsetting response while discussing the assigned material is exhausting. As a student, I would prefer to utilize the nonthreatening environment of the classroom to discuss controversial topics instead of feeling limited by uncertain restrictions.
The above methods do not entirely invalidate the need for trigger warnings on certain material. They are an excellent temporary solution. It would be more practical long term to negotiate coping mechanisms in a classroom environment before students are released into the “real world.” Throughout elementary and into high school, students are sheltered to protect them. A safe space is provided to nurture students as they mature. In the infamous life after school people keep talking about, it is likely you will have to engage with people and ideas that make you uncomfortable. Outside of the classroom, there will be significantly less protection from any potential stressors. The long-term benefits of developing proper coping mechanisms should not be diminished.
This post was inspired by “The Coddling of the American Mind” written by Jonathan Haidt and Greg Lukianoff for The Atlantic’s September 2015 issue. You can view their full article here: http://www.theatlantic.com/magazine/archive/2015/09/the-coddling-of-the-american-mind/399356/
To read more on trigger warnings, check out the Centre for Teaching Excellence’s tip sheet here: https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/creating-positive-learning-environment/trigger-warnings-and-content-notes-inviting-all-students
(1) The Atlantic – http://www.theatlantic.com/magazine/archive/2015/09/the-coddling-of-the-american-mind/399356/
(2) The Mayo Clinic – http://www.mayoclinic.org/tests-procedures/cognitive-behavioral-therapy/home/ovc-20186868
My introduction to feminist pedagogy took place as an undergraduate student in an upper-level geography class at the University of British Columbia. The final assignment for this class involved conducting a community-based research project. Students were assigned small groups, in which they partnered with community organizations to design and carry out research based on the organization’s needs. My small group partnered with a local food bank, which asked us to investigate how the food bank could partner with community gardens in order to contribute to food security in Vancouver.
This learning experience was transformative. Ultimately, it shifted my thinking about social life in Vancouver and motivated me to pursue my interests through research. Importantly, through this class, I was introduced to spaces within the university that were orientated towards building community and social action. This is one explicit intention of feminist pedagogy. In classrooms grounded in feminist values, “there is a need and desire to move learning beyond the walls of the classroom” (Shrewsbury 1993, p. 171). Students are encouraged to extend theory to action, and then action is brought back into the classroom in order to inform theory.
What is feminist pedagogy?
My initial experience with feminist pedagogy not only shaped my research, it laid the foundations for my approach to teaching and learning. Broadly, feminist pedagogy is a way of thinking about teaching and learning that is grounded in feminist values. It is therefore more appropriate to speak about feminist pedagogies, and then to highlight what these different approaches to teaching and learning have in common. Common principles of feminist pedagogies include:
- Resisting hierarchy and empowerment: In a traditional university classroom, the instructor holds power over the class and knowledge is passed from instructor to students. In contrast, feminist pedagogy involves the de-centering of power. Instructor and students exist in a “symbiotic” relationship and knowledge is constructed through discussion, dialogue, and critical inquiry. Importantly, students are also invited to play a role in influencing the design of the class and to participate in the delivery of content.
- Building community and using personal experience as a resource: Like a traditional university classroom, many classrooms grounded in feminist values continue to rely on traditional sources of information, such as textbooks and academic journals. However, feminist pedagogy also involves drawing on students’ and teachers’ own experiences as learning materials. This promotes critical thinking, as students are challenged to bridge scholarship with “real-life.” Drawing on personal experience in order to deepen and widen understanding of course content also encourages students to value difference and diversity.
- Transformative learning: This is the principle that was put into action in my first experience with feminist pedagogy as an undergraduate student. Traditional university classrooms may provide limited space for critical thinking and problem solving. However, in classrooms grounded in feminist values, teaching and learning aim to shift thinking in new directions. Students are asked to examine either their own experiences or social phenomena in new and critical ways. This often involves creating learning experiences that draw attention to real world problems or power differences that contribute to inequality.
Importantly, feminist pedagogy is a way of thinking about teaching and learning that shapes both what we teach and how we teach it. But how can this translate into classrooms that might, at first glance, not appear conducive to such approaches? In other words, are there practical applications of the principles of feminist pedagogy in all university classes? Or, is feminist pedagogy best reserved for certain levels of learning and certain disciplines?
Feminist pedagogies in practice
Research suggests that there is space for feminist pedagogy in all university classrooms and at all levels of learning; however, what this looks like in practice is likely to vary significantly. Here are some examples of feminist pedagogy in practice, along with links to research in support of this practice:
- Feminist pedagogy can be used to change teaching strategies and deepen learning outcomes in engineering. For example, Cashman and Eschenbach (2004) use labs to teach students how to work in small-groups to design their own approach to solving real-world problems. They ask students to solve problems that are community-based or locally relevant. Cashman and Eschenbach (2004) have found that this empowers students to extend their classroom learning into the community – some students even become involved in community projects or local politics. This approach also encourages students to approach exams and homework assignments using real-life scenarios.
- Feminist pedagogy can be used in psychology education to de-centre power in the classroom, contribute to more diverse curriculum, and to guide students in developing their feminist consciousness. For example, Robinson-Keilig et al. (2014) adopted the photovoice research methodology for a classroom project on violence against women. The project encouraged self-disclosure of students’ own lived experiences, as a means of integrating student knowledge into classroom content. The authors of this study found that this particular project facilitated critical consciousness, as students became more aware of the multiple systems of oppression that exist in society and became empowered via new insights and self-reflection to make change.
- Feminist pedagogy can be integrated into economic classrooms to include students in the learning process and urge students to grow as critical thinkers. For example, Nelson and Goodwin (2005) argue that economics learning materials often do an inadequate job at integrating feminist concerns into introductory economics courses. Accordingly, they have published alternative learning materials that introduce students to neoclassical economic principles but also make room for ecological and feminist concerns. They argue that this deepens students’ understanding of economics, as they are exposed to less mainstream debates about the interconnection between economics, gender, and the environment.
Ultimately, if we understand feminist pedagogy as a way of teaching and learning that empowers students to be reflective and critical learners and to apply their learning through social action, it is possible to imagine how this approach is relevant across the university and beneficial to students, regardless of discipline.
Crawley, S. L., Lewis, J. E., & Mayberry, M. (2008). Introduction—feminist pedagogies in action: Teaching beyond disciplines. Feminist Teacher, 19(1), 1-12.
GEA – Gender and Education Association (n.d.). Feminist Pedagogy.
Shrewsbury, C. M. (1997). What is Feminist Pedagogy? Women’s Studies Quarterly, 25(1/2), 166-173.
Charis Enns is a Graduate Instructional Developer in CTE and a PhD Candidate at the University of Waterloo and the Balsillie School of International Affairs.
Teaching using case studies has typically been used in Business Schools, Law Schools, and Medical Schools but it is a technique being used by other disciplines to provide exposure to complex real world problems for which there is no “right” or “wrong” answer. At Waterloo, cases have been used in disciplines including Engineering, Biology, Accounting, Social Work, Environment and Business, English and others.
The traditional “Case Method” used in Business Schools involves a three stage process where:
- students are given the case and asked to work on it individually to come up with a recommendation or course of action (done outside of class time). The key here is for students to be able to justify and support their choices or decisions.
- students meet in small groups of 4 or 5 to discuss the case and their recommendations (done outside of class time) – the objective here is to share perspectives, not come to a consensus as a group
- the case is discussed in class with the entire class with the Professor acting as a facilitator to guide discussion.
The amount of learning increases over each stage with exposure to different perspectives.
While this is the typical method used in MBA programs where cases are used in most courses, it can be modified and adapted. For instance, students can read the case and prepare before class and class time can be used for small group discussion and then discussing the case as a large group (i.e. the entire class). It is important to communicate expectations to students about coming to class prepared as the quality of discussion depends on proper preparation. One technique to encourage students to prepare is to give them questions about the case to answer and submit before class begins.
These techniques of using small group work for peer teaching (i.e. small group work to share perspectives) and facilitating a discussion with the entire class can be adapted and used for other contexts than just cases.
If you’re interested in learning more about the Case Method or using cases in your course, contact Scott Anderson in the Centre for Teaching Excellence.
Waterloo Cases in Design Engineering also writes and supports the use of cases in Engineering courses.
Erskine, J., Leenders, M., and Mauffette-Leenders, L. (2012). Learning with Cases, 4th Edition, Ivey Publication Services, Richard Ivey School of Business, Western University, London, Ontario, Canada.
Erskine, J., Leenders, M., and Mauffette-Leenders, L. (2003). Teaching with Cases, 3rd Edition, Ivey Publication Services, Richard Ivey School of Business, Western University, London, Ontario, Canada.
Mauffette-Leenders, L., Erskine, J. and Leenders, M. (2001) Writing Cases, 4th Edition, Ivey Publication Services, Richard Ivey School of Business, Western University, London, Ontario, Canada.