Centre for Teaching Excellence Blog – Page 18

Mid-term Feedback – Monica Vesely

Do you ever find yourself wishfully thinking “it would have been nice if I had known…” while reading over you course evaluations? Or have you ever implemented changes based on previous end-of course evaluations only to discover that the new group of students would have preferred the original iteration of your class? If you find yourself wishing you could implement changes based on feedback received for the same group of students, mid-term feedback is for you!

Mid-term evaluations are formative feedback tools that can provide valuable information about how students are experiencing a course. When properly constructed and implemented, both you and your students can benefit from the potential enhancements to the learning experience. You will gain a sense of satisfaction that the learning experience that you have developed is being received by your students as you intended and your students will be grateful for your efforts as they help to shape their own learning environment to better suit their needs.

Ideally, the tools used to obtain feedback should pose some simple questions that can be answered within the class period. Brevity and anonymity are best.

Some midterm feedback strategies include:

Traditional Evaluation Form: These questionnaires can be prepared with a number of Likert-style statements along with a few open-ended questions.
Start, Stop, Continue: Students are asked to take note of the things that they would like to see “start” in the class, “stop” in the class, or “continue” taking place in the class.
The One Minute Paper: By posing 2-3 guiding questions, students are able to identify the most significant things they would like changed in the course. For example: ”What are the two or three significant concepts that you have learned thus far?”, “What questions do you still have about the topics we have covered?” and “What could I have done differently to help you understand the lecture material?”

Depending on the experience of the students, you may have to provide more or less instruction and examples in order to obtain useful feedback. While upper year students will tend to be more skilled at providing constructive feedback, first and second year students may not be used to being asked for their opinion on teaching and learning. Make it clear that you are looking for constructive feedback that you can respond to immediately, this term, for their benefit.

When constructing the feedback questions, make certain that you are only collecting data that you can and will use or respond to. Regardless of the class and level, let students know why you are asking for their input, how you will share it and what you will do with it. Do not mislead the students through your choice of questions or lack of explanation into believing that everything is open for discussion.

Once you have collected the feedback, summarize and interpret it as soon as you can. Then, share it back at the next class if at all possible. Sharing the feedback with all students lets them know that what they say matters and it also lets the students know what their peers value or have difficulty with in the class. Next, identify how you intend to respond and why. If you can’t change something, that’s fine, but make certain you let the students know why. Often students are not aware of certain limitations associated with the course and they appreciate knowing. Clarify what role you as an instructor will play in implementing the changes as well as what role the students will need to play to make the change a success.

The preparation of a summary that highlights what things can be changed, what things can only be changed the next time the course is taught and those things that cannot be changed at all can provide a good overview, particularly in large classes where you will need to group and categorize the responses you receive. This type of transparent and honest exchange goes a long way towards building trust and respect with your class even if you are unable to immediately address a recommendation that has been made.

Consider using mid-term evaluations as another component in your teaching professional development. The creation, use and response to mid-term feedback is a proactive way to help avoid the risk that problems may persist unresolved throughout the course. Not only can this mid-stride feedback help to improve the learning environment for the students, but it can help improve your teaching evaluations at the end of the term as well.

Whether you are looking to bounce around ideas or for specific resources on collecting and using midterm feedback, do not hesitate to contact me, Monica Vesely, or your faculty liaison for a meeting.

References:

Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.

Davis, B. G. (2009). Tools for teaching. 2nd ed. San Francisco, CA: Jossey-Bass.

Svinicki, M., & McKeachie, W. J. (2011). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers. Belmont, CA: Wadsworth.

Yao, Y., & Grady, M. L. (2005). How do Faculty make formative use of student evaluation feedback?: A multiple case study. Journal of Personnel Evaluation in Education, 18(2), 107-126.

September Welcoming Events for New Faculty – Monica Vesely

Last week, over 40 new faculty attended a series of Welcoming Events prepared to help them acclimatize to their new roles as faculty members at the University of Waterloo. On Wednesday, September 4^th new faculty gathered in E5 for a day filled with information sessions and opportunities to meet one another and members of the larger University of Waterloo community. After a brief welcome from Ian Orchard (Vice-President, Academic and Provost), the day got underway with a presentation entitled Navigating your uWaterloo Roles with campus administrators addressing the faculty triumvirate of teaching, research and service along with some words of guidance about how co-operative education interfaces with them all. At the subsequent refreshment break, participants had the opportunity to explore the Academic Support Units Resource Fair showcasing services and resources available across campus.

Next came the Adjusting to Waterloo panel discussion where peers spoke openly about their own experiences as new faculty members and shared thoughts and insights with the audience. This year we were joined by Shannon Dea (Philosophy), Carey Bissonnette (Chemistry) and Christopher Small (Statistics & Actuarial Science/Faculty Association). The post-session Q & A period allowed new faculty to seek answers to a variety of questions ranging from academic (What types of tenure and promotions considerations do I need to be aware of?) to broader community interest inquiries (Where do I find the best pub?).

The morning was capped off by a luncheon with the Chairs, Directors and Deans in the Festival Room at South Campus Hall accompanied by more conversation and an informal information exchange. Later that day, new faculty and their families attended a BBQ at Victoria Park Pavilion in Kitchener. After words of welcome from University President, Feridun Hamdullahpur, and FAUW President, David Porreca, attendees were able to enjoy a casual meal and socialize with their fellow new colleagues and their families.

These welcoming activities were intended as a brief introduction to faculty life at the University of Waterloo and to provide a forum for our incoming class of 2013-2014 new faculty to share experiences and start making connections with their colleagues and the broader University of Waterloo community. The day’s events were planned and hosted by the New Faculty Committee which is composed of representatives from the Centre for Teaching Excellence, the Faculty Association and WatPort.

Peeling Back the Layers: Uncovering Organizational Culture and the Place of Teaching — Donna Ellis, CTE Director

At CTE, we work collaboratively with a wide variety of our campus colleagues – it’s an integral part of what we do. But we also work collaboratively with our colleagues at other institutions. I have been very fortunate to be part of a research group with my teaching centre colleagues from seven other Ontario universities. And our project has been an absolutely fascinating one: how can we uncover the value that our institutions place on teaching?

Our group’s underlying belief is that one fundamental way to ensure quality teaching at our institutions is to foster an organizational culture that values teaching. Full stop. This organizational culture comprises the deep structure of an organization that is rooted in its members’ values, beliefs, and assumptions (Denison, 1996). These elements lead to norms and patterns of behavior. Austin (1990) identified various factors that contribute to the perceptions of university members about their institutional culture, including institutional mission and goals, governance structure, administrators’ leadership style, curricular structure, academic standards, student and faculty characteristics, and the physical environment. Hénard and Roseveare (2012) provided seven levers for promoting an institutional culture that values quality teaching which significantly influenced our research study.

To dig deeper into our research question and underlying belief, we secured a provincial Productivity and Innovation Fund (PIF) grant to review existing literature, develop a survey instrument, and run a pilot study at three of our institutions in the Winter 2014 term. Nearly 4,000 faculty members and students at Western University, McMaster University, and the University of Windsor completed the pilot version of our Teaching Culture Perception Survey. Follow-up focus groups were also run to collect further feedback and insights.

We included two main scales on our survey: perceived existence (agreement rating) and perceived importance of a variety of indicators related to an institutional culture that values teaching. A sampling of the items includes:

there is a strategic plan that positions teaching as a priority
teaching effectiveness is considered in hiring
evidence of effective teaching is considered in the evaluation of faculty members’ job performance (e.g., tenure, promotion, annual evaluations)
there are rewards for effective teaching
learning spaces such as classrooms, labs, and/or studios are designed to facilitate learning
educators are encouraged to use the teaching feedback they receive to improve their teaching
there is an adequately resourced teaching support centre
educators can get financial support to develop their teaching (e.g., grants programs, teaching conferences)
opportunities exist for educators to develop leadership in teaching (e.g., Teaching Fellows program)
programs are evaluated based on student learning outcomes

The factor analyses completed on the data from the faculty and the student versions of the surveys revealed some differences between what is perceived as being in place and what is perceived as important at an institution. Consistently, the importance ratings were higher than the agreement of existence ratings, suggesting that respondents valued the various elements of a potential institutional teaching culture more than they perceived them to actually be in existence. The results also revealed differences between the faculty members’ perceptions and those of the students. The focus groups helped to uncover some of the complexity of the perceptions. For example, when discussing awards to recognize excellent teaching, some participants indicated that such awards are not valued, particularly in relation to research. Others spent time discussing the barriers to effective teaching that stem from aging and inappropriately designed teaching spaces. Another common theme involved issues surrounding poor existing methods for evaluating teaching.

While our analyses have indicated that we need to further refine our survey instruments, we are encouraged by the interest in our work from our colleagues across Canada and beyond. We have also launched a website where we can share information about our ongoing project, including the results as we are able to release them.

So what’s the value placed on teaching at the University of Waterloo? I hope that in the near future we can run the revised survey at our institution so that we can better understand our university community members’ perceptions about the value being placed on this critical part of our fabric: teaching. I think it’s time to peel back the layers and take a closer look.

By Donna Ellis

References:

Austin, A.E. (1990). Faculty cultures, faculty values. New Directions for Institutional Research, 68, 61-74.

Denison, D.R. (1996). What is the difference between organizational culture and organizational climate? A native’s point of view on a decade of paradigm wars. Academy of Management Review, 21, 619-654.

Hénard, F. & Roseveare, D. (2012). Fostering quality teaching in higher education: Policies and practices. France: Organization for Economic Co-operation and Development.

Jurinals — Mark Morton

An email I received this morning momentarily pleased me: I was being invited to submit to a journal called the American Journal of Education Research a version of a presentation I recently delivered at the annual conference of the Higher Education Teaching and Learning Association.

But then I noticed some odd grammatical errors in the email. My suspicions raised, I went to the journal’s website, and then to the website of its publisher, Science and Education Publishing. On the surface, everything looked legit: the 70 journals published by Science and Education Publishing all have hundreds of articles written by academics from all over the world. Each journal also claims to be peer-reviewed, and provides a list of its peer reviewers. But then I came across a tab that explained the “processing fees” that an author must pay in order to have his or her paper considered for publication.

A bit more sleuthing revealed that Science and Education Publishing (and all of its subsidiary journals) are considered by the scholarly community to be “predatory” publishers – that is, they are bogus. They even invent journal names that are easily confused with legitimate journals. For example, the American Journal of Educational Research appears to be trying to ride on the coattails of the American Educational Research Journal, published by the highly regarded Sage Journals.

On the one hand, I wasn’t surprised by all this, as I’d recently read an article about a Canadian journal called Experimental and Clinical Cardiology that used to be legitimate, but which has been lately purchased by an offshore corporation that has turned it into a predatory publisher. But what I was surprised by was how much digging I had to do to confirm that Science and Education Publishing journals are bogus. As a former English professor, I was usually able to detect plagiarism in seconds – and I guess I thought that that skill would transfer over into the realm of fake journals.

Anyway, there are apparently hundreds or even thousands of bogus “academic” journals out there. Here’s helpful a list that’s published annually by a librarian at the University of Colorado.

What I’m still unclear about is this: are the academics who publish in these predatory journals being duped? Or are they knowing participants in this ruse? Is the market for academic research so saturated that even bona fide articles by good scholars can’t find publication in legitimate journals?

Trees of Knowledge — Mark Morton

We all have things we don’t want to know and/or don’t want other people to know. Last week, a video of an ISIS militant beheading an American journalist was released on the web. I’m not going to watch that video, because (among other reasons) I don’t want to know what a beheading looks like. I also don’t want my kids to watch it. I warn them that once you know something, it’s pretty hard to unknow it. I tell them that just as swallowing poison will damage their bodies, consuming disturbing images can harm their minds.

I didn’t always think this. I used to espouse a view that John Milton, author of Paradise Lost, articulated in his prose work Areopagitica: “I cannot praise a fugitive and cloistered virtue, unexercised and unbreathed, that never sallies out and sees her adversary…. the knowledge and survey of vice is in this world [is] so necessary to the constituting of human virtue.” William Blake, the nineteenth-century author and visual artist who wrote an epic poem about Milton, believed something similar. For Blake, humans must progress from a state of childlike innocence (a guileless naivete), to adult experience (with all its horrors), and finally back to a state of renewed innocence (an innocence that encompasses and transcends human horrors). Both Milton and Blake might have been thinking of an adage attributed to the Roman playwright Terence, who said “Homo sum, humani nihil a me alienum puto” — that is, “I am human, I consider nothing human to be alien to me.” I take this to mean that all things — whether they are amazing, joyful, depressing, or horrific — are worthy of human study. And I guess they are. It’s just that I don’t want to be the one studying the horrific things. So, for better or worse, I more and more find myself changing the channel when the news comes on. There are so many things I just don’t want to know.

On the other hand, there are also things that I want to know, but other people want to keep them from me. A case in point: in February, after attending a conference in Anchorage, I took a five-hour boat cruise that got us up close to 21 different glaciers. The tour guide — who was actually a National Parks Forest Ranger — was excellent: informative, articulate, and passionate about the environment. As we approached each glacier in turn, she pointed out where the glacier was a decade ago and where it was now. In each case, the glacier had receded, sometimes by thousands of meters. Never, though, did she allude to global warming as causing the retreat of the glaciers. At the end of the cruise, I approached her and asked her about this omission. She paused, gave me a knowing look, and then said, “We’re not allowed to talk about global warming or climate change.”

Maybe the US government thinks it’s protecting me from dangerous knowledge about climate change, in the same way that I try to protect myself and my kids from disturbing images. And maybe Stephen Harper’s government is also trying to keep me safe by preventing those know-it-all scientists from sharing their troubling research with me. In reality, though, I think that neither the US government nor the current Canadian government has my best interests in mind when it comes to “dangerous” knowledge. Information about fisheries, rivers, forests, tar sands, and so on are not the same as pictures and videos of people being beheaded. We can grieve for the executed American journalist, and work toward ending such conflicts, without having to see his head fall onto the sand. But if we’re going to save the planet — or at least the ecosystems in it that support us — we need access to all the knowledge that’s out there.

Integrated Testlets- What are they?–Samar Mohamed

Last month I attended the annual Society for Teaching and Learning in Higher Education (STLHE) 2014 Conference that was held in Queens University, Kingston. It was an excellent opportunity for me to learn from colleagues across Canada and exchange ideas with them. One of the workshops that attracted my attention was facilitated by Aaron Slepkov and Ralph Shiell from the Dept. of Physics at Trent University. In their workshop, Ralph and Aaron focused on their newly developed testing technique: “Integrated Testlet (IT)”. The presenters started by talking about the benefits of Constructed Response (CR) questions, a common term for questions where students must compose the answer themselves, and how these types of questions enable instructors to gauge their students’ learning. CR questions also allow instructors to give part marks to partially correct answers. The presenters also commented on the trend to switch from CR questions to Multiple Choice (MC) questions in the field of Physics due to increasing class size and the resulting contraints on time and personel resources. However, traditional MC questions don’t allow for part marks or, more importantly from a pedagogical standpoint, enable the instructors (and students) to know where the students went wrong. The integrated testlet method is different in that not only does it allow the students to keep trying each MC question until they get the correct answer, “answer-until-correct question format” enabling the granting of partial marks, but student do not leave the question without knowing the correct answer enabling them to move on to the next integrated question. The method presented changed complex CR physics questions into IT questions. The IT method is based on a traditional testlet, which is a group of MC questions that are based on a common stem (or scenario). In an IT, an answer to a question (task) leads to the next task procedurally, and in this way the students’ knowledge of how various concepts are connected can be assessed. Therefore, items in an integrated testlet are presented in a particular sequence in which the answer for part (a) is used to solve for part (b) and so on. The IT rely on the use of an “answer-until-correct response format”, where the students can keep on making selections on a MC question until the correct response is identified and can be used in subsequent related items. The presenters used the Immediate Feedback Assessment Technique (IF-AT) to allow the students to do several attempts and to get part marks for their response. For more information about IF-AT cards, see Epstein Educational Enterprises website. Moreover, for a sample application of the IF-AT cards at the University of Waterloo see the CTE blog by my colleague Mary Power, The faculty of Science Liaison. In their published paper, the presenters explain the method they have used to transform CR questions to IT questions and analyzed the students’ responses for both question types; it is a very useful and interesting reading that I recommend for instructors thinking about this method.

Providing Authentic Learning Experiences – Katherine Lithgow

This past May, I had the great pleasure of presenting at Laurier’s Integrated and E ngaged Learning Conference with Jill Tomasson Goodwin (Associate Professor -Faculty of Arts teaching in the Digital Arts Communication (DAC) specialization program; Scott O’Neill (Associate Director, Marketing and Communications within the Marketing and Undergraduate Recruitment (MUR)department and Madhulika Saxena (a student in the W2014 DAC 300 course and a recent graduate from uWaterloo’s Arts & Business program).

We wanted to explore how we might bring high quality high impact practices (HQ HIPs) into the classroom. Our presentation focused on DAC 300’s collaborative project that provided students with an authentic experiential learning opportunity where the students worked in teams to address an on-campus community partner’s real world need. Our goal was to highlight how a course might embody the characteristics of HQ HIPs and what can be done in terms of course design and course delivery to make a course a high quality high impact practice. Using DAC 300 as an example, throughout the presentation, we provided ‘tips’ which we hope will help others incorporate high quality high impact learning opportunities into their classrooms.

Experiential education has always been important in education, and it is of particular importance at uWaterloo. We say it is in our DNA. We’re known for our co-op program; experiential learning is one of our Undergraduate Degree Level Expectations and our strategic plan promises ‘Experiential Education for All’. We know that when done well, that is, where learning is “as much social as cognitive, as much concrete as abstract,” and emphasizes both judgment and exploration, experiential education helps students better absorb, retain and transfer knowledge (Lombardi, 2007).

So… what are the characteristics of a high quality high impact practice?

Performance expectations set at appropriately high levels
Significant investment of time and effort by students over an extended period of time
Interactions with faculty and peers about substantive matters
Experiences with diversity
Frequent,timely and constructive feedback
Periodic, structured opportunities to reflect and integrate learning
Opportunities to discover relevance of learning through real-world applications
Public demonstration of competence

(Kuh, G. D., O’Donnell, K., & Reed, S., 2013)

You can view our presentation here to see how these characteristics came to life in DAC 300.

A lot of things came together to make the DAC 300 course a great learning experience. A couple that I want to highlight centre around 1) collaboration and 2) the impact on the instructor and students.

Experiential learning opportunities often bring students into meaningful contact with future employers, customers, clients, and colleagues. What struck me about the DAC 300 project was the extent to which Jill collaborated with an on-campus ‘community partner’ (Scott O’Neill and the MUR department) to provide her students with this real-world, relevant learning opportunity. In turn, Jill’s students collaborated together to provide MUR with a solution to address their real-world need. If we want to make more of these high impact practices available to our students, we will likely have to collaborate with campus partners -campus partners from writing centres, student affairs, living learning communities, residence life and librarians are just a few examples of who these campus partners might be. More important, the collaboration has to benefit all parties.

The role of the instructor often changes when you provide authentic learning experiences to your students. Prepare to learn along with your students. Incorporating authentic learning experiences into your course can be disorienting and uncomfortable for you AND your students. Your role shifts from ‘instructor’ to ‘coach’. Students will come up with solutions or approaches that you have never thought of. That can be a good thing, but it also means relinquishing a certain amount of control, being flexible, and adapting to circumstances- just as we do in the real world.

Jill Tomasson Goodwin has kindly created and shared these 6-Tips-and-10-Tricks-to-Facilitate-Classroom-based-Experiential-Learning. Jill encourages you to adapt them to your needs and invites you to email her (jtomasso@uwaterloo.ca) to chat with her further about how these choices worked in practice.

DAC 300 is a 12-week reflexive theoretically-informed, practice-based course in User Experience Design (the art of understanding, designing, and creating an ‘end-to-end’ experience of technology for users). The course design choices are based on a very real-world application of knowledge — facilitated inside, and tested outside, the classroom, for an actual client, with a pressing need.

During the W2014 offering, Professor Jill Tomasson Goodwin and her third-year Digital Arts Communication class consulted with UWaterloo’s MUR department to design an augmented reality version of a tour brochure. To complete the project, teams of undergraduate students drew upon their knowledge of user experience design, interviewed high school students, and then iteratively prototyped a range of augmented reality experiences, all designed to engage and inform students as they visit and explore the campus. The project and technology have been so successful that UW will use augmented reality to enhance other recruitment publications.

Resources

Kuh, G. D. (2008). High-Impact Educational Practices: What They Are. Who Has Access to Them, and Why They Matter. Association of American Colleges and Universities.

Kuh, G. D. (2008). Excerpt from “High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter”. Association of American Colleges and Universities. https://www.aacu.org/leap/hip.cfm

Kuh, G. D., O’Donnell, K., & Reed, S. (2013). Ensuring quality and taking high-impact practices to scale . Association of American Colleges and Universities.

Lombardi, M. M. (2007). Authentic learning for the 21st century: An overview. Educause learning initiative,1(2007), 1-12. http://www.educause.edu/library/resources/authentic-learning-21st-century-overview

Integrative and Applied Learning Value Rubric (AAC&U) http://www.aacu.org/value/rubrics/integrativelearning.cfm