Course Design Broke my Brain – Crystal Tse

Aaron Silvers Attribution

I took Course Design Fundamentals a few weeks ago, and it broke my brain – in a good way! I have taught before, but this was a great opportunity for me to revisit the course that I’ve been teaching for the past few years from a fresh perspective.

Here are a couple of my take-aways from this workshop that lays out the best practices for course design:

  • Alignment, alignment, alignment – between the intended learning outcomes for your students in the course, the course activities, and the assessment of students’ learning. It was great to have this connection made explicit. However, it was also a jarring experience as some of the concepts I wanted my students to learn were not made explicit in the activities the students engaged in. Time to remedy that!
  • Concept maps for your course are tough to make! I had never created one before for my course and was at a loss at first of how to structure it and what the main concepts I wanted my students to get out of my course. A bit of brainstorming and lots of sticky notes later, I finally fleshed out the main concepts. Two of them were actually not about course content. One was about helping first year students transition to university life (e.g., coping with stress effectively, how to study and take tests). I spend my first lecture telling students about my own experiences as a first year student – that it’s difficult and stressful, but that this stress was temporary and would soon be overcome. I revisit this point by telling stories of my own failures and successes, talking about healthy living, and checking in with students throughout the term. Another way to help with students’ transition is to build community in your classroom so students have support networks they can draw on in times of stress and uncertainty.
  • The other concept was to encourage metacognitive skills (i.e., how to encourage students to reflect and think about their own learning). I do different lecture wrappers (e.g., one minute summaries where students spend a minute writing about the main take-away from the class and what questions they still have that can be addressed in the next class). CTE has a great tipsheet on strategies you can use to encourage self-regulation in students’ learning that can be quick and don’t require a complete overhaul of your course. There are also many evidence-based strategies based on psychological research that can help students study more effectively and engage in more critical thinking.
  • Thinking more about incorporating students’ own experiences into the course in addition to my own perspective. Students come with a wealth of prior knowledge and life experiences that can be drawn on. In the past I have solicited students’ anonymous comments about a topic in the course (especially one that can be particularly controversial or sensitive) prior to class so they are ready for discussion. I’m excited to do this more!


Image provided by Aaron Silvers under the Creative Commons “Attribution” license.

Crystal Tse

Crystal Tse

Crystal is the Educational Research Associate at the CTE where she contributes to the Scholarship of Teaching and Learning work and to program evaluation. She received her PhD in social psychology in the Department of Psychology at the University of Waterloo, where her research involved applying psychological theory to inform evidence-based interventions that address different social issues.

More Posts

Online Math Numbers at Waterloo, and Comparative Judgments as a Teaching Strategy — Tonya Elliott, CEL

math equationOnline Math Numbers

If you weren’t already aware, here are a few numbers about online math at the University of Waterloo:

  • The Math Faculty has been offering fully online courses since Fall 2003 and, since that time, has offered 55 unique online courses to more than 21,000 students
  • The Centre for Education in Mathematics and Computing (CEMC), with support from the Centre for Extended Learning (CEL) and local software company Maplesoft, was the first group on campus to release a large set of open educational resources (OERs). Called CEMC courseware, the OERs include lessons, interactive worksheets, and unlimited opportunities for students to practice skills and receive feedback. At the time of this post, the resources have received over 1.8 million hits from 130,000 unique users in 181 different countries.
  • In 2015, the Canadian Network for Innovation and Excellence (CNIE) recognized CEMC, CEL, and Maplesoft for their OERs through an Award of Excellence and Innovation.
  • The Master for Mathematics for Teachers (MMT) program has the highest enrolment of all the fully online Masters programs offered at the University of Waterloo. MMT and CEL staff who work on the program were one of three teams from Waterloo who won a 2016 Canadian Association for University Continuing Education program award.
  • Maplesoft is using a focus group from Math, CEL, and CTE to develop a new authoring environment that will specifically target the needs of online STEM course authors. It is anticipated that this tool will be released in early 2017 and, over time, should save development costs by 50%.
  • The Math Faculty, together with the Provost’s office, has dedicated $1.2 Million over the next three years for additional work on online projects; over 90 course development slots allocated by CEL have already been filled.

These numbers are some of the reasons Waterloo is considered a leader in the area of online math education.

Comparative Judgments

From June 19 – 22, a small group from Waterloo and I joined an international team of mathematics educators to discuss digital open mathematics education (DOME) at the Field’s Institute in Toronto. Lots of great discussions happened including opportunities and limitations of automated STEM assessment tools, integrity-related concerns, and practical challenges like lowering the bar so that implementing fully online initiatives isn’t the “heroic efforts” for Faculty it’s often viewed as being today. Of all the discussion topics, however, the one that got me most excited – and that my brain has returned to a few times in the month since the conference – is using Comparative Judgement (CJ) in online math courses.

colour shadesThe notion behind CJ is that we are better at making comparisons than we are at making holistic judgments, and this includes judgments using a pre-determined marking scheme.  It doesn’t apply to all types of assessments, but take this test on colour shades to see an example of how using comparisons instead of holistic rankings makes a lot of sense. Proof writing and problem solving may also lend themselves well to CJ and three journal articles are listed at the end of this blog for those who would like to read more.

Here are some of the questions I’ve been pondering:

  • Are there questions we aren’t asking students because we can’t easily “measure” the quality of their responses using traditional grading techniques? How much/when could CJ improve the design of our assessments?
    • Example: Could CJ, combined with an online CJ tool similar to No More Marking, be used by students in algebra courses as a low-stakes peer assessment activity so students could see how different proofs compare to one another? Perhaps awarding bonus credit to students whose proofs were rated in the top X%.
  • Which Waterloo courses would see increases in reliability and validity if graders used CJ instead of traditional marking practices?
  • How much efficiency could Waterloo departments save if high-enrolment courses used CJ techniques instead of marking schemes to grade exam questions or entire exams? Could CEMC save resources while using CJ to do their yearly contest marking?

I don’t have answers to any of these questions yet, but my brain is definitely “on” and thinking about them. I encourage you to read the articles referenced below and send me an email (  If you like the idea of CJ, too, or have questions about anything I’ve written.  If you have questions about Waterloo’s online math initiatives, you’re welcome to email me or Steve Furino.


Jones, I., & Inglis, M. (2015). The problem of assessing problem solving: can comparative judgement help? Educational Studies in Mathematics, 89, 3, pp. 337 – 355.

Jones, I., Swan, M., & Pollitt, A. (2014). Assessing mathematical problem solving using comparative judgement. International Journal of Science and Mathematics Education, 13, pp. 151–177.

Pollitt, A. (2012). The method of Adaptive Comparative Judgement. Assessment in Education: Principles, Policy, & Practice. 19, 3, pp. 281 – 300.


Blackboard image courtesy of AJC1.




Tonya Elliott

Tonya Elliott

In her role as an Online Learning Consultant (OLC) with the Centre for Extended Learning (CEL), Tonya Elliott provides instructional design and project management support to faculty and staff who wish to design, develop, and/or deliver fully online courses, programs, and resources. The majority of her projects are with members of the Faculty of Mathematics; however, she really enjoys working on a variety of online projects from faculty and staff from all areas of campus.

More Posts

Working outside the box

Over 70% of our courses offered on-campus use LEARN to some extent to manage course content and communications and to support online learning activities. Data extracted from Waterloo’s LEARN system can provide us with the details of which courses are using its various built-in tools, such discussion forums, quizzes and rubrics. This data can give us a preliminary snapshot of our innovative courses that use instructional technologies.

However, this snapshot is incomplete because we are currently unable to identify blended courses that use external instructional technologies, such as Piazza, Twitter, peerScholar, Diigo, Top Hat, mobile apps, and so on. As a result, we have not been able to document the full extent of courses that are innovative by virtue of the technologies that they employ to support active, student-centred learning. Such technologies often offer learning opportunities that are not otherwise available, but tracking these non-LEARN instructional technologies is challenging and had not been attempted in the past. However, we know that many instructors are working outside the box. girl looking over a box

Innovative Instructional Technologies Project

This spring the CTE Faculty liaisons, myself, and the SID Emerging Technologies, Dr. Mark Morton, embarked on a project to gain a fuller understanding of the extent of use of instructional technologies outside of the LEARN environment on campus. CTE’s work in this area supports the Outstanding Academic Programming part of the current strategic plan, with the goal to “to expand the appropriate use of technologies to enhance students’ learning experience”. Our first step was to send a request to all instructors to indicate whether they were using Twitter, Quizlet, TopHat, IF-AT cards or Dropbox and to identify tools they were using in categories such as Google tools, polling tools, blogging tools, wikis, or screencasting tools. Many instructors replied and we have started to build a picture of the number and variety of external tools that are being used across campus.

What we found

Over 50 different tools were identified by instructors. Piazza, Google Tools, Camtasia (a screencasting technology), Dropbox (the external Dropbox, not the Learn one) and Twitter were the most frequently mentioned, and the categories with the most variety of tools were presentation tools, blogging tools and polling tools. Screencast-o-matic, Explain Everything, WordPress, MediaWiki, SurveyMonkey and Doodle are just a few of the tools that instructors are integrating into their teaching and learning activities. We will continue to collect information on the many and varied instructional technologies that are being used across campus. We will also be adding a new section to the CTE website in the fall outlining the objectives instructors have for using some of these technologies (for example, dissemination of course content, supporting group work, fostering a community of learners, etc.) as well as identifying  “friendly contacts” for specific tools (instructors who are willing to talk to their colleagues about how and why they are using these technologies in their courses).

As we look to the future and how technologies will enhance students’ learning the trend seems to be towards a more modular or LEGO-like learning ecosystem rather than an LMS, but we may already be there as more courses use a diverse, and likely dynamic, set of technologies for a variety instructional purposes.

photo credit: t whalen via flickr cc

Jane Holbrook

Jane Holbrook

As Senior Instructional Developer (Blended Learning), Jane Holbrook helps to develop faculty programming that promotes the effective use of the online environment in on-campus courses. Working closely with Faculty Liaisons, CEL (Centre for Extended Learning) and ITMS (Instruction Technologies and Multimedia Services), she helps manage initiatives related to “blended learning” courses. She received her BSc and MSc from Dalhousie University but has also studied Graphic Design at the Nova Scotia College of Art and Design.

More Posts - Website

What do students value in online courses? — Dina Meunier, Centre for Extended Learning

CAUCE CNIE logoWhat do students value in online courses?

I had the pleasure of attending the 2016 CAUCECNIE joint conference here in Waterloo from May 30 to Jun 2. There were many interesting sessions, including a keynote by Marc Rosenberg describing Learning Ecosystems and another by Ken Steele highlighting the latest innovations in teaching and learning. A panel interview led by the Ontario Ministry of Training, Colleges and Universities examined how provincial consortia, including our own eCampus Ontario, are promoting an environment of innovation in higher-education technology-enabled learning.

One session I particularly enjoyed presented the results of an online student survey at Wilfrid Laurier University. WLU surveyed students who had recently taken one or more fully online courses and asked them what they valued most in an online learning experience. The top three responses related to online course design were:

  • a well-organized course syllabus;
  • clear course expectations and requirements; and
  • a well-organized course structure (for example, information presented in manageable chunks, segments or modules).

As far as online teaching was concerned, students most valued:

  • fair and consistent grading of assignments and exams; and
  • clear and meaningful feedback on assignments and exams.

It seems to me that these values apply equally to face-to-face, on campus courses and to blended courses as they do to fully online courses. In fact, these items are closely aligned with Chickering & Gamson’s seminal work, Seven Principles for Good Practice in Undergraduate Education. Furthermore, the results of this survey remind us that overall, students are searching for high quality in online education, a recommendation clearly articulated by the Ontario Undergraduate Student Alliance in their recent policy brief:[1]

“At the heart of this policy is a focus on the quality of online learning.  It is crucial that the same standards of quality that apply to traditional, in-classroom courses apply to fully-online courses as well. Ideally, instructors should be capable of teaching an online course as effectively as they would a traditional class.”

But how do you know if your course structure is well-organized or if you have clearly articulated course expectations online? Here are 3 suggestions:

Are there other ways you can ensure quality in your online course design and in your teaching? Share your thoughts by leaving a comment below.

[1] Ontario Undergraduate Student Alliance (March 2016). Policy Briefing: Online Learning.

Dina Meunier is Associate Director of Online Learning, Centre for Extended Learning, University of Waterloo

“Learning from Challenge and Failure”: Resources — Julie Timmermans

Michael Starbird
Michael Starbird, keynote speaker at the 2016 Teaching and Learning Conference.

Presenters at CTE’s recent Teaching and Learning conference explored the theme of Learning from Challenge and Failure. As a follow-up to the Conference, we’d like the share the following list of compiled resources:


Articles and Blog Postings

Podcasts and Talks

Growth Mindset Resources


Learning to Leap via Experiential Education — Michelle Gordon

michelle gordonI can’t say enough about experiential learning.  By stepping outside of textbook learning and living the experience, you develop personal connections to the theory. In my experience, this personal connection creates a drive to learn more about a topic, similar to how when you meet a person you like, you want to know more about them. Through experiential learning, I have also found that I develop soft skills that are not replicable in classroom learning, and which stay with me long after the experience is over.

This fall I was fortunate to be one of six student delegates selected from the University of Waterloo to attend the 21st Conference of the Parties under the United Nations Framework Convention on Climate Change (UNFCCC) which was held in Paris, France. Also known as COP21 for short, this conference resulted in the Paris Agreement — an agreement to limit climate change to well below 2C of warming — being adopted with the consensus of 195 states. This was a historic moment to be a part of, where climate change was front and center on the world stage and it was finally agreed upon that quick and drastic action needs to be taken on a global level. Climate change is one of the global challenges of our century, and I hope that COP21 will be written in history as the turning point towards a cleaner and brighter future without fossil fuels.

Through this experience I learned much more about climate change than I could have in an entire semester in the classroom, but I think the most important thing I learned is confidence in my ability to leap. I believe to leap, or to jump into something new and unfamiliar when the opportunity presents itself instead of waiting until you feel “good enough,” is an essential skill to succeed in what you want in life.

Theoretical Knowledge

When I first applied to be a student delegate for COP21, I was hesitant as I thought I was less knowledgeable than many of my peers who were applying. Because I am in the School of Environment, Resources and Sustainability program, I had a working knowledge of climate change but by no means considered myself anywhere close to an expert! I applied anyway, and was thrilled to be selected. I studied climate change negotiations leading up to COP21, and observed them all around me during the experience. From this I gained a deeper knowledge than I had expected, and I am glad to have made the leap to apply and learn as I went, even if I was hesitant about my experience beforehand!

Social Media

Before attending COP21 I used social media such as Facebook, but I was shy about voicing my thoughts about social and environmental causes. Leading up to and during COP21, it was our job as student delegates to involve the wider campus community in awareness of the conference and climate change. It felt very uncomfortable at first, but I began posting on Facebook, joined Twitter, and then decided to make the leap by volunteering to be one of the lead students on the delegation’s communications and social media team. I felt out of my element at first, but through working in a team with two other students we created a successful and engaging campaign.

Networking and Meeting Influential People

At the COP21 conference, you are surrounded by people from all around the world, many of whom are very influential and knowledgeable. At first I felt a bit intimidated and timid in approaching people. However, I gained confidence when professor Ian Rowlands arranged for a few students and me to chat with Marlo Raylonds (the Chief of Staff to Catherine McKenna, the Minister of Environment and Climate Change) as well as David Miller (the President and CEO of World Wildlife Fund Canada and former mayor of Toronto). Chatting with these intelligent people help me build confidence in knowing that influential people are just like anyone else, and I will now feel more comfortable approaching leaders in the future.

Bringing Experiential Learning into the Classroom

I understand that not many teachers can simply take their students abroad on a whim.  However, experiential learning opportunities are out there — they just need to be found and acted upon!

I think that classroom and lecture studies are important, and can serve their purpose as theoretical foundations for experiences. However, I strongly urge students to be always searching for opportunities to experience their passions outside of the classroom, be it conferences, volunteering, or through work experience.  Remember, Google is your friend!  For example, an afternoon spent searching can uncover field courses you can take for credit abroad or in Canada, bursary programs, and much more. Teachers can support students by sharing opportunities that they become aware of, and urging students to leap: to apply, follow through, and have the confidence to make it happen.

Michelle Gordon is a third year undergraduate student in the Environment and Resource Studies co-op program. Michelle was part of the delegation of students from UW that attended COP21. Michelle’s other interests include outdoor education, ecological restoration, and illustration.

Open Educational Resources: A Call to Action — Dina Meunier

open signOn February 8th, Dr. Rajiv Jhangiani from Kwantlen Polytechnic University in B.C. presented a strong case for the use of open textbooks in higher education to an interested audience at the University of Waterloo1. Open textbooks, he argues, such as those provided through OpenStax College or BC Campus OpenEd, benefit students, professors and institutions.

What are open textbooks? Open textbooks are “licensed under an open copyright license [such as a Creative Commons license] and made available online to be freely used by students, teachers and members of the public.”2 How do open textbooks differ from electronic versions of traditional textbooks? Some textbook publishers provide students with an online or digital version of a traditional hard copy textbook, but access to this electronic version is not free and it is under a limited license, that is, students loose access to the digital textbook after a period of time, for example, 6 months after purchase. Open textbooks, due to the nature of being openly available also promote lifelong learning, says Jhangiani.

There is no denying that traditional textbooks are expensive. Textbook costs have increased by 82% in the last decade, according to Jhangiani and these costs contribute to crippling student debt.  In Canada, the average student graduates with a debt of over $28,000 and three years after graduation, only about one-third of graduates are debt free, according to the Canadian Centre for Policy Alternatives. Jhangiani argues that professors can mitigate this unfortunate situation simply by deciding to use an open textbook or a series of open educational resources to replace their traditional text.

Another advantage of open textbooks, for both students and instructors, lies in their flexibility. Open textbooks, Jhangiani explains, “aren’t just free, they’re free with permissions.” These permissions include the ability not only to retain, reuse and redistribute the resource, but to potentially remix and redistribute it based the instructor’s pedagogical goals for the course.

Universities also benefit from the use of open textbooks. There is a direct relationship, Jhargiani says, between textbook costs and student success and retention. Research shows that students enrolled in courses using OERs, had lower withdrawal rates, had better grades and enrolled in more courses in the current and subsequent semesters3.

So why aren’t more instructors using open textbooks?  Lack of awareness about where to find open textbooks and uncertainty around their quality are two of the main reasons.4 But the quality issue is an issue of perception.  Jhangiani states that quality has improved dramatically in the last 5 years and recent research shows that 75% of faculty who have an opinion about OERs, rate them as equivalent or better than the traditional textbook.5

Want to learn more about open educational resources? March 7 to 11th is Open Education Week so there is no better time to start than right now.

  • Check out how the Faculty of Mathematics is leading the way in Waterloo’s own open courseware initiative: math.uwaterloo and courseware.cemc.uwaterloo;
  • Explore the possibility of incorporating an open textbook in your upcoming course this spring or fall;
  • Are you interested in creating your own set of open educational resources to replace a costly textbook in your large enrollment course? Contact the Centre for Extended Learning (, we may be able to help you!


1To view “Open Educational Practices by Dr. Rajiv Jhangiani,” Centre for Teaching Excellence, (published to YouTube on Feb 12 2016) go to

2Open Textbook FAQ. BCCampus OpenEd.

3 Fischer, L., Hilton, J., Robinson T. J., & Wiley, D. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post-secondary students. Journal of Computing in Higher Education, 27(3), 159-172. doi:10.1007/s12528-015-9101-x

4 Green, K. (N.D.). Going Digital: Faculty Perspectives on Digital and OER course materials. Retrieved from The Campus Computing Project.

5 Allen, I.E., & Seaman, J. (Oct 2014). Opening the Curriculum: Open Educational Resources in U.S. Higher Education, 2014, Babson Research. Retrieved from

The author of this post, Dina Meunier, is Associate Director of Online Learning at Waterloo’s Centre for Extended Learning