Course Design Broke my Brain – Crystal Tse

Aaron Silvers Attribution

I took Course Design Fundamentals a few weeks ago, and it broke my brain – in a good way! I have taught before, but this was a great opportunity for me to revisit the course that I’ve been teaching for the past few years from a fresh perspective.

Here are a couple of my take-aways from this workshop that lays out the best practices for course design:

  • Alignment, alignment, alignment – between the intended learning outcomes for your students in the course, the course activities, and the assessment of students’ learning. It was great to have this connection made explicit. However, it was also a jarring experience as some of the concepts I wanted my students to learn were not made explicit in the activities the students engaged in. Time to remedy that!
  • Concept maps for your course are tough to make! I had never created one before for my course and was at a loss at first of how to structure it and what the main concepts I wanted my students to get out of my course. A bit of brainstorming and lots of sticky notes later, I finally fleshed out the main concepts. Two of them were actually not about course content. One was about helping first year students transition to university life (e.g., coping with stress effectively, how to study and take tests). I spend my first lecture telling students about my own experiences as a first year student – that it’s difficult and stressful, but that this stress was temporary and would soon be overcome. I revisit this point by telling stories of my own failures and successes, talking about healthy living, and checking in with students throughout the term. Another way to help with students’ transition is to build community in your classroom so students have support networks they can draw on in times of stress and uncertainty.
  • The other concept was to encourage metacognitive skills (i.e., how to encourage students to reflect and think about their own learning). I do different lecture wrappers (e.g., one minute summaries where students spend a minute writing about the main take-away from the class and what questions they still have that can be addressed in the next class). CTE has a great tipsheet on strategies you can use to encourage self-regulation in students’ learning that can be quick and don’t require a complete overhaul of your course. There are also many evidence-based strategies based on psychological research that can help students study more effectively and engage in more critical thinking.
  • Thinking more about incorporating students’ own experiences into the course in addition to my own perspective. Students come with a wealth of prior knowledge and life experiences that can be drawn on. In the past I have solicited students’ anonymous comments about a topic in the course (especially one that can be particularly controversial or sensitive) prior to class so they are ready for discussion. I’m excited to do this more!

 

Image provided by Aaron Silvers under the Creative Commons “Attribution” license.

Crowdmark – Online grading for large courses

This Fall 2016, the University of Waterloo will have 25 courses with stockvault-pile-of-paper117595a class size of between 500 and 1000 students and 10 courses of  between 1000 and 2000 students.

The amount of paper handling to administer the potential 33,000 final exam papers from these large courses will be monumental. (For fun, guestimate the volume of paper this amounts to.)

The Mathematics Faculty has been  successfully experimenting for a year with a online grading system called Crowdmark, a company founded by Professor James Colliander of the Mathematics Department of the University of Toronto.

Professor Colliander was faced with a similar problem: grading 5000  Canadian Open Mathematics Competition (COMC) papers each year with  100 volunteers.  As with final exams, each paper is typically graded by a number of markers so keeping track of which questions are graded on which papers and when the papers are free to be passed to another marker is a time consuming and error prone business.

Crowdmark (CM) attempts to eliminate some of the time and trouble spent managing the grading process.   We are not talking about a quiz system with automatic grading. Crowdmark is hand-marking done online.  Skilled people still grade, and tests and assignments are still created for printing on paper so there is nothing new in this part of an instructor’s routine.

So, what is it that makes the marking process more efficient when done online?

  • Markers are able to grade the same paper at the  same time.  No more locating and waiting for a paper that someone else is grading.  Or waiting for a batch of papers to arrive at your location to begin your stage of grading.  Grading can be done concurrently at multiple locations and times.
  • Grades can be automatically summed, collected, summarized, distributed and recorded in a Learning Management System without needing to check for arithmetic or transcription errors.
  • No time needs to be spent returning piles of exam papers.

There is a time and money cost to using online grading.  The physical papers have to be scanned into digital format (PDF file) before grading can start. High speed scanners (500 pages per minute) can process 1000 10-page exams  in 20-30 minutes once delivered to the scanning machine.

Here I’ll briefly discuss how instructors and students use CM.

Steps for an instructor:

  • upload one test or exam pdf file into CM (leave 2 inches blank on the top of each page for CM ID info and set 1 question per page)
    • CM duplicates the test pdf for each student and adds a paper and page ID to each page
  • download from CM the pdf file of student tests and print it
  • after the test scan all written test papers into a pdf file and upload the file into CM
    • CM arranges the pdf file pages into a grid pattern: each row holds a student’s test pages
  • each marker clicks on a page in the grid to read, comment, and grade it
    • when grading is complete page grades are summed for each test paper by CM
  • match each test paper cover page student ID with a student name in your CM course (assigned seating at UW can eliminate this step)
  • you choose whether CM sends each student their grade and a CM link to their graded test paper or to keep the grades and graded papers private and just download the grades for inclusion into a course grade

Steps for a student:

  • write the test paper by hand as usual
  • may receive an email from CM with a link to a CM page showing their test results

The links at the end of this post provide further details about Crowdmark.  In addition, 2 live sessions demonstrating Crowdmark are coming up at the end of August and the beginning of September.   The first is an introduction to Crowdmark on Wednesday August 31 and  the second follows up a week later on Wednesday September 7 (1:30-3 PM) with details about a University of Waterloo system named Odyssey that works with Crowdmark.  Odyssey organizes test papers, students and exam room seating providing relief from some time-consuming management overhead.

Crowdmark is not a free service, but the University of Waterloo has a licence so there is no charge to individuals (instructors or students) at the university.

If you are interested in learning more about online grading for your course please get in touch with me.

Paul Kates
Mathematics Faculty CTE Liaison
pkates@uwaterloo.ca, x37047, MC 6473

Intro to Online Marking using Crowdmark: Wednesday, August 31, 2016 – 10:30 AM to 11:30 AM EDT
Crowdmark home page,   help pages and  youtube channel.
UW Odyssey Examination Management

Online Math Numbers at Waterloo, and Comparative Judgments as a Teaching Strategy — Tonya Elliott, CEL

math equationOnline Math Numbers

If you weren’t already aware, here are a few numbers about online math at the University of Waterloo:

  • The Math Faculty has been offering fully online courses since Fall 2003 and, since that time, has offered 55 unique online courses to more than 21,000 students
  • The Centre for Education in Mathematics and Computing (CEMC), with support from the Centre for Extended Learning (CEL) and local software company Maplesoft, was the first group on campus to release a large set of open educational resources (OERs). Called CEMC courseware, the OERs include lessons, interactive worksheets, and unlimited opportunities for students to practice skills and receive feedback. At the time of this post, the resources have received over 1.8 million hits from 130,000 unique users in 181 different countries.
  • In 2015, the Canadian Network for Innovation and Excellence (CNIE) recognized CEMC, CEL, and Maplesoft for their OERs through an Award of Excellence and Innovation.
  • The Master for Mathematics for Teachers (MMT) program has the highest enrolment of all the fully online Masters programs offered at the University of Waterloo. MMT and CEL staff who work on the program were one of three teams from Waterloo who won a 2016 Canadian Association for University Continuing Education program award.
  • Maplesoft is using a focus group from Math, CEL, and CTE to develop a new authoring environment that will specifically target the needs of online STEM course authors. It is anticipated that this tool will be released in early 2017 and, over time, should save development costs by 50%.
  • The Math Faculty, together with the Provost’s office, has dedicated $1.2 Million over the next three years for additional work on online projects; over 90 course development slots allocated by CEL have already been filled.

These numbers are some of the reasons Waterloo is considered a leader in the area of online math education.

Comparative Judgments

From June 19 – 22, a small group from Waterloo and I joined an international team of mathematics educators to discuss digital open mathematics education (DOME) at the Field’s Institute in Toronto. Lots of great discussions happened including opportunities and limitations of automated STEM assessment tools, integrity-related concerns, and practical challenges like lowering the bar so that implementing fully online initiatives isn’t the “heroic efforts” for Faculty it’s often viewed as being today. Of all the discussion topics, however, the one that got me most excited – and that my brain has returned to a few times in the month since the conference – is using Comparative Judgement (CJ) in online math courses.

colour shadesThe notion behind CJ is that we are better at making comparisons than we are at making holistic judgments, and this includes judgments using a pre-determined marking scheme.  It doesn’t apply to all types of assessments, but take this test on colour shades to see an example of how using comparisons instead of holistic rankings makes a lot of sense. Proof writing and problem solving may also lend themselves well to CJ and three journal articles are listed at the end of this blog for those who would like to read more.

Here are some of the questions I’ve been pondering:

  • Are there questions we aren’t asking students because we can’t easily “measure” the quality of their responses using traditional grading techniques? How much/when could CJ improve the design of our assessments?
    • Example: Could CJ, combined with an online CJ tool similar to No More Marking, be used by students in algebra courses as a low-stakes peer assessment activity so students could see how different proofs compare to one another? Perhaps awarding bonus credit to students whose proofs were rated in the top X%.
  • Which Waterloo courses would see increases in reliability and validity if graders used CJ instead of traditional marking practices?
  • How much efficiency could Waterloo departments save if high-enrolment courses used CJ techniques instead of marking schemes to grade exam questions or entire exams? Could CEMC save resources while using CJ to do their yearly contest marking?

I don’t have answers to any of these questions yet, but my brain is definitely “on” and thinking about them. I encourage you to read the articles referenced below and send me an email (tonya.elliott@uwaterloo.ca)  If you like the idea of CJ, too, or have questions about anything I’ve written.  If you have questions about Waterloo’s online math initiatives, you’re welcome to email me or Steve Furino.

References

Jones, I., & Inglis, M. (2015). The problem of assessing problem solving: can comparative judgement help? Educational Studies in Mathematics, 89, 3, pp. 337 – 355.

Jones, I., Swan, M., & Pollitt, A. (2014). Assessing mathematical problem solving using comparative judgement. International Journal of Science and Mathematics Education, 13, pp. 151–177.

Pollitt, A. (2012). The method of Adaptive Comparative Judgement. Assessment in Education: Principles, Policy, & Practice. 19, 3, pp. 281 – 300.

__________

Blackboard image courtesy of AJC1.

 

 

 

Working outside the box

Over 70% of our courses offered on-campus use LEARN to some extent to manage course content and communications and to support online learning activities. Data extracted from Waterloo’s LEARN system can provide us with the details of which courses are using its various built-in tools, such discussion forums, quizzes and rubrics. This data can give us a preliminary snapshot of our innovative courses that use instructional technologies.

However, this snapshot is incomplete because we are currently unable to identify blended courses that use external instructional technologies, such as Piazza, Twitter, peerScholar, Diigo, Top Hat, mobile apps, and so on. As a result, we have not been able to document the full extent of courses that are innovative by virtue of the technologies that they employ to support active, student-centred learning. Such technologies often offer learning opportunities that are not otherwise available, but tracking these non-LEARN instructional technologies is challenging and had not been attempted in the past. However, we know that many instructors are working outside the box. girl looking over a box

Innovative Instructional Technologies Project

This spring the CTE Faculty liaisons, myself, and the SID Emerging Technologies, Dr. Mark Morton, embarked on a project to gain a fuller understanding of the extent of use of instructional technologies outside of the LEARN environment on campus. CTE’s work in this area supports the Outstanding Academic Programming part of the current strategic plan, with the goal to “to expand the appropriate use of technologies to enhance students’ learning experience”. Our first step was to send a request to all instructors to indicate whether they were using Twitter, Quizlet, TopHat, IF-AT cards or Dropbox and to identify tools they were using in categories such as Google tools, polling tools, blogging tools, wikis, or screencasting tools. Many instructors replied and we have started to build a picture of the number and variety of external tools that are being used across campus.

What we found

Over 50 different tools were identified by instructors. Piazza, Google Tools, Camtasia (a screencasting technology), Dropbox (the external Dropbox, not the Learn one) and Twitter were the most frequently mentioned, and the categories with the most variety of tools were presentation tools, blogging tools and polling tools. Screencast-o-matic, Explain Everything, WordPress, MediaWiki, SurveyMonkey and Doodle are just a few of the tools that instructors are integrating into their teaching and learning activities. We will continue to collect information on the many and varied instructional technologies that are being used across campus. We will also be adding a new section to the CTE website in the fall outlining the objectives instructors have for using some of these technologies (for example, dissemination of course content, supporting group work, fostering a community of learners, etc.) as well as identifying  “friendly contacts” for specific tools (instructors who are willing to talk to their colleagues about how and why they are using these technologies in their courses).

As we look to the future and how technologies will enhance students’ learning the trend seems to be towards a more modular or LEGO-like learning ecosystem rather than an LMS, but we may already be there as more courses use a diverse, and likely dynamic, set of technologies for a variety instructional purposes.

photo credit: t whalen via flickr cc

Reflections of a TAWF – Stephanie Verkoeyen

Since next month marks my last workshop as a TA Workshop Facilitator (TAWF), I wanted to use my last blog post to reflect on my experience over the past year.

21194820_d6bfc6f5c9_qWhen I first started as a TAWF last September, I had no way of anticipating all of the wonderful additional opportunities for professional and personal development. As a member of the graduate staff you are welcomed into the CTE community. Living out of town and only occasionally coming to campus, I’ve found it difficult at times to feel a sense of belonging at the university. So the chance to connect with like-minded people and engage in conversations about teaching and learning has been wonderful and proven invaluable.

TAWFs are generally paired with one or two workshops that they will be responsible for for the duration of their appointment. These workshops are offered at least once per term. This means that you have several chances to deliver the same material and receive feedback on your delivery. As a teacher, this is a wonderful opportunity to flex your facilitation skills and gain a greater appreciation for your strengths and areas that may need improvement.

Multiple iterations of a workshop also mean that you will learn about at least one teaching and learning topic in-depth. Before I became a TAWF I had never thought to consult the literature on best teaching practices, and yet there is an entire body of work devoted to this very topic. Referred to as the Scholarship of Teaching and Learning (or SoTL for short), post-secondary practitioners share their experiences of their own classroom successes and failures so that others can reflect on their findings and build upon teaching and learning processes.

The TAWF position also opens the door to other opportunities. There’s the option of getting involved with teaching and learning projects that are happening at the Centre or elsewhere within the University. Several TAWFs have also gone on to take on a GID (Graduate Instructional Developer) position.

I leave this position not only with a greater understanding of my approach to teaching and learning and improved knowledge in this field, but also with invaluable connections to both my graduate colleagues in other faculties and the staff of CTE. It’s been an honour and a privilege!

 

Why “Following Your Passion” Is Bad Advice – Elorm Agbeyaka

9318593026_fa45b15338_mFollow your passion; the mantra that successful people will preach to inspire those who will listen. I become a little weary every time I hear it. I have heard it preached from the likes of spokespeople, teachers, parents, mentors, and so on. I soon learned that this is not always the case, and in fact, it can be a hindering concept to believe in.

I recently watched a video of a commencement speech by Mike Rowe, best known as the host for the television series Dirty Jobs, given to PragerU graduates. In five minutes, he explained why he thinks following your passion is terrible advice. The main idea of his speech, what he referred to as “The Dirty Truth”, is this; just because you’re passionate about something, doesn’t mean you won’t suck at it [1].  This implies that the hard work and effort that you put into your passion, does not necessarily mean you will obtain a successful lifestyle in that passion.

He goes on to describe the ineffectiveness of telling someone to follow their dreams, since it may not actually be clear as to what they are dreaming, and if that dream is feasibly attainable. The fact remains that passion and ability are two separate things.

To be successful, Rowe explains how following opportunity will more likely lead to prosperity, and that following passion can mean missing out on plenty of available opportunities. He uses the “skills gap” to illustrate his point; there are plenty of jobs that are available to which very few people are trained to do. This compares to the large number of people who are skilled at certain jobs, to which have few openings. Imagine how successful you could be if you went against the current and followed the path less beaten?5863884809_7dcbcea2e5_m

This was, personally, a very eye-opening concept that I had never really thought about. For most of my life, I was under the impression that hard work and perseverance meant that you could do anything you wanted. However, I believe much of what Rowe said is very true.

Now, this is not to say that following one’s passion is completely illogical; it’s not impossible to be successful at your dream. It is, however, important to gauge the pay-off towards your goal, versus how much effort, time, and money is going into it. To paraphrase Mike, staying the course only makes sense if you’re headed in a sensible direction. Learn to pick your battles, and know when to tap out of the ring and take on a different battle that you believe can be concurred.

So now what? Instead of following passion in advance, what should be done instead? I decided to do some research and see what others had to say, and much of it was a relative reiteration of Rowe’s speech.

In an interview, Cal Newport, an author and 30-year-old assistant professor of computer science at Georgetown University, advocated for cultivating your passion, instead of following it. In essence, pay-off is greater when one decides to build a passion for their job. This is done by sharpening your skills and abilities within your work like a craftsperson, then shaping your working life towards the lifestyle that you desire [2].

Here are a few guidelines to help navigate you through your passion cultivation in the job market:

  1. Understand what skills and activities you excel at. Are you a good listener, or problem solver, or have always taken the leadership role in a group setting? What are you good at which you believe has been impactful or has made you particularly happy at one point?
  2. Understand where your skills can be applied. In what jobs and industries would you be able to hone your skills like a craftsperson, and gain more opportunities as a result of your improvement? Think broadly about your possible opportunities. For example, I love using Microsoft Excel, so I know there are a number of positions that would require proficiency in Excel than just Data Analysis jobs.

Seeing that it is convocation season for many University students, this might have some valuable insight for a number of graduates. Some will be entering the workforce, and may have hopes of landing that dream job, or pursuing that one thing they are passionate about. The real takeaway point: keep a mindful and realistic goals, and allow your passions to drive you throughout the journey. Take the time to understand if your passion is really what you’re meant to do, or if it is just meant to be a dream to hold on to.

Sources:

[1] Mike Rowe – Don’t Follow Your Passion

[2] Joshua Fields Millburn – ‘Follow Your Passion’ Is Crappy Advice

[3] Nathaniel Koloc – Why “Follow Your Passion” is Pretty Bad Advice

[4] Lauren Friese –  Gen Y career advice: Don’t follow your passion. Do this instead.

What do students value in online courses? — Dina Meunier, Centre for Extended Learning

CAUCE CNIE logoWhat do students value in online courses?

I had the pleasure of attending the 2016 CAUCE – CNIE joint conference here in Waterloo from May 30 to Jun 2. There were many interesting sessions, including a keynote by Marc Rosenberg describing Learning Ecosystems and another by Ken Steele highlighting the latest innovations in teaching and learning. A panel interview led by the Ontario Ministry of Training, Colleges and Universities examined how provincial consortia, including our own eCampus Ontario, are promoting an environment of innovation in higher-education technology-enabled learning.

One session I particularly enjoyed presented the results of an online student survey at Wilfrid Laurier University. WLU surveyed students who had recently taken one or more fully online courses and asked them what they valued most in an online learning experience. The top three responses related to online course design were:

  • a well-organized course syllabus;
  • clear course expectations and requirements; and
  • a well-organized course structure (for example, information presented in manageable chunks, segments or modules).

As far as online teaching was concerned, students most valued:

  • fair and consistent grading of assignments and exams; and
  • clear and meaningful feedback on assignments and exams.

It seems to me that these values apply equally to face-to-face, on campus courses and to blended courses as they do to fully online courses. In fact, these items are closely aligned with Chickering & Gamson’s seminal work, Seven Principles for Good Practice in Undergraduate Education. Furthermore, the results of this survey remind us that overall, students are searching for high quality in online education, a recommendation clearly articulated by the Ontario Undergraduate Student Alliance in their recent policy brief:[1]

“At the heart of this policy is a focus on the quality of online learning.  It is crucial that the same standards of quality that apply to traditional, in-classroom courses apply to fully-online courses as well. Ideally, instructors should be capable of teaching an online course as effectively as they would a traditional class.”

But how do you know if your course structure is well-organized or if you have clearly articulated course expectations online? Here are 3 suggestions:

Are there other ways you can ensure quality in your online course design and in your teaching? Share your thoughts by leaving a comment below.

[1] Ontario Undergraduate Student Alliance (March 2016). Policy Briefing: Online Learning.

Dina Meunier is Associate Director of Online Learning, Centre for Extended Learning, University of Waterloo