Many of us have likely heard the term wrapper or cognitive wrapper used when discussing ways to help our students in becoming more independent and self-aware learners. In particular, this term comes up when discussing assessment as a learning opportunity. So what exactly is a cognitive wrapper and how can it be used to aid learning?
In brief, a cognitive wrapper is a tool to guide students before, during or after a teaching and learning event to help them identify their own approaches to the teaching and learning event and what aspects of their behavior are productive and which aspects are not. It encourages students to purposefully examine what they can and should change so as to improve the teaching and learning experience. Wrappers are a structured way to guide students through a reflective process that increases their self-awareness and leads to a modification of behavior through self-regulation.
Whenever I talk with instructors here about how my job is to support them in their writing and communication instruction, I hear some version of the same response: “My students are brilliant, but they can’t write a sentence to save their lives!” No matter whom I’m talking to, regardless of discipline, job title, teaching experience, linguistic background, educational background, or teaching load, nearly everyone has the same anxieties around the role of communication in their courses. But I’m always glad to have the chance to talk about these concerns. If you’re one of those instructors I’ve talked with about teaching writing and communication in your discipline, you’ve probably seen my eyes light up as I eagerly launch into my spiel about the research on teaching writing and communication across the curriculum.
Most often we approach the design of our main course elements – intended learning outcomes (ILOs), formative and summative assessments, and teaching and learning activities – by turning to Bloom’s Taxonomy (and most frequently the cognitive domain) to help us determine the appropriate level of thinking required and to help us express that accurately in our descriptions.
If you weren’t already aware, here are a few numbers about online math at the University of Waterloo:
The Math Faculty has been offering fully online courses since Fall 2003 and, since that time, has offered 55 unique online courses to more than 21,000 students
The Centre for Education in Mathematics and Computing (CEMC), with support from the Centre for Extended Learning (CEL) and local software company Maplesoft, was the first group on campus to release a large set of open educational resources (OERs). Called CEMC courseware, the OERs include lessons, interactive worksheets, and unlimited opportunities for students to practice skills and receive feedback. At the time of this post, the resources have received over 1.8 million hits from 130,000 unique users in 181 different countries.
Maplesoft is using a focus group from Math, CEL, and CTE to develop a new authoring environment that will specifically target the needs of online STEM course authors. It is anticipated that this tool will be released in early 2017 and, over time, should save development costs by 50%.
The Math Faculty, together with the Provost’s office, has dedicated $1.2 Million over the next three years for additional work on online projects; over 90 course development slots allocated by CEL have already been filled.
These numbers are some of the reasons Waterloo is considered a leader in the area of online math education.
From June 19 – 22, a small group from Waterloo and I joined an international team of mathematics educators to discuss digital open mathematics education (DOME) at the Field’s Institute in Toronto. Lots of great discussions happened including opportunities and limitations of automated STEM assessment tools, integrity-related concerns, and practical challenges like lowering the bar so that implementing fully online initiatives isn’t the “heroic efforts” for Faculty it’s often viewed as being today. Of all the discussion topics, however, the one that got me most excited – and that my brain has returned to a few times in the month since the conference – is using Comparative Judgement (CJ) in online math courses.
The notion behind CJ is that we are better at making comparisons than we are at making holistic judgments, and this includes judgments using a pre-determined marking scheme. It doesn’t apply to all types of assessments, but take this test on colour shades to see an example of how using comparisons instead of holistic rankings makes a lot of sense. Proof writing and problem solving may also lend themselves well to CJ and three journal articles are listed at the end of this blog for those who would like to read more.
Here are some of the questions I’ve been pondering:
Are there questions we aren’t asking students because we can’t easily “measure” the quality of their responses using traditional grading techniques? How much/when could CJ improve the design of our assessments?
Example: Could CJ, combined with an online CJ tool similar to No More Marking, be used by students in algebra courses as a low-stakes peer assessment activity so students could see how different proofs compare to one another? Perhaps awarding bonus credit to students whose proofs were rated in the top X%.
Which Waterloo courses would see increases in reliability and validity if graders used CJ instead of traditional marking practices?
How much efficiency could Waterloo departments save if high-enrolment courses used CJ techniques instead of marking schemes to grade exam questions or entire exams? Could CEMC save resources while using CJ to do their yearly contest marking?
I don’t have answers to any of these questions yet, but my brain is definitely “on” and thinking about them. I encourage you to read the articles referenced below and send me an email (email@example.com) If you like the idea of CJ, too, or have questions about anything I’ve written. If you have questions about Waterloo’s online math initiatives, you’re welcome to email me or Steve Furino.
Jones, I., & Inglis, M. (2015). The problem of assessing problem solving: can comparative judgement help? Educational Studies in Mathematics, 89, 3, pp. 337 – 355.
Jones, I., Swan, M., & Pollitt, A. (2014). Assessing mathematical problem solving using comparative judgement. International Journal of Science and Mathematics Education, 13, pp. 151–177.
Pollitt, A. (2012). The method of Adaptive Comparative Judgement. Assessment in Education: Principles, Policy, & Practice. 19, 3, pp. 281 – 300.
Presenters at CTE’s recent Teaching and Learning conference explored the theme of Learning from Challenge and Failure. As a follow-up to the Conference, we’d like the share the following list of compiled resources:
The dream of creating a device that can replicate human behaviour is longstanding: 2500 years ago, the ancient Greeks devised the story of Talos, a bronze automaton that protected the island of Crete from pirates; in the early thirteenth century, Al-Jazari designed and described human automata in his Book of Knowledge and Ingenious Mechanical Devices; in the eighteenth-century, the clockmaker Henri Maillardet invented a “mechanical lady” that wrote letters and sketched pictures; and in 2016, Ashok Goel, a computer science instructor at Georgia Tech, created a teaching assistant called Jill Watson who isn’t a human – she’s an algorithm.
Goel named his artificial teaching assistant after Watson, the computer program developed by IBM with an ability to answer questions that are posed in ordinary language. IBM’s Watson is best known for its 2011 victory over two former champions on the gameshow Jeopardy! In Goel’s computer science class, Watson’s job was to respond to questions that students asked in Piazza, an online discussion forum. Admittedly, the questions to which Watson responded were fairly routine:
Student: Should we be aiming for 1000 words or 2000 words? I know, it’s variable, but that is a big difference.
Jill Watson: There isn’t a word limit, but we will grade on both depth and succinctness. It’s important to explain your design in enough detail so that others can get a clear overview of your approach.
Goel’s students weren’t told until the end of the term that one of their online teaching assistants wasn’t human – nor did many of them suspect. Jill Watson’s responses were sufficiently helpful and “natural” that to most students she seemed as human as the other teaching assistants.
Over time – and quickly, no doubt – the ability of Jill Watson and other artificial interlocutors to answer more complex and nuanced questions will improve. But even if those abilities were to remain as they are, the potential impact of such computer programs on teaching and learning is significant. After all, in a typical course how much time is spent by teaching assistants or the instructor responding to the same routine questions (or slight variations of them) that are asked over and over? In Goel’s course, for example, he reports that his students typically post 10,000 questions per term – and he adds that Jill Watson, with just a few more tweaks, should be able to answer approximately 40% of them. That’s 4000 questions that the teaching assistants and instructor don’t have to answer. That frees up a lot of their time to provide more in-depth responses to the truly substantive questions about course content.
More time to give better answers: that sounds like a good thing. But there are also potential concerns.
It’s conceivable, for example, that using Watson might not result in better answers but in fewer jobs for teaching assistants. Universities are increasingly keen to save money, and if one Watson costs less than two or three teaching assistants, then choosing Watson would seem to be a sound financial decision. This reasoning has far broader implications than its impact on teaching assistants. According to a recent survey, 60% of the members of the British Science Association believe that within a decade, artificial intelligence will result in fewer jobs in a large number of workplace sectors, and 27% of them believe that the job losses will be significant.
Additionally, what impact might it have on students to know that they are being taught, in part, by a sophisticated chatbot – that is, by a computer program that has been designed to seem human? Maybe they won’t care: perhaps it’s not the source of an answer that matters to them, but its quality. And speaking for myself, I do love the convenience of using my iPhone to ask Siri what the population of Uzbekistan is – I don’t feel that doing so affects my sense of personal identity. On the other hand, I do find it a bit creepy when I phone a help desk and a ridiculously cheery, computerized voice insists on asking me a series of questions before connecting me to a human. If you don’t share this sense of unease, then see how you feel after watching 15 seconds of this video, featuring an even creepier encounter with artificial intelligence.
With constant media stimulation, increase in competitiveness, and stress overload, “Is it possible to slow down” (1)? Our culture can be self-driven and individualistic so it is no surprise that for many, time is a finite resource that is draining away. As a result, we try to do as much as we can in a very short time period. Our minds are filled with constant distraction, thus limiting opportunities for self-reflection to ask oneself “Am I well or am I happy?” (1).
We’d like to believe that we have been a good friend, partner, or child at various points in our life. However, upon remembering that significant person in your life, do you know or have you ever asked what were the moments when they were the happiest? The times when they were crying from tears of joys to the time when they felt the most accomplished? Surprisingly for many, we are unaware of these stories that ultimately define whom that individual has become today. We mindlessly pass every day without pondering about the conversations that we had or the connections that were made. By simply being mindful of the questions that we pose, more specifically “questions that people have been waiting for their wholes lives to asked … because everybody in their lives is waiting for people to ask them questions, so they can be truthful about who they are and how they become what they are,” as beautifully said by Marc Pacher (2).
So what is the action plan?
1.Invite people to tell stories rather than giving answers. Instead of “How are you” substitute
What’s the most interesting thing that happened today?
What was the best part of your weekend?
What are you looking forward to this week? (3).
2. Enter a conversation with the willingness to learn something new
Celeste Headlee in her TED Talk 10 Ways to Have a Better Conversation describes how she frequently talks to people whom she doesn’t like, and with people whom she deeply disagrees yet is still able to have engaging and great conversations. She is able to do this as she is always prepared to be amazed and she seeks more to understand rather than to listen and state her own opinion and thoughts.
3. Lastly “being cognizant of [your] impact is already the first step toward change. It really does start at the individual level” my friend once said (5).
Brene Brown in her Power of Vulnerability talk said, “Many pretend like what we’re doing doesn’t have a huge impact on other people”. But we’d be surprise of what we are capable of when you allow yourself to be vulnerable as this “can be the birthplace of joy, of creativity, of belonging of love… the willingness to say, “I love you” the willingness to do something where there are no guarantees” (6).
That being said, you don’t have to be the most intellectual or outspoken person in the room, but what is key is the willingness to be open and the questions that are posed. There are many simple things that can be easily integrate into our daily lives, by being more mindful of the question that we ask to ultimately have a more memorable and enriching conversation. In the end it is to have better connections, new understanding and awareness to savor the moment.
At CTE, Microteaching Sessions are offered where you can choose from various topics to conduct an interactive teaching lesson. For my first topic I will be talking about the importance of communication. All participants will not only be giving feedback but will receive constructive feedback and ways to improve from knowledgeable facilitators. It’s a safe environment where you have the chance to present to fellow graduate students from various departments. Many have found these sessions beneficial as you are working on skills relevant to work, field of study or for your own personal growth. I am excited and nervous for this opportunity to talk about something I am passionate about and I hope I can successfully engage others and deliver the content well. In order to help participants formulate an effective teaching plan, the Centre for Teaching Excellence website has provided many resources such as well written guidelines, lesson plans outlines, and facilitators review the lesson before you present.
As a follow-up post, I had the chance to facilitate an hour session for an AIESEC conference for participants from various universities such as Toronto, Waterloo, Laurier, and York, that recently returned from their international exchanges. There were lots of discussion so thank you to the Graduate Instructor Developers, Charis Enns and Dave Guyadeen, and Instructional developer, Stephanie White for their great feedback and helping me make this session more successful!