Transitioning between on-campus and online learning environments, and its effect on student learning

My own disciplinary background is in second language development, and specifically, German language learning. Over the last decade or so, as online and blended learning has proliferated, languages have sought ought means by which to teach a foreign language online using educational technology. This may seem like a difficult task – how do you teach a language when the very act of speaking the language and practicing pronunciation is challenging, if not outright impossible. And indeed, this is a challenge. As learning management systems develop and improve, this challenge has been circumvented to some extent by at least allowing asynchronous communication between learners to occur, but nevertheless, the reality is that online language learning is often seen to be at a disadvantage compared to on-campus, traditional classroom-based language learning.

With this in mind, Mat Schulze, Professor in Germanic and Slavic Studies, Sara Marsh, MA student in Germanic and Slavic Studies, and myself undertook a research study as part of a LITE grant (read more about it here) to investigate how transitioning between online and on-campus language learning environments may impact student learning. Online learning is no longer just for students who are physically not on-campus; students may be on a co-op term and want to continue their studies, or they may find their schedules already packed and want to alleviate things a bit. Now more than ever, students have a real choice as to how they want to learn, and especially with introductory and intermediate German language courses (specifically GER 101, GER 102, GER 201, and GER 202) they can take these either on-campus or online and the content is largely the same, although activities and assessments vary to some degree.

By looking at enrollment and grade data from the past decade of German language learners at the University of Waterloo (n=6920), as well as surveying (n=157) and interviewing (n=24) current students enrolled in German language courses, we were able to capture learning trajectories of students and determine what happens to learners who transition between learning environments as they progress through the German language program, as well as general attitudes that language learners have of on-campus and online language learning.

Here are some of our data-driven findings:

  • Of the 5906 students who took GER 101, 102, and 201, 44% took it online
    • Many students are benefitting from taking German language courses online
  • Only one third of students who take GER 101 continue on to take GER 102
  • Learners who elect to only take one language course to fulfill breadth requirements are attracted to online course offerings over their on-campus equivalents
    • Students tend to believe that the online course would be easier than the on-campus version
  • Most students enrolled in the German language program take their language courses in the same environment (74.5%)
    • Therefore 25.5% of language learners learn through a combination of online and on-campus language learning environments
  • Online students are more likely to continue learning in the same environment
    • This is again likely due to the perceived notion that online courses are easier than the on-campus equivalent

Reading more into our data, we found the following as well:

  • Both online and on-campus language courses prepared students sufficiently for the subsequent German language course
  • Most students however held negative or ambivalent views of online learning, feeling it is inferior to the on-campus classroom environment
    • Yet these views primarily come from students who have never taken an online language course
  • Students who tend to take both online and on-campus language courses are more motivated language learners
    • They seek out any opportunity to continue studying German, even if it means it is done in the online learning environment as to avoid having a one or two-term gap between courses
  • Perhaps more importantly, students with higher motivation to continue tend to be more successful, and students with higher performance and better grades are more likely to continue studying languages

Three additional themes emerged:

  • Students desire personal contact, and tend to believe that online interaction was NOT personal contact
    • Communicative interaction is being conceptualized as only spoken or face-to-face, and although there is not an overwhelming amount of one-on-one instructor to student interaction in the on-campus learning environment, it is idolized
  • Feedback in the online environment was found lacking or insufficient due to delays, or believed to not exist at all
    • Delays in feedback received in-class were not brought up, seemingly considered acceptable as opposed to similar delays online
  • Self-regulation of learning and motivation impeded learners’ beliefs about their ability to succeed in the online environment
    • Completing tasks and assignments at the last minute may have caused learners to believe they were not learning as successfully as in the on-campus classroom

Of great interest as well were what the students themselves thought about transitioning between online and on-campus learning environments. Students who went from an online to on-campus course focused on shifts in workload and the social interaction that accompanied an in-class learning context. Interestingly, none of the learners who were interviewed discussed anxiety about switching to on-campus instruction or felt that it affected their learning (either negatively or positively).

What does this all mean? Too few students learn in both environments, and there may in fact be some benefits to do so. Chief among these is the natural focus that each environment places on the type of learning that occurs. In the on-campus environment, the spoken language receives ample attention, but the online environment focuses so much on written communication that it can be incredibly beneficial to dedicate an entire course to the online environment. We also need to invest in creating sustainable, synchronous communication channels available and fully integrated into the online course curriculum in order to address the perceived deficiencies with online language learning. Finally, although these results are specific to the German language learning context, we believe these results are transferable to other language programs when deciding how best to offer online language courses in conjunction with traditional on-campus offerings.

Kyle Scholz

Faculty of Arts and University Colleges Liaison

More Posts - Website

UW STEM Education and Mobius – Paul Kates

Back in August 2016 my colleague Tonya Elliott from the Center for Extended cemc university of waterlooLearning wrote a post on Online Math Numbers at Waterloo, and Comparative Judgments as a Teaching Strategy in this space.

In that piece Tonya talked about the thousands of UW students who have taken online UW math courses and of the recognition of excellence received for the educational resources in these courses.  Today I present more information about the new online environment named Mobius first introduced there.   This platform offers authors and students expanded opportunities for rich, interactive learning.

As examples are three UW-Mobius project collaborations (CEMC, CEL) with the evolving Mobius system available free to the public:

uw open math mobius site
http://open.math.uwaterloo.ca
cemc university of waterloo
http://courseware.cemc.uwaterloo.ca

 

 

 

uw open eng mobius
http://open.engineering.uwaterloo.ca

To get a feel for what Mobius can do I’ll describe and link to specific features from each of these sites.

In the Chemistry for Engineers course you will see short (approx 5 minutes) narrated videos and animation.  Live self-check concept questions using the ordering question type let students know which part of their answers are right or wrong. Other locations in the course make use of the Maple mathematical engine underlying Mobius allowing students to check their skill at doing calculations.  These questions provide a motivating hint if students feel unsure.

The Linear Algebra1 1 Open Math site is designed differently, offering longer 20 minute presentations alternating with live quizzes.  Each quiz question has its own template for generating tens or hundreds of different question variations giving students the chance to repeat and master the material.

The example chosen from the CEMC site is an interactive demonstration of the cross product of two vectors.  On screen controls allow students to manipulate the size and orientation of two vectors while displaying the vectors and their cross product.  This is an example of a Mobius math app.  Math apps are great for letting students visualize concepts, experiment with dynamic objects and explore what-if questions.

UW has created thousands of questions for use in our Math, Physics, Chemistry, and Engineering courses.  They are all freely available to use in any course you teach.  As are the many Math apps on the Maplesoft Math app gallery page and on the Maplesoft shared content Maplecloud web site.

If you are curious about Mobius and want to learn more there is a hands-on seminar in two weeks (Thu Mar 2 11:15 AM), part of CTE’s very popular EdTech week.

You don’t have to wait two weeks though.  If you have an idea for a Math app and want help realizing it, want to browse through the question banks, want to see how a lesson is created or just want to play around with Mobius then please get in touch.

Paul Kates
Mathematics Faculty CTE Liaison
pkates@uwaterloo.ca, x37047, MC 6473

Paul Kates

Paul Kates

Centre for Teaching Excellence (CTE) Liaison to the Faculty of Mathematics (pkates@uwaterloo.ca)

More Posts

A case study of a new approach to a blended course — Meagan Troop, Centre for Extended Learning

Musical scoreI am an Online Learning Consultant (OLC) at the Centre for Extended Learning at the University of Waterloo. As OLCs we pride ourselves on a scholarly approach to course design and, as such, 20% of my time is allotted to research. One of the research projects that I began in Winter 2016 is a case study examination of a blended learning opportunity jointly offered by Wilfrid Laurier University and UOIT. In this case, not only did I have the opportunity to conduct research, but also to teach and contribute design changes to the course being researched. Both the research and teaching dimensions of this experience have been invaluable, greatly enhancing my perspective as an instructional designer. Continue reading A case study of a new approach to a blended course — Meagan Troop, Centre for Extended Learning

Designing for the user experience — Pia Zeni and Matt Justice, Centre for Extended Learning

user-experienceWe’ve all encountered scenes like the one pictured above – you may even be looking at one outside your office window: pedestrians choosing to ignore the nicely-constructed, costly, often very pretty footpaths designed for them, and choosing instead to forge their own path.  But have you ever thought about what scenarios like this say about design?  Why aren’t pedestrians selecting the paths constructed for them? What do their choices say about the paths designers have constructed? What goal(s) motivate them to forge their own?  These are the types of questions user experience (UX) designers ask.

The picture presents a useful allegory for designers of any stripe: the idea being, of course, that if we want to design valuable things, we need to consult the needs, expectations, and yes, even wants, of our users.

Let’s translate that principle to an online learning context:  “If we want to design valuable online learning experiences for students, we need to take their needs, expectations, and yes, even wants into account.” Whether this strikes you as common sense, or fairly radical, it is a design approach that the Centre for Extended Learning (CEL) has recently adopted with our User Experience Design for Learning (UXDL) framework, an adaptation of UX Honeycomb, developed by leading user-experience advocate Peter Morville.

You can learn more about our UXDL framework and how our design process is evolving to put our users – our students – front and centre at cel.uwaterloo.ca/honeycomb. This is a new initiative for us, so we welcome your ideas, thoughts, and reflections.

Pia Zeni (pzeni@uwaterloo.ca)

Matt Justice (matt.justice@uwaterloo.ca)

 

[Photo Source: Kalve, S. (2014, September 11). Design vs UX I Nydalen. [Photo]. Retrieved from https://twitter.com/steffenk/status/510005338545074177]

VoiceThread Project: Call for Participation — Gillian Dabrowski

voicethreadAre you looking for ways to engage your students in learning? Consider partnering with the Centre for Extended Learning (CEL) and the Centre for Teaching Excellence (CTE) to pilot a new instructional tool in Waterloo’s on-campus and online classes: VoiceThread.

You may be interested in learning about the pilot if your goal is to engage students in any of the activities below:

  • Idea sharing and interaction
  • Community building
  • Social learning
  • Peer instruction
  • Critical reflection
  • Presentation practice
  • Digital literacy skills building
  • Language practice

What is VoiceThread?

VoiceThread is a media-based discussion tool. A key feature of VoiceThread is that it enables you and your students to create digital presentations and make them the centre of a discussion. Presentations can include documents, images, PowerPoint slides, audio, or video. Students attach comments to the presentation using a keyboard (text), a microphone or telephone (audio), or a webcam (video). Discussions are asynchronous, meaning students are not online at the same time.

Why use VoiceThread?

Penn State’s Use Case Introduction gives several examples of why instructors use VoiceThread:

  • On-campus, create digital presentations on difficult to comprehend concepts and processes. Students can review content multiple times and ask the instructor questions on specific slides.
  • Enable students to present knowledge and research digitally. The class benefits from exposure to a multitude of topics. The presenter benefits from practice articulating themselves verbally and peer feedback.
  • Actively engage students in online lectures by prompting them to comment on specific slides or respond to questions posed within the presentation.
  • Increase your online instructor teaching presence and build online class community by initiating weekly kick-off discussions.
  • Create an online ‘seminar’ course experience where students grapple with heavy readings together in both written and verbal formats.

Pilot Details

The VoiceThread pilot is scheduled to run from Winter 2017–Winter 2018. Faculty who participate in the pilot will receive a VoiceThread account linked to their LEARN user account and a course site. Training and support for the pilot will be supported by CEL, CTE, and LEARN Help. Faculty participants and course participants will be asked to provide feedback via survey response, panel discussion, and interview.

If you would like to volunteer to be a part of this pilot, please contact CEL’s Gillian Dabrowski, gdabrows@uwaterloo.ca, or your CTE Liaison with the following details:

  1. Name
  2. Course information (CourseID, name, section) and expected number of students
  3. A description of how you will use VoiceThread in your course to support student engagement and assessment. How might VoiceThread help solve a problem you are experiencing with discussions or assessment as you currently use them?

More Information

References

Gao, F. & Sun, Y. (2010). Supporting an online community of inquiry using VoiceThread. In C. Maddux et al. (Eds.) Research Highlights in Information Technology and Teacher Education 2010 (pp.9-18). Chesapeake, VA: Society for Information Technology and Teacher Education (SITE).

SAM vs. ADDIE and 5 other takeaways from Madison — Tonya Elliott

madisonThe Distance Teaching and Learning Conference held in Madison, Wisconsin, has been running for 32 years and is the largest and longest-running distance education conference in the USA.

I’m writing this blog from Frank Lloyd Wright’s gorgeous Monona Terrace after 3 full days of #UWdtl keynotes, presentations, demo booths, ePosters, and discussions.  I was one of 4 Canadians who attended and presented to a room of 45 people about the online STEM/Math work we’ve been doing at Waterloo.

It will take me some time to fully digest everything from the conference, but here are 6 takeaways that immediately stood out.

1. Most instructional design models are some derivation of ADDIE (analyze, design, develop, implement, and evaluate), but ADDIE’s applicability to digital environments has been under scrutiny for some time. Other instructional design models are emerging, such as Allen’s Successive Approximation Model (SAM), shown below, and McKenney and Reeves’ problem-focused Education Design Research (EDR). Their books are referenced below and I now have two new books on my Amazon wish list!

ADDIE design model2. Learning analytics are becoming more prevalent and the potential to better understand our learners with concrete data is awesome. Learning analytics happens at three levels and each level involves both understanding what’s happening and sharing the information with students in a way that’s useful for them.  Here are the levels:

  • what students are doing today/have been doing in the past,
  • where students are likely going (using predictive modelling), and
  • where students have the potential to go/what is their optimal path.

Unfortunately, nobody at the conference had concrete examples about implementing levels 2 and 3 in any great depth, but I enjoy thinking about analytics in these three parts.

3. Some institutions spend a lot of money on remote proctoring services like Examity.  Math or other courses that can’t easily require students to complete their timed work electronically are trying out things like requiring students to position their web cameras downwards towards their papers and hands.  Unsurprisingly, privacy issues are surfacing. For example, same -sex options are now required at some institutions after female students reported being uncomfortable having unknown male proctors watching them work in their bedrooms.

4. You’re more likely to get buy-in for new initiatives if you start small. People are almost always willing to let you pilot something, and pilots can quickly and easily turn into beta versions.  If a beta version works out for a project, it’s almost always seamless to fully implement (and find funding for) it.  This path is much more efficient than trying to find approval for or fund something “big”.

5. Hooks are necessary: courses and classes should start with stories, problem questions, or other “hooks” instead of a bulleted list of outcomes.   Similarly, rather than nicely wrapping up a class, they should end with another “hook” to get students thinking about the next class.  Cognitive psychologists refer to this process as an open-loop.

6. Wisconsin’s recently launched Online Teaching Experiences site has been very well received and their site analytics reveal that the most popular part of their site is the instructor videos. I wonder if, in addition to our Instructor Community of Practice, CEL should investigate/create (digital) resources and videos for our fully online instructors.  Would this kind of resource be valuable at Waterloo? I’d love to hear our online instructors thoughts about this (so please email me your thoughts – tonya.elliott@uwaterloo.ca).

References and resources

Tonya Elliott

Tonya Elliott

In her role as an Online Learning Consultant (OLC) with the Centre for Extended Learning (CEL), Tonya Elliott provides instructional design and project management support to faculty and staff who wish to design, develop, and/or deliver fully online courses, programs, and resources. The majority of her projects are with members of the Faculty of Mathematics; however, she really enjoys working on a variety of online projects from faculty and staff from all areas of campus.

More Posts

Online Math Numbers at Waterloo, and Comparative Judgments as a Teaching Strategy — Tonya Elliott, CEL

math equationOnline Math Numbers

If you weren’t already aware, here are a few numbers about online math at the University of Waterloo:

  • The Math Faculty has been offering fully online courses since Fall 2003 and, since that time, has offered 55 unique online courses to more than 21,000 students
  • The Centre for Education in Mathematics and Computing (CEMC), with support from the Centre for Extended Learning (CEL) and local software company Maplesoft, was the first group on campus to release a large set of open educational resources (OERs). Called CEMC courseware, the OERs include lessons, interactive worksheets, and unlimited opportunities for students to practice skills and receive feedback. At the time of this post, the resources have received over 1.8 million hits from 130,000 unique users in 181 different countries.
  • In 2015, the Canadian Network for Innovation and Excellence (CNIE) recognized CEMC, CEL, and Maplesoft for their OERs through an Award of Excellence and Innovation.
  • The Master for Mathematics for Teachers (MMT) program has the highest enrolment of all the fully online Masters programs offered at the University of Waterloo. MMT and CEL staff who work on the program were one of three teams from Waterloo who won a 2016 Canadian Association for University Continuing Education program award.
  • Maplesoft is using a focus group from Math, CEL, and CTE to develop a new authoring environment that will specifically target the needs of online STEM course authors. It is anticipated that this tool will be released in early 2017 and, over time, should save development costs by 50%.
  • The Math Faculty, together with the Provost’s office, has dedicated $1.2 Million over the next three years for additional work on online projects; over 90 course development slots allocated by CEL have already been filled.

These numbers are some of the reasons Waterloo is considered a leader in the area of online math education.

Comparative Judgments

From June 19 – 22, a small group from Waterloo and I joined an international team of mathematics educators to discuss digital open mathematics education (DOME) at the Field’s Institute in Toronto. Lots of great discussions happened including opportunities and limitations of automated STEM assessment tools, integrity-related concerns, and practical challenges like lowering the bar so that implementing fully online initiatives isn’t the “heroic efforts” for Faculty it’s often viewed as being today. Of all the discussion topics, however, the one that got me most excited – and that my brain has returned to a few times in the month since the conference – is using Comparative Judgement (CJ) in online math courses.

colour shadesThe notion behind CJ is that we are better at making comparisons than we are at making holistic judgments, and this includes judgments using a pre-determined marking scheme.  It doesn’t apply to all types of assessments, but take this test on colour shades to see an example of how using comparisons instead of holistic rankings makes a lot of sense. Proof writing and problem solving may also lend themselves well to CJ and three journal articles are listed at the end of this blog for those who would like to read more.

Here are some of the questions I’ve been pondering:

  • Are there questions we aren’t asking students because we can’t easily “measure” the quality of their responses using traditional grading techniques? How much/when could CJ improve the design of our assessments?
    • Example: Could CJ, combined with an online CJ tool similar to No More Marking, be used by students in algebra courses as a low-stakes peer assessment activity so students could see how different proofs compare to one another? Perhaps awarding bonus credit to students whose proofs were rated in the top X%.
  • Which Waterloo courses would see increases in reliability and validity if graders used CJ instead of traditional marking practices?
  • How much efficiency could Waterloo departments save if high-enrolment courses used CJ techniques instead of marking schemes to grade exam questions or entire exams? Could CEMC save resources while using CJ to do their yearly contest marking?

I don’t have answers to any of these questions yet, but my brain is definitely “on” and thinking about them. I encourage you to read the articles referenced below and send me an email (tonya.elliott@uwaterloo.ca)  If you like the idea of CJ, too, or have questions about anything I’ve written.  If you have questions about Waterloo’s online math initiatives, you’re welcome to email me or Steve Furino.

References

Jones, I., & Inglis, M. (2015). The problem of assessing problem solving: can comparative judgement help? Educational Studies in Mathematics, 89, 3, pp. 337 – 355.

Jones, I., Swan, M., & Pollitt, A. (2014). Assessing mathematical problem solving using comparative judgement. International Journal of Science and Mathematics Education, 13, pp. 151–177.

Pollitt, A. (2012). The method of Adaptive Comparative Judgement. Assessment in Education: Principles, Policy, & Practice. 19, 3, pp. 281 – 300.

__________

Blackboard image courtesy of AJC1.

 

 

 

Tonya Elliott

Tonya Elliott

In her role as an Online Learning Consultant (OLC) with the Centre for Extended Learning (CEL), Tonya Elliott provides instructional design and project management support to faculty and staff who wish to design, develop, and/or deliver fully online courses, programs, and resources. The majority of her projects are with members of the Faculty of Mathematics; however, she really enjoys working on a variety of online projects from faculty and staff from all areas of campus.

More Posts