Project Info

Summary of Main Goals

  • Incorporate a computer-based laboratory experience into Calculus II instruction to connect calculus concepts to disciplinary STEM practice through:
    • analysis and interpretation of results of numerical simulations
    • exposure to methods from mathematical modeling
  • Provide a rich learning experience to complement traditional knowledge transfer in a mathematics classroom, including:
    • team-work that takes advantage of diverse experiences and expertise, where the group output is greater than the total of individual contributions
    • active learning with emphasis on problem-solving using mathematics outside traditional classroom contexts, in a setting that more closely mirrors actual practice in STEM careers
  • Assess qualitatively the impact on student learning, with emphasis on discovery of student engagement along the emotional, behavioral, and cognitive dimensions
    • Specifically: while data concerning traditional learning outcomes (grades on standardized summative assessments, DFW rates) are gathered to inform us of any possible trade-offs, the project does not prioritize this dimension
  • Discover and document logistical requirements for implementation, and their impact on student learning, to gather data for future developments in this and other courses.

Project Description

We explore the logistics for and impact of including computational, team-based laboratory experiences into mainstream calculus instruction at a large public university, with the goal of supplementing (or partially supplanting) the traditional knowledge-transfer instructional model with rich experiences.

This project is motivated by the present cultural upheaval concerning the purpose of mathematics instruction, specifically those classes traditionally labeled as “service” or “general education” courses (as opposed to courses designed primarily for mathematics majors). For classes such as the mainstream calculus sequence1 the majority of whose enrollment come from students intending to major in the physical sciences or engineering disciplines, the traditional emphasis on knowledge-transfer and rote calculation is increasingly losing its relevance outside of classroom settings. In many of such classes, the main learning goal has long been reduced to fluency in pen-and-paper computations, a skill rapidly obsoleted by modern technological innovations and is now de-emphasized in recent policy recommendations2. Simultaneously, there is a growing consensus toward the importance of interpersonal and transferable skills training3 in higher education. On a local level, in addition to this cultural shift slowly taking root among the mathematics faculty, we also face external pressure and incentive to revisit our instruction of the mainstream calculus sequence. Specific feedback from Engineering college colleagues brought to light several deficiencies in how we approach calculus instruction. The University, on the other hand, initiated a push for interdisciplinary research programs surrounding the use of computational mathematics in other STEM disciplines, bringing the inclusion of computational methods in STEM coursework also to the spotlight.

Our project is a three-year (six-semester) initiative aiming toward the large-scale deployment of computational labs into Calculus II instruction. We are to design new curricular material, pilot their deployment (first in small classroom settings ramping up toward large lectures), and assess its benefits (to students) and costs (to the department). The material developed and the data gathered will form important local data to inform the ongoing discussion within our department to update our instruction of gateway and service courses.


  1. Typically several-semester long, with content including integration and differentiation in one variable, applications of single-variable calculus, sequences and series, multi-variable calculus, introduction to differential equations. ^
  2. For example, the reports of the Curriculum Foundations Project. See also Ganter, S. and Barker, W. (Eds.) 2004, The Curriculum Foundations Project: Voices of the Partner Disciplines. Washington, DC: Mathematical Association of America; and Ganter, S. and Haver, W. (Eds.) 2011. Partner Discipline Recommendations for Introductory College Mathematics and the Implications for College Algebra. Washington, DC: Mathematical Association of America. ^
  3. These skills are frequently referred to under the umbrella of “21st century skills”; see National Research Council. (2012). Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Committee on Defining Deeper Learning and 21st Century Skills, J.W. Pellegrino and M.L. Hilton, Editors. Board on Testing and Assessment and Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. ^

Project Findings

PRIMUS report

Our experience implementing this project is detailed in our PRIMUS article:

  • Andrew J. Krause, Ryan J. Maccombs & Willie W.Y. Wong (2020) Experiencing Calculus Through Computational Labs: Our Department’s Cultural Drift Toward Modernizing Mathematics Instruction, PRIMUS, DOI: 10.1080 / 10511970.2020.1799457 (the version of record)

You can read the accepted version of the author manuscript here.

Acknowledgements

Funding

This project was supported in part by a generous grant from MathWorks through their MathWorks Academic Support program.

Special Thanks

The PIs would also like to thank Dan Normand, Arman Tavakoli, and Rocco Tenaglia—the TAs for Pilots 3 and 4 of the project—for their many observations and feedback. Additionally, the TAs for Pilots 5 and 6 of the project have our gratitude for their patience and gusto teaching in this new and unfamiliar format.

People

Team Lead

Avatar

Willie WY Wong

Assistant Professor

Avatar

Mark Iwen

Associate Professor

Team Members

Avatar

Andrew J Krause

Instructor

Avatar

Craig P. Gross

Graduate Assistant

Avatar

Yilin "Erika" Zheng

Undergraduate Assistant

Avatar

Rachael M Lund

Instructor

Avatar

Ryan J Maccombs

Specialist

Avatar

Shih-Fang Yeh

Graduate Assistant

Project Timeline

 
 
 
 
 

Project start

Oct 2015 - Jan 2016
Initial approach by the Chair (Keith Promislow) to Mark Iwen and me for development of engineering specific calculus branch that includes more motivation of the course material by real-life examples and incorporates numerical/computational aspects. Early meeting with Engineering College undergraduate studies committee to establish basic parameters, identified ‘sequence and series’ as one particular problematic area.
 
 
 
 
 

Information gathering

Feb 2016 - Mar 2016

Mark Iwen and I produced an initial proposal of changes that we intend to implement. We reached out to individual units within the Engineering College, asking to meet to discuss our proposal. The goal is to set expectations and clearly establish parameters for our project, and to gather ideas for the lab projects that will be used in the course.

Outcome of the meetings: our mandate requires us minimizing changes to the “calculus content” of the course, while including hands-on exploratory labs based on real-life settings with computational aspects using the Matlab programming language. A gradual ramp up of the enrollment is agreed upon, and the AY2016-2017 edition of the pilot program will enroll only selected engineering major students recommended by the college advisors.

 
 
 
 
 

Initial lab development

Jun 2016 - Nov 2016
First round of development of lab material. For deployment we used JupyterHub (administered by DECS of the Engineering College). The Jupyter Notebook is an established web-aged interactive computational environment, allowing us to mix narrative text, executable computer code, mathematics, illustrations, and other rich media in a unified document. Matlab interoperability is provided by third-party, open-source software. This allowed us to deploy the labs to students in a cloud environment without needing to worry about the logistics of installing Matlab on individual laptops.
 
 
 
 
 

Pilot #1

Aug 2016 - Dec 2016

MTH133 s62 has enrollment restricted only to incoming freshmen with AP credit, with a declared interest in the engineering majors, and recommended by their college academic advisors. Mark Iwen and I collaborated with Rachael Lund for running this course, which consisted of three 50-minute “traditional” lectures (using the black board) and one 2-hour-long lab per week. Each lab session involved first a “traditional quiz” on standard content material, followed by 80 minutes of computer-based activity.

This course can be described as a “standard Calculus II class” augmented with lab material.

We collected student feedback concerning labs and overall structure for this first pilot through weekly surveys.

 
 
 
 
 

Pilot #2

Jan 2017 - Apr 2017

MTH133 s37 has enrollment restricted only to engineering students recommended by their college academic advisors. We again co-taught with Rachael Lund. Course structure is the same as Pilot #1, with the addition of more “real-life” examples presented during lecture.

To facilitate the latter, we partially flipped the course. Students are expected to watch short course videos (prepared by Ryan Maccombs of the Math Department independently of this project) prior to coming to class. Class starts with a 5-minute quiz on basic concepts covered by the video, and continues with motivational material and examples.

Lectures are presented in mixed media: black board, overhead projector, and slides.

Student feedback was collected through weekly surveys.

 
 
 
 
 

WeBWorK video integration, lab redevelopment

Jun 2017 - Aug 2017

Additional changes are inspired by observations that:

  • The 2-hour-long lab sessions are decided to be impractical and not sustainable.
  • Some success were observed for the flipped lecture format of Pilot 2.

We revisited the labs, with the main aim of shortening them to be approachable in 60 minutes. Additionally, to facilitate transition to larger lecture sizes and still maintain the partially flipped structure, we developed WeBWorK “reading check” quizzes to be completed after the students watched the video but prior to arriving to class. The graded work incentivizes student participation in the pre-course videos, allowing us to include “engineering applications” in lecture without sacrificing curricular material.

 
 
 
 
 

Pilot #3

Aug 2017 - Dec 2017
MTH133 s40 and 41 again have enrollment restricted only to engineering students recommended by college advisors. In preparation for ramp-up in student participation, this time Mark Iwen and I lead the lectures, while the labs/recitations are run by two graduate student TAs. Some logistical experiments:

  • Lab sessions are reduced to 80 minutes in length total, with first 15-20 minutes dedicated to a weekly quiz, and the remainder to the lab.
  • To facilitate labs, instead of the typical recitations where one graduate TA handles 26-32 students, we allow enrollment of up to 45 students per section, and have both graduate TAs present during labs. This change is based on our experience running the labs ourselves in the first two pilots.
  • Lectures are presented using “write-on” slides. Completed slides are made available to students after lecture.
  • For my lectures I experimented with “untethered lecturing” technology: I carry a tablet computing device, wirelessly connected to the podium and to the projector, and lectured while I roamed up and down the aisles in the classroom. The intent is that this could break some invisible barrier and improve student engagement with the lectures.
  • To collect student feedback, we piggy-backed on the calculus pre-/post-course surveys that the math department runs, and gathered informal student comments.

 
 
 
 
 

Pilot #4

Jan 2018 - May 2018
MTH133 s40 and 41 are run largely in the same form as Pilot #3, except that our experience and student feedback concerning the write-on slides presentation method for the lectures suggested that it was a failure. We reverted to presenting the course on blackboard.
 
 
 
 
 

WeBWorK data analysis

Feb 2018 - Apr 2018

With the help of an undergraduate student assistant, Katrina Gensterblum, I did some statistical analysis of student performance data gathered on the WeBWorK platform. We made various preliminary findings some of which confirmed and some of which refuted conventional wisdom concerning student engagement with WeBWorK. In the context of sections 40 and 41 (Pilot #3), however, our main findings are that student participation rates on the video check questions that Mark and I developed in the summer of 2017 are low. Additionally, using as a metric the average number of attempts before reaching the correct answer, the video check questions appear to be overwhelmingly harder than the standard homework questions.

We have several hypotheses to explain this:

  1. The video check questions tend to be more conceptual rather than computational, and this may indicate students engaging with the course material at a lower level (in terms of Bloom’s Taxonomy) than one would hope.
  2. The computational portions of the video check questions are often very similar modifications of examples presented in the course videos; this could indicate that the expectation that students learn by modelling their responses after presented examples is incorrect.
  3. For the standard homework questions, the students have access to tutoring services provided by the Math Learning Center offered by the Math department. We’ve asked the students not to bring video check questions to the MLC.
  4. Perhaps the students don’t watch videos linearly, and with the perverse incentive of the WeBWorK questions only peck at the video to find the part that answers the questions.

We are forced to conclude that our pre-class video structure is not appreciatively improving student learning, and may actually contribute more to student frustration with the class.

 
 
 
 
 

Redevelopment of course

May 2018 - Aug 2018

In preparation for the large-scale deployment of Pilots #5 and 6 (150 students each semester), we decided to modify the course yet again based on what we learned in Pilots #3 and 4. During the summer of 2018 we rebuilt our curricular material to reflect some structural changes that we will be making.

Logistic changes:

  • Course will have the same amount of face-time as traditional sections. Three lectures per week at 50 minute each plus one lab/recitation at 50 minutes.
  • Lab will be supervised by one GTA but enrollment capped at 27.
  • Instead of weekly combo of quiz+lab, restructured the lab/recitations so that during 7 of the weeks the entire 50 minutes are devoted to labs, and during 5 of the weeks the whole recitation is devoted to summative assessment of student learning.
  • Course will no longer be flipped. To promote active learning and hands on practice, during lectures the first 35-40 minutes the lecturer will present the course material, and the final 10-15 minutes is given to a “group quiz”, where students are allowed to collaborate in groups of up to three students. The intent is for this to be a condensed think-pair-share like activity, with the main pedagogical purpose for students to discover whether they have sufficiently understood the course material to apply it to problem solving (and if not, they are welcome to ask the lecturer for more explanation).
  • Course will no longer be restricted to engineering students.
  • Funding was secured through MathWorks Academic Support to collect data about student engagement in the course and evaluate impact.

About the summative assessments:
Instead of weekly 15-20 minute quizzes, we run “mini-tests” that last 40 minutes every two to three weeks. The mini-tests are designed to be exactly half the length of a midterm exam, with exactly the same formatting and grading style. The intention is to borrow from backwards design principles: for traditional sections the three types of student assessments (WeBWorK homework, quizzes, and exams) are formatted and graded differently. While there was not much we can do about the necessary evil that is WeBWorK, the least we can do is to replace the quizzes by mini-tests that give students lower-stakes accurate preparation for their exams.

About the labs:

  • The number and length of the labs are reduced, drawing from student participation data in previous years, to fit the new schedule.
  • Labs are now rewritten as Matlab LiveScripts, which was first introduced by MathWorks in their 2016a release of Matlab, about which we were only aware after our initial commitment to using JupyterHub. By 2018 the LiveScript functionality has matured significantly and offers a viable and well-integrated replacement for Jupyter Notebooks. Additionally the availability of the Matlab Online product which is included as part of the campus-wide subscription to Matlab also solves the potential problem of supporting student installation on laptops.

Evaluation of program: Program assessment is designed and implemented by Andy Krause, and includes:

  • Observation (video-taped) and analysis of student engagement with lab material.
  • In-person interviews with student volunteers.
  • Additional pilot-specific questions included into the departmental pre-/post-course survey for calculus classes.

Additional informal feedback from students are collected through an anonymous comment box.

 
 
 
 
 

Pilot #5

Aug 2018 - Dec 2018
MTH133 sections 16-20 and 23 are run according to the outline above. Lectures are given using chalk and blackboard.
 
 
 
 
 

Pilot #6

Jan 2019 - Apr 2019
MTH133 sections 22-28 are run according to the outline above. Due to material constraints of the lecture hall, the lectures are given using digital overhead projector in lieu of blackboard.
 
 
 
 
 

Lab videos, data analysis

May 2019 - Aug 2019

One of the student feedbacks that we received for this new format is that many students wish that they can “find out the correct answer” for the labs. The labs are designed to be supplementary to student learning, with the emphasis on the process and not the results. Therefore “correct answers” were not made available. Students however indicated that knowing what they should be doing and seeing the activities demonstrated could bring them emotional closure and satisfy their curiosity. Therefore we decided that an additional task is to prepare videos that address these issues. It was also decided that these lab videos will make good training tools for graduate TAs implementing the labs.

Analysis of the collected data will also be done, in preparation of publication.

 
 
 
 
 

Full deployment

Aug 2019
Starting Fall of 2019, the labs will be deployed uniformly across all Calculus II sections at Michigan State University during the Fall and Spring semesters. (Logistically it is more complicated to deploy during the Summer session.) We will continue to gather student feedback and make improvements to the program, as well as produce additional labs.

Contact

  • +1(517)353-8146
  • 619 Red Cedar Road, Wells Hall, Department of Mathematics, Michigan State University, East Lansing, MI 48824