assessing standards

As promised, today I want to describe my plan for assessing the standards in my multivariable calculus class. I’ve pretty much settled on the “common standards” that I think would be appropriate for any intermediate college math class, and thanks to some feedback I’ve received since yesterday, I’m refining the list of “content-specific standards” for this class. (For some of the reasons I’m using standards-based grading in this class, see this post, or these slides by T. J. Hitchman from last week’s Joint Math Meetings.) As I see it, there are 4 issues to deal with in scoring standards:

• what scale to use;
• how to assess;
• how to re-assess;
• how to convert to a letter grade at the end of the semester.

I’m almost scared to bring up the last one, because it’s the issue that could unravel the whole process, but I’m certain my (highly driven and motivated students) will panic without it being addressed. If there are suggestions for other issues that should be ranked with these, please let me know. I’ll cover each of these briefly.

What scale I will use

I’ve seen several proposals, including the very simplest, a 2-point system for each standard. (To be fair, I think that works when the list of standards is more refined, so that very specific skills are treated separately and not clustered.) After thinking about what I believe will be the most useful to students, and based on my experience using a 3-point system, I’ve decided to score each standard out of a possible range of 0–4, with 0 indicating “complete unfamiliarity” and 4 indicating “complete mastery”. To aid the students in seeing what I expect at each level, I’ve written sentences they should be able to read and agree with when assessed at the various levels. This is another idea that I’ve borrowed from somewhere, but am having trouble finding at the moment. In my syllabus, I’m describing a standard as a set of closely related skills that represent a piece of knowledge towards mastering the class material, which should explain some of the language below.

1. “I have some idea of what this skill set and its vocabulary mean, but I don't really know how to use it.”
2. “I can complete basic exercises that involve these skills as long as I have some guidance.”
3. “I can use these skills in familiar situations with generally good accuracy.”
4. “I can use this skill set effectively and explain its significance. I can recognize when the skills are useful and apply them to both familiar and new situations.”

(I did not write a sentence for 0-level, as it would be hard for someone completely unfamiliar with a topic to muse on her understanding of it.)

How I will assess

In brief, there will be homework, two midterm exams, and a final exam. All of these will be assessed on the basis of individual standards, and each time a standard appears, its new score replaces the previous score.

I know the debate rages on about whether or not to grade homework, but because the learning time is compressed in a college class, and I do not get to see my students everyday, I think it’s important to have some way to encourage and recognize work done outside of class. That said, the homework grades will not be based on “completion”. Instead, they will provide an opportunity for students to set a “base-level” for their understanding. The report from each homework assignment will list the relevant standards and how the student’s work rates on those standards. This gives them immediate feedback, as well as a chance to see how prepared they are in advance of the exams. I suppose a student could just copy someone else’s work to inflate their scores, but I will explain that in that case their Presentation score (which is part of every assignment) will suffer; their work should be original.

Exams are larger collections of standards, integrated into a broader context. By the time a student gets to a test, she should have a good sense of which areas she will do well in, thanks to homework and earlier self-assessments. Part of the review for each test will include a list of the standards that have been covered to date and may be expected to appear. (This is another good reason for my standards to be a bit coarse, rather than drilling down to specific types of computations—it’s easier to guarantee that a test covers “parametrized curves” than “parametrizing lines”, “parametrizing circles”, “parametrizing spirals”, “checking for smooth points of a curve”, etc.) Again, I suppose a student could not have done any homework before the test and demonstrate total mastery of the material, but that outcome is not, in principle, outside of my goals for SBG.

How I will re-assess

This will be tricky to explain. For many students, tests have always been about how much they contribute to the final grade, rather than how much they say about the current level of understanding. I want to make clear that tests are important and useful only insofar as they create a rich opportunity for learning (through synthesizing the material) and showcasing one’s abilities. Whereas homework assessment is intended to establish a base level of understanding about a student’s ability from week to week, an exam provides a snapshot of her ability, and often a stressful one, at that. After the test, I want to give every student a chance to prove herself in the areas where she may have previously struggled. The experience of other teachers using SBG suggests that this not be done indiscriminately.

Thus, my policy (initially) will be to have students contact me to schedule reassessments for specific standards (during or outside of my usual office hours), at any point in the semester after a standard has been tested. This reassessment could take the form of either an oral examination or an expository presentation by the student. It is unlikely that another written assessment will be given, since I believe the obstacle is often precisely that written tests provoke anxiety. No standard can be reassessed more than once a week, and no more than three two standards can be reassessed in a week. The main point among these practical considerations is that if a student proves she has mastered a course standard, then she receives credit for doing so.

How I will convert to a final grade

This is the least important of the four issues, and yet it is the one that leaves the most lasting record. (In contrast, I hope that what leaves the most lasting overall effect is the knowledge and confidence the students gain.) I don’t want to encourage students to fiddle with a fixed formula, especially since this is my first time using SBG, but I do want to make it clear that mastery of standards is directly correlated with the final letter grade. So here’s what I’m starting with:

• In order to guarantee an A in the class, a student should attain 4s on at least 80% of the course standards and have no scores below 3.
• In order to guarantee a B in the class, a student should attain 3s on at least 80% of the course standards and have no scores below 2.
• In order to guarantee a C in the class, a student should attain 2s on at least 80% of the course standards.

This emphasizes that the goal is mastery. It is also commensurate with what one might expect of the scoring levels in any case: “mostly 4s” should look like an A/A-, “mostly 3s” should look like some form of B, etc.

The score that will be counted for each standard towards the final grade will be the average of the latest score and the highest score. That way earlier gains will not be wiped out by later retreats, but it is still important to keep up each set of skills. Because there are no opportunities for reassessment after the final exam, any prior standards that reappear on the final can only be raised by the scores on that test, not lowered.

And that’s it! That’s my plan for assessing the 20–25 standards that will finally form the basis for grading multivariable calculus this spring. Thoughts and advice are welcome.

standards for multivariable calculus, first pass

OK, it’s time to get real with this. In my last post, I explained some of my reasons for attempting to use SBG this spring and listed seven general standards for college-level mathematics classes. Now I’m listing the standards I have created specifically for multivariable calculus. There’s still time to tweak these, so I would certainly appreciate any feedback over the next few days (or, indeed, at any time!).

I have grouped the standards into five larger categories: “geometry of vectors”, “functions, curves, and surfaces”, “differentiation”, “integration”, and “classical theorems”. In some SBG implementations, the competencies listed below might be more finely sorted out, but I have come to believe that giving college students a somewhat broader classification will encourage them to guide their own education and to think holistically about the material. I have tried to sort these according to three basic principles:

• they cover roughly the same amount of course material;
• they are roughly of the same importance towards mastering the content;
• they can be more-or-less independently measured (although there are indisputably dependences among them).

During the semester, the content on which a standard is based may be introduced gradually over time. For this reason as well as the general expectation that skills should remain honed, many of the standards will be assessed several times. This is also one of my main sources of concern for confusion—what does it mean to have “mastered differentiation operators” at the level of computing partial derivatives and gradients, but not curl and divergence? If I were to distinguish these standards further simply because some parts are separated temporally, however, the number would increase two- or three-fold, making grading an intractable problem for me.

Despite these misgivings about the list itself, I feel it’s important to go ahead and publish it so that I and others can reflect on it. So here goes: (Update 1/18: Following some feedback and discussion, I have revised this list from its original form. Please ignore the parts that have been struck out.)

Geometry of vectors

• Operations – compute and interpret sum, scalar multiples, dot product, determinant, and cross product of vectors in the plane or in space
• Objects – describe points, lines, planes, spheres, and other surfaces using equations, vectors, set notation, or geometric objects of a different kind

Functions, curves, and surfaces

• Visualization – sketch or predict appearance of the graph of a function or curve based on a formula or other description; sketch or describe level sets of a function; use computer software to examine shapes of graphs
• Parametrization (added) – find parametrizations of lines, circles and other curves, as well as planes, spheres, and other surfaces (e.g., tori and graphs of functions)
• Analysis – find domain and range of functions of 1, 2, and 3 variables, and describe these using set notation or geometric terminology; determine continuity

Derivatives

• Operators – apply and interpret partial derivative, Jacobian, and gradient operators to functions; divergence and curl to vector fields
• Operations on functions (added) – compute and interpret partial derivatives and gradient of a function of 2 or 3 variables; apply and explain equality of mixed partial derivatives, including sufficient conditions for such equality to hold
• Operations on vector fields (added) – compute and interpret divergence and curl of a vector field
• Linearization – find tangent vectors, tangent lines, and tangent planes; use these to approximate curves and surfaces near a point
• Higher derivatives – apply and explain equality of mixed partial derivatives; use higher derivatives to collect data about shape of the graph of a function, including classifying critical points
• Optimization (added) – use higher derivatives to collect data about shape of the graph of a function; classify critical points; Lagrange multipliers
• Differential equations – interpret partial differential equations (such as the wave equation, the heat equation, and the Laplace equation) in terms of their solutions; verify that a given function is a solution to a PDE; use a computer to solve or approximately solve PDEs

Integrals

• Multiple integrals – accurately describe regions over which double or triple integrals are computed; perform calculations of double or triple integrals; apply and justify change-of-coordinate formulas; use computers to find integrals
• Line and surface integrals – effectively parametrize curves and surfaces, and use these use parametrizations of curves and surfaces to compute length, area, work, and flux integrals
• Applications – use double and triple integrals to solve problems in geometry and probability; use line and surface integrals to investigate physical phenomena

Classical theorems

• Integrability conditions – check conditions for a vector field to be irrotational or divergence-free (“closed” with respect to curl or divergence operators), and explain the meaning of these conditions; find potential functions
• Generalizations of FTC – explain the meaning and significance of Green’s Theorem, Divergence Theorem, and Stokes’ Theorem
• Applications – use Green’s Theorem, the Divergence Theorem, and Stokes’ Theorem to convert integrals between various forms
• Generalizations of FTC – explain the meaning and significance of Green’s Theorem, Divergence Theorem, and Stokes’ Theorem; use these to convert integrals between various forms

In my next post (probably tomorrow), I’ll explain how I plan to grade these, including what scale I’ll use, how assignments will be broken into their component standards, and how students may improve their score on an individual standard.

some common standards for college-level math

As I mentioned in my last post, this spring I will be using standards-based grading (SBG) in my multivariable calculus class. I won’t belabor what that means, since this is my first time using this method of assessment, and others have written far more expertly on the topic. In short, to me SBG means two things:

• refocusing the nature of grades from periodic goalposts to instructive feedback; and
• honing in on specific expectations so that students know where they are doing well and where they need improvement, from our perspective.

It’s often easy, for example, to become frustrated when a student does all the correct computations, but uses parentheses incorrectly. Or when he or she can differentiate polynomials but not factor them. When grading with points and percentages, we can try to leave helpful notes in the margins or at the end of an assignment, but the presence of THE NUMBER or THE LETTER at the top mostly overcomes our efforts to give informative guidance. Number and letter grades obscure the distinction between “generally understands the course material” and “understands this class well but struggles with prerequisites” and “follows set procedures neatly but has only surface understanding”. A set of standards separates out those cases and handles them differently.

After a stimulating conversation last month with T. J. Hitchman, Dana Ernst, and Jon Hasenbank (with some additional feedback from Bret Benesh), I began to rethink what should (or does, or could—there’s never been just one way to implement SBG) constitute a standard in a college/university math class. Up until that point, I had been planning to split up the course material into every possible type of computation or problem that I would expect a student to be able to complete at the end of the class. It became clear to me that this was probably too fine a gradation; it treats the material reductively rather than holistically, and moreover there’s almost no chance I could equitably test all of the pieces in a 14-week semester, with three class meetings per week.

At the time, I had already planned to have two standards appear on every assignment: algebra and presentation. These constantly trip up students at the introductory calculus level, and without SBG I always struggled to convey to certain students how much their performance would improve just by focusing on one or the other of these. During the aforementioned conversation, it was also proposed that problem solving could be its own standard. In some sense, it is an skill that is independent of the particular course material, but requires some kind of content to be implemented.

I began to conceive of two types of standards: those which would apply directly to the new material of the class, and those which would be expected of students in any early-to-intermediate level college math class. My model was inspired by Euclid’s division of axioms into “common notions”—applicable to any mathematical realm—and “postulates”—assumptions tailored to the study of geometry. Or, to use a liturgical metaphor, one might think of the “ordinaries” and “propers” of the mass; they are equally important, but the propers change at each service, while the ordinaries remain the same. The ordinaries provide a constant framework for interpreting and experiencing the propers, which focus on the day or season.

Below is a list of seven common standards I have devised, along with a brief description of what each entails. Next week I will post the content-specific standards for my multivariable calculus class.

• Algebra – accurately simplify, expand, and otherwise manipulate symbolic expressions involving variables and common functions
• Mathematical literacy – correctly use mathematical vocabulary, set notation, equality, and logical implication
• Technological literacy – use computers appropriately for computation, visualization, and research 
• Modeling – translate fluidly between verbal, symbolic, and graphical descriptions of both abstract and “real-world” objects
• Problem solving – determine what question or questions are relevant in a given situation and choose an appropriate strategy for answering the question(s)
• Estimation – anticipate the nature and/or size of solutions and evaluate whether a solution makes sense in the given situation
• Presentation – submit neat, organized, clearly written, and independently produced work, with appropriate context and a clear progression of ideas

These are skills I try to develop in all of my classes, but not always so explicitly. Adding estimation to the list, for instance, was inspired by the books Street-fighting Mathematics and Misteaks … and how to find them before the teacher does…, which emphasize what mathematicians often think of as “common sense” guesses as to what solution a problem will yield, as well as checks that a final answer makes sense. It promotes numeracy. It gives me something I can point to when I wish to explain to a student, “It’s less important that you made a sign error early in your calculation than that you failed to notice at the end that your car is going faster than the speed of light.” Modeling is, for some students, one of the hardest skills to learn. I’m partly using Dan Meyer’s notion of the “ladder of abstraction” to shape my thoughts in this area. Both modeling and estimation could be classified under problem solving, but are sufficiently large subcategories that I chose to separate them out and focus the problem solving standard more narrowly.

To some extent, these are standards for myself. Am I providing guidance on when and how to use computers? Am I demonstrating proper mathematical grammar? Am I distinguishing the (neat, organized) presentation of a solution from the (often messy) process of discovering it? I’m looking forward to this way of assessing and highlighting material.

thinking about thinking about thinking

Last year, I learned the word “metacognition”—roughly, “thinking about thinking”—and its usefulness in pedagogy. Put simply, both students and instructors can better direct the learning process when they are aware not only of what students understand about the course material, but of what the students think they understand. In the fall, I saw my students doing this almost automatically as they wrote brief essays over the course of the semester; many chose to write about their own understanding of mathematics (specifically calculus), what doing math means, and how math is used in the wider world. (More on these essays later.)

This spring, I want to make the metacognitive aspect more explicit in my classes. I will be teaching a multivariable calculus class and an “advanced calculus” class (which will be focused on the geometry, topology, and calculus of manifolds in Euclidean space of arbitrary dimension). Each class will have a distinguishing feature that lends itself to self-reflection. In multivariable calculus, I will be using standards-based grading (SBG) in place of traditional number-and-letter grades, in order to better focus students’ efforts towards improving specific skills, rather than a single, not very helpful grade. In advanced calculus, students will complete biweekly projects that tie together ideas from lectures and prod them to explore new ideas on their own.

How could I use these to encourage “thinking about thinking”? In the first case (multivariable calc), I’ve been considering also having brief, weekly self-reflections in which students will describe what they think the main ideas for the week were, and what they found confusing. (I’ve borrowed this idea from some other source, which I can’t find at the moment.) It seems like it would be easy enough to put the relevant standards on the self-reflection form and have them mark where they think they stand. In the second case (advanced calc), I am trying to follow this principle in creating projects:

It’s OK for new ideas to be abstract, as long as they’re familiar in some way. And it’s OK for them to be unfamiliar, as long as they’re concrete. If I, as a teacher, try to introduce an idea that is both abstract and unfamiliar, then it is likely most students will be lost.

(Paraphrased from a comment made by Steven Strogatz about the expository essays he has written for NYTimes.) I plan on stating this principle outright to my students: when you start a project, you should expect to find ideas that are either familiar or concrete. Look for those, and build your understanding around them. Even better, take that principle and use it to transform your approach to learning. If an idea is unfamiliar, find a way to make it concrete (often by looking at specific examples). If an idea is abstract, connect with something familiar.

I think these two ways of dealing with metacognition are appropriate for the different levels of the classes. Multivariable calculus remains among the set of math classes that are not always taken for the sake of math itself; often students approach it with the paradigm of “mathematics as tool”, and so their reflection on their understanding itself needs a concrete form and guidance. This advanced calculus course, on the other hand, is a transitional course into higher areas of mathematics and is more likely to have students who are interested in math for its own sake; these students should be practicing independent learning and encouraged to make self-reflection a part of that process. These ideas are relatively recently formed in my own head, and I plan to fill them out more over the next two weeks as I prepare for the start of the semester.

One final credit: I first encountered the idea of “metacognition” during a presentation by a representative of McGraw–Hill about one of their online products. While I have not had a chance to explore this product in depth, I am grateful to the presenter for sharing this idea.