formative assessment isn’t scary

I get a little jumpy around nomenclature. This probably comes from being a mathematician; we spend a lot of time coming up with names for complex ideas so that they’re easier to talk about. Naming a thing gives you power over it and all that. So when we come across a new name, it could take anywhere between a few minutes and a few months to unpack it. An abelian group, for instance, can be completely and formally defined very quickly, whereas a rigorous definition of Teichmüller space often takes several weeks in a course to reach. Some things are in between, easy to define but not-so-easy to figure out why the object has a special name (see dessin d’enfant). Very often a major step along the way to understanding something is grasping the simplicity—the inevitability, even—of its definition.

So it is with formative assessment. When I first learned about the formative/summative assessment distinction, I got nervous. I thought, “So, besides giving tests and quizzes, I need to be doing a whole bunch of other things in class to find out what students are thinking? How much more class time will this take? How much more preparation will it take? How will I ever incorporate this new feature into my class, and how bad will it be if I don’t manage to?” I think I got caught up in the impressiveness of the term assessment; that seemed like a big “thing”, and doing any kind of assessment must require a carefully crafted and substantial process.

So let’s back up a bit. In teaching, assessment means anything that provides an idea of students’ level of understanding. If it’s not graded, it’s formative.

That’s it.

As a teacher, unless you have literally never asked “Are there any questions?”, you have done formative assessment. Asking “Are there any questions?” is a crude and often ineffective means of formative assessment, but it is assessment nonetheless. You and I are already doing formative assessment, which means that we don’t have to start doing it; we can instead turn to ways of doing it better. Somehow I find that easier.

“Formative assessment” is more like “abelian group” than “Teichmüller space”. If you have ever added integers, you have worked with an abelian group. But having an easily-grasped definition doesn’t have to mean than a concept is limited. In fact, simple definitions can often encompass a broad range of ideas, which happen to share a few common features. There are entire theorems and theories built on abelian groups. Naming a thing gives you power over it. Now that we’ve named formative assessment, let’s see how we can build on it.

David Wees has a collection of 56 different examples of formative assessment, which range from the “Quick nod” (“You ask students if they understand, and they nod yes or no”—possibly virtually, which enables anonymity) to “Clickers” to “Extension projects” (“Such as: diorama, poster, fancy file folder, collage, abc books. Any creative ideas students can come up with to demonstrate additional understanding of a topic.”) John Scammell has a similar collection of Practical Formative Assessment Strategies (some overlap with Wees’s list), grouped into sections like “Whole Class Strategies”, “Individual Student Strategies”, “Peer Feedback Strategies”, “Engineering Classroom Discussion Strategies”, and so on.

Formative assessment doesn’t have to take much time or preparation. You’re probably already doing it without realizing it. Adding some variety to the methods of assessment, however, can provide a more complete picture of students’ understanding, to their benefit. Feel free to add more resources in the comments.

a reflection on course structure, and standards for calculus

Here’s what I’ve learned about writing standards: it’s hard to get them balanced properly. This challenge is inherent in developing any grading system. I used to fret about whether quizzes should count for 15% or 20% of the final grade; now I fret about whether the product, quotient, and chain rules should be assessed together or separately. (I’m happier trying to solve the latter.)

Another challenge is in setting up standards so that assessments have some coherence. I’ll explain. My first couple of times creating standards, I sat down and made a list of all the things I wanted my students to be able to do by the end of the semester, grouped into related sets, with an eye towards having each standard be of roughly equal importance (as I mentioned in the previous paragraph). After all, that’s what standards are, right? All the skills we want students to develop? That done, I told myself, “Okay, now every assessment—every homework, quiz, and test—will have to be graded on the basis of items in this list.” In principle, it’s nice to have this platonic vision of what students should do and know, including all the connections between related ideas (parametrization means imposing coordinates on an object; it doesn’t really matter what dimension it has, so parametrizing curves and surfaces should go together as a single standard). However, while this list said a lot about what I thought students should do, it didn’t say much about what I was going to do. It didn’t fit the structure of the course, just of the ideas (oh, wait, we’re parametrizing curves in week 2 and surfaces in week 10—why didn’t I notice that before?). Looking back, I can see that a lack of contiguousness within a standard does reflect a conceptual distinction between the concepts involved (hmmm, maybe the idea of drawing a curve through space is conceptually different from laying out a coordinate system on a curvy surface). I ended up assessing “partial” standards at various points in the semester, which is absurd on the face of it. It’s one thing to assert that a standard may be assessed at different points in the semester, based on how the skills are needed for the task at hand; it’s another to say, “Well, you’re learning part of a skill now, and I’ll test you on that, and you’ll learn the rest of this same skill later.”

I’ve had fewer slip-ups of this sort as time goes on, but I’ve never quite been happy with how the standards match up with the time spent in class. Both of the problems above keep rearing their heads. So for this fall, I decided to look at the schedule of the class and write standards based on what we do in 1–2 days of class. (Reading this blog post by Andy Rundquist earlier in the summer helped push me in this direction.) If it seemed like too little or too much was getting done in a day, well, that’s an indication that the schedule should be modified. In a semester with 38 class meetings, there should be sufficient time allotted for review, flexibility, and a few in-depth investigations, which leads me to having 25–30 content standards for the course. That’s a few more than I’ve had in the past, but not by many.

Here’s the conclusion I’m coming to: standards both shape and are shaped by the structure of the class. Part of what we as instructors bring to a class is a personal view of how the subject is organized and holds together. If you and I are both teaching calculus, there will be a great deal of overlap in what skills we believe should be assessed, but there will be differences, and we’ll find different dependencies. A fringe benefit of writing out standards is that we can see this structure clearly—even better, I believe, than just by looking at the order of topics. They force us to be honest about our expectations, thereby combatting a certain tendency, observed by Steven Krantz in How to Teach Mathematics, to give tests based on “questions that would amuse a mathematician—by which I mean questions about material that is secondary or tertiary. … In the students’ eyes, such a test is not about the main ideas in the course.” You may want students to use calculus mostly in applied settings where exact formulas for the functions involved are not known, whereas I may be primarily concerned with students’ ability to deal formally with closed-form expressions and to deeply understand classical functions. We can both be right. We should both let our students know what we expect of them, rather than making them guess. In short, standards are not completely standardized—they highlight the commonalities and the particularities among courses that treat basically the same material.

With all that said, here I will share my list of standards for Calculus 1 this semester. Because of the length of the list, I’ll just link to a Google document that contains them: Standards for MTH 111, Fall 2014. They are grouped into twenty-six “Content standards” and three “General standards”. Over time, I’ve settled on these last three as skills that I want to assess on every graded assignment: Presentation, Arithmetic and algebra, and Mathematical literacy and numeracy. These are essential skills for doing anything in calculus, and struggles in calculus can often be attributed to weaknesses in these areas. We’ve all had students who are fine at applying the quotient rule to a rational function, but are stymied when it comes to expanding and simplifying the numerator of the result. That can hamper solving certain kinds of problems, and I want to be able to point to “algebra”, not anything calculus-related, as the area that needs attention. The descriptions of the content standards are shaped in part by our textbook, Calculus: Single Variable by Deborah Hughes-Hallett et al. I like to introduce differential equations fairly early in the course—this follows a tradition at my college, too—so some standards related to that are sprinkled throughout. I should also confess an indebtedness to Theron Hitchman for the language of using verb clauses to complete the sentence “Student will be able to …”

In addition to the 29 standards in the document linked above, I have one more for this class: Homework. Oh, homework. The calls to treat homework purely formatively and to stop grading it (link goes to Shawn Cornally’s blog) have not quite reached the halls of post-secondary education. Many college and university instructors believe homework is so important that they make it worth a substantial fraction of the students’ grades. And it is important, but solely as a means for practicing, taking risks, developing understanding, and making mistakes. (See this video by Jo Boaler* on the importance of making mistakes: “Mistakes & Persistence”.) Grading homework almost always means that its usefulness as a place to take risks is undermined. Last semester I didn’t grade homework at all, although I did have a grader, who made comments on the homework that was submitted. On average, about 1/3 of the class turned anything in. At the end of the semester, I got two kinds of feedback on homework. A few students expressed appreciation that the pressure to make sure that everything in the homework was exactly right was relieved. Several, however, said they realized how important doing homework is to their understanding—often because they let it slip at some point—and urged me to again make it “required”. I want to honor both of these sentiments. I want to encourage students to do the homework and to feel like it is the safest of places to practice and make mistakes, and thereby improvements. So I will count both submissions and resubmissions of homework towards this standard. A student who turns in 20 homework assignments or thoughtfully revised assignments will earn a 4 on this standard, 15 will earn a 3, and so on. I hope this will have the desired effect of giving students maximum flexibility and responsibility in their own learning, while also acknowledging the work and practice they do.

All of the rest of the standards, general and content, will also be graded out of 4 points, with the following interpretations: 1 – novice ability, 2 – basic ability, 3 – proficiency, 4 – mastery. (I’ve adapted this language from that used by several other SBG instructors). At the end of the semester, to guarantee an “A” in the class, a student must have reached “mastery” in at least 90% of the standards (that is, have 4s in 27 out of 30 standards), and have no grades below “proficiency”. To guarantee a “B”, she must have reached “proficiency” in at least 90% of the standards, and “basic ability” in the rest. A final grade of at least “C” is guaranteed by reaching “basic ability” in at least 90% of the standards.

Two other blog posts about standards in college-level math classes went up yesterday:

• Bret Benesh wrote about his near-final list of standards for calculus 1, and again explained his idea to have students identify for which standards they have demonstrated aptitude when they complete a test or quiz. I really like this idea, as it essentially builds metacognition into the assessment system. I will have to consider this for future semesters.
• Kate Owens posted her list of standards for calculus 2, which she has organized around a set of “Big Questions” that highlight the main themes of the course. This is particularly important in calculus 2, which can sometimes seem like a collection of disconnected topics. In an ensuing discussion on Twitter, it was pointed out that these kinds of Big Ideas are what can really stick with students, far beyond the details of what was covered.

After reading Kate’s post, I looked at my monolithic list of standards, and attempted to organize them into groups based on three big questions: “What does it mean to study change?” (concepts of calculus), “What are some methods for calculating change?” (computational tools), and “What are some situations in which it’s useful to measure change?” (applications). I was not particularly successful at sorting my standards into these categories, but I like the questions. I may ask the students how they would use the various standards to answer these questions. There are trade-offs in any method of developing a set of standards. I am grateful for these other instructors who are also working on changing how we think about grading and sharing their ideas.

* Jo Boaler’s online courses on “How to Learn Math” are currently open:
For teachers and parents until October 15 ($125)
For students until December 15 (free)

standards for analysis

Writing standards for a proof-based class is a different beast than for introductory calculus, or even probability. In my last post, I described a bit of the structure of the analysis class I’m teaching this fall: inquiry-based, primarily structured around group work, running on a weekly cycle of tackling a problem, agreeing on an approach, and presenting a solution to the class for discussion. My usual way of compiling standards—looking through the course content and breaking it into 20–30 skill sets of roughly equal importance—sort of falls apart here. Do I want students to be able to prove that every Cauchy sequence in the set of real numbers is convergent, and to explain what this implies about the completeness of the reals? Yes, but what I really want is for them to be able to assimilate new concepts and make sense of them by creating examples and fitting the definitions into proofs. Do I want them to be able to compute integrals with respect to both Lebesgue measure and singular Dirac measures? Yes, but what I really want is for them to see how these represent the interplay of mathematics and other sciences—how the exigencies of other fields of science led to the development of both the Lebesgue integral and the Dirac delta—and to feel part of a scientific community, both in and out of the classroom.

While considering these questions, I determined that there are six standards I want students to actively develop during the semester, and on which I want to be giving targeted feedback. These skills will be grounded in the content of the course, but they will also provide the benchmarks of success in mastering the content. Here they are:

1. Correct use of vocabulary and notation: Using mathematical terminology and symbols, especially those particular to analysis, correctly and appropriately.
2. Correct and convincing argumentation: Creating and recognizing complete proofs, with their various pieces presented in a logical order.
3. Clear written exposition: Organizing a paper for the benefit of the reader, making it easy to read and using proper English grammar.
4. Broad vision of the subject: Providing context in papers, including statements of solved problems, a guide to the structure of proofs, and connections with other ideas in the class (previous work or larger themes).
5. Effective verbal presentation: Using good speaking habits (e.g., speaking confidently, talking to the class and not to the board, being sensitive to the audience, handling questions well) to present mathematical content.
6. Collaboration and participation in discussion: Attending class regularly, engaging in discussion through questions and critical feedback, seeking ways to serve the overall community.

(As usual, I’m grateful to Bret Benesh and Theron Hitchman for helping me think through these at an early stage.) As I will acknowledge to my students, some of these standards depend to a certain extent on others. For example, it’s hard to make an effective presentation without mastering the vocabulary of the topic. But I believe these are distinguishable skills, all of which are important for students’ development as mathematicians. And I believe the students should be reflecting on their mastery of these skills as much as their mastery of analysis, and have the chance to show when they’ve improved.

My grading scheme for this class is somewhat of a compromise. I am keeping as many of the features of standards-based grading as I can—including scoring individual assignments by standards and providing opportunities for reassessment—but in order to take into account how well the content has been mastered, at the end of the semester I will weight and total points to determine a final grade. This last step is a kludge made necessary by the continued use of letter grades. If I had my druthers, I would leave the final assessment in terms of the students’ demonstrated mastery of the standards on the individual assignments, so that their focus would always be on improving in those areas rather than reaching a particular grade. I have tried to set this up in a way that, to quote T.J., “if you tried to ‘game the system’ to improve your grade, you would be doing exactly the kinds of things I wanted you to do, and improving your abilities as a mathematician.” (This suggests that we’re having to work against the current grading system to encourage students to grow in the ways we want. I suppose it’s a bit idealistic to believe that we can create a grading and reporting method that will provide both useful feedback to students and a helpful summary to those outside, but I digress.)

Of the standards I’ve listed, 1–4 are basically about writing and 5–6 are basically about active involvement. They will be handled separately in the grading scheme. Each student will write, as part of a group, eleven papers that state and solve a particular problem. These papers will be graded on the basis of standards 1–4, with each standard receiving either a 0 or a 1. After a paper has been graded, the groups will have the benefit of feedback from me and from their classmates, and they will revise, if necessary, until the paper merits at least 3 of the possible 4 points. This final version will be included in a document for the whole class to share. There will be a midterm and a final exam, as required by the college. Both will be take-home, and the individual problems on the exams will be graded according to the same standards as the papers. Following the midterm, students will have the chance to revise their solutions, as they do with the group papers.

Standards 5 and 6 will be graded over the whole semester. Each student will have approximately four chances to present in front of the class; although they will be presenting as part of a group, I will give individual presentation grades, again out of 4 points. The baseline will be 2 points. Grades of 3 or 4 will be achieved based on the quality of the presentation and adherence to the principles stated in the description of the standard. I’ll only consider the highest presentation grade at the end of the semester. For the participation grade, the baseline will again be 2 points, for regular attendance. (This is my first time giving an attendance grade. I generally believe college students should be free to decide for themselves whether coming to class is useful or not. In this case, however, the presence and participation of individual members is essential for the class to work, so I think this grade is justified.) Grades of 3 or 4 will be achieved based on involvement in class discussion, either during meetings or online in the class forum (where each week’s papers will be posted), and in general contributing to a supportive, scientific atmosphere. Since this grade is not given on any particular assignment, I will meet with students individually a couple of times during the semester to gauge their progress and experiences, and to discuss their level of participation.

Now, at the end of the semester, I want students’ work on the group papers and the exams to count about equally towards their final grade, and I want each of those to count about four times as much as their presentation and participation grades. So I will convert everything to a 40-point scale (16 possible points for papers, 16 for exams, 4 for presentation, and 4 for participation Edit: I’ve clarified these numbers in the comments). A letter grade of A will require at least 38 points, with no grades lower than 3 on any assignment (paper or exam problem) or standard (presentation and participation). A B will require at least 28 points, with no grades lower than 3. A C will require at least 18 points. This is as close as I can get to my usual way of assigning final grades: a 4 on 80% of standards (or 90%, depending on the class), with no grade below 3, and so on. It also follows relatively closely the French grading system based on 20 points, with 10 required for passing.

It’s not perfect, but that’s my current grading plan for this inquiry-based Introduction to Analysis course. Thoughts?

low-threshold exercises for analysis

This fall, one of my courses will be Introduction to Analysis. At my school, this has been taught using a modified-Moore method for the last few years, and I will be largely adopting the structure and content of these previous years. In this IBL implementation, students work in groups on one problem per week. Each week has three assigned problems (so generally multiple groups are working on the same problem) that are loosely related. At the end of the week one class period is devoted to presentations: for each problem, one group is selected to present their solution in about 20 minutes, and the rest of the class is expected to be engaged in discussion with the presenters. Many of the problems were developed by David Cohen (now professor emeritus), who described the method in an article for the American Mathematical Monthly. Further developments were made by Christophe Golé, with whom I co-taught the course two years ago. From my first exposure to the materials for this class, I have been impressed by the clever way students are led through standard material by a non-standard path.

As with many introductory analysis courses, one goal of this class is to help students transition to more formal mathematics, giving them experience with absorbing definitions and writing proofs. The problems themselves guide students through much of this process. I felt, however, that at times students could benefit from having exercises that allow them to interact more rapidly and immediately with new definitions. So one aspect I’m adding this year is a collection of “Warm-up exercises”, one per week. These are intended to be “low-threshold” activities, in the sense that a student should be able to work on them and produce results even with just a superficial understanding of the definitions involved. My hope is that by interacting with the definitions in a meaningful and productive way, they will feel more prepared to grapple with the assigned problems.

Here is a list of the exercises I’ve written, together with a rough description of the corresponding week’s topic. In addition to being “low-threshold”, several of these are also “high-ceiling”, meaning that immediate extensions and generalizations are evident. (For most of the course, however, the “high ceiling” is provided by the main set of problems.)

• (Counting and cardinality) Prove that the sets {1,2,3} and {4,5,6} have the same cardinality. Prove that {1,2} and {1,2,3} do not.
• (Balls in metric spaces) Recall |x|=x if x≥0 and |x|=−x if x < 0. Prove |x+y|≤|x|+|y| for any real numbers x,y.
• (Topology of real numbers) Prove that if x is isolated from a set T ⊂ R, then x cannot be an accumulation point of T.
• (Topological properties) In R, is a set that contains just one point compact? (A bit of clarification here: in this course, the definition given for “compact” is a variant of sequential compactness, namely, that every infinite subset has an accumulation point.)
• (Continuity) Prove that x^n is continuous at zero for any n∈N.
• (Properties of functions) Prove that x^n is differentiable at zero for any n∈N.
• (Sequences of functions) Use the algebraic identity (1–r)(1+r+r^2+…+r^n) = 1–r^(n+1) to prove that the series 1+r+r^2+r^3+… converges to 1/(1–r) if |r| < 1. (I keep finding that students have forgotten the sum of a geometric series in classes after calculus, so I figured it made sense to remind them of this fact while also suggesting they prove it.)
• (Uniform convergence and degrees of differentiability) For any k∈N, give an example of a function that is C^k but not C^(k+1).
• (Borel sets) Suppose X is any set and A is the power set of X, i.e., the collection of all subsets of X (including ∅ and X itself). Show that A is a countably complete Boolean algebra.
• (Lebesgue integration) Show that a sum of simple functions is a simple function.

There’s one more week’s worth of problems—all focused on properties of the Cantor set—which don’t require any new definitions.

I’m not quite sure what role to give these in the course. I don’t want them to be required, and I definitely don’t want to make them “extra credit”. I do want them to provide a useful entry into playing around with definitions and not seem like extra work. Thoughts?