a reflection on course structure, and standards for calculus

Here’s what I’ve learned about writing standards: it’s hard to get them balanced properly. This challenge is inherent in developing any grading system. I used to fret about whether quizzes should count for 15% or 20% of the final grade; now I fret about whether the product, quotient, and chain rules should be assessed together or separately. (I’m happier trying to solve the latter.)

Another challenge is in setting up standards so that assessments have some coherence. I’ll explain. My first couple of times creating standards, I sat down and made a list of all the things I wanted my students to be able to do by the end of the semester, grouped into related sets, with an eye towards having each standard be of roughly equal importance (as I mentioned in the previous paragraph). After all, that’s what standards are, right? All the skills we want students to develop? That done, I told myself, “Okay, now every assessment—every homework, quiz, and test—will have to be graded on the basis of items in this list.” In principle, it’s nice to have this platonic vision of what students should do and know, including all the connections between related ideas (parametrization means imposing coordinates on an object; it doesn’t really matter what dimension it has, so parametrizing curves and surfaces should go together as a single standard). However, while this list said a lot about what I thought students should do, it didn’t say much about what I was going to do. It didn’t fit the structure of the course, just of the ideas (oh, wait, we’re parametrizing curves in week 2 and surfaces in week 10—why didn’t I notice that before?). Looking back, I can see that a lack of contiguousness within a standard does reflect a conceptual distinction between the concepts involved (hmmm, maybe the idea of drawing a curve through space is conceptually different from laying out a coordinate system on a curvy surface). I ended up assessing “partial” standards at various points in the semester, which is absurd on the face of it. It’s one thing to assert that a standard may be assessed at different points in the semester, based on how the skills are needed for the task at hand; it’s another to say, “Well, you’re learning part of a skill now, and I’ll test you on that, and you’ll learn the rest of this same skill later.”

I’ve had fewer slip-ups of this sort as time goes on, but I’ve never quite been happy with how the standards match up with the time spent in class. Both of the problems above keep rearing their heads. So for this fall, I decided to look at the schedule of the class and write standards based on what we do in 1–2 days of class. (Reading this blog post by Andy Rundquist earlier in the summer helped push me in this direction.) If it seemed like too little or too much was getting done in a day, well, that’s an indication that the schedule should be modified. In a semester with 38 class meetings, there should be sufficient time allotted for review, flexibility, and a few in-depth investigations, which leads me to having 25–30 content standards for the course. That’s a few more than I’ve had in the past, but not by many.

Here’s the conclusion I’m coming to: standards both shape and are shaped by the structure of the class. Part of what we as instructors bring to a class is a personal view of how the subject is organized and holds together. If you and I are both teaching calculus, there will be a great deal of overlap in what skills we believe should be assessed, but there will be differences, and we’ll find different dependencies. A fringe benefit of writing out standards is that we can see this structure clearly—even better, I believe, than just by looking at the order of topics. They force us to be honest about our expectations, thereby combatting a certain tendency, observed by Steven Krantz in How to Teach Mathematics, to give tests based on “questions that would amuse a mathematician—by which I mean questions about material that is secondary or tertiary. … In the students’ eyes, such a test is not about the main ideas in the course.” You may want students to use calculus mostly in applied settings where exact formulas for the functions involved are not known, whereas I may be primarily concerned with students’ ability to deal formally with closed-form expressions and to deeply understand classical functions. We can both be right. We should both let our students know what we expect of them, rather than making them guess. In short, standards are not completely standardized—they highlight the commonalities and the particularities among courses that treat basically the same material.

With all that said, here I will share my list of standards for Calculus 1 this semester. Because of the length of the list, I’ll just link to a Google document that contains them: Standards for MTH 111, Fall 2014. They are grouped into twenty-six “Content standards” and three “General standards”. Over time, I’ve settled on these last three as skills that I want to assess on every graded assignment: Presentation, Arithmetic and algebra, and Mathematical literacy and numeracy. These are essential skills for doing anything in calculus, and struggles in calculus can often be attributed to weaknesses in these areas. We’ve all had students who are fine at applying the quotient rule to a rational function, but are stymied when it comes to expanding and simplifying the numerator of the result. That can hamper solving certain kinds of problems, and I want to be able to point to “algebra”, not anything calculus-related, as the area that needs attention. The descriptions of the content standards are shaped in part by our textbook, Calculus: Single Variable by Deborah Hughes-Hallett et al. I like to introduce differential equations fairly early in the course—this follows a tradition at my college, too—so some standards related to that are sprinkled throughout. I should also confess an indebtedness to Theron Hitchman for the language of using verb clauses to complete the sentence “Student will be able to …”

In addition to the 29 standards in the document linked above, I have one more for this class: Homework. Oh, homework. The calls to treat homework purely formatively and to stop grading it (link goes to Shawn Cornally’s blog) have not quite reached the halls of post-secondary education. Many college and university instructors believe homework is so important that they make it worth a substantial fraction of the students’ grades. And it is important, but solely as a means for practicing, taking risks, developing understanding, and making mistakes. (See this video by Jo Boaler* on the importance of making mistakes: “Mistakes & Persistence”.) Grading homework almost always means that its usefulness as a place to take risks is undermined. Last semester I didn’t grade homework at all, although I did have a grader, who made comments on the homework that was submitted. On average, about 1/3 of the class turned anything in. At the end of the semester, I got two kinds of feedback on homework. A few students expressed appreciation that the pressure to make sure that everything in the homework was exactly right was relieved. Several, however, said they realized how important doing homework is to their understanding—often because they let it slip at some point—and urged me to again make it “required”. I want to honor both of these sentiments. I want to encourage students to do the homework and to feel like it is the safest of places to practice and make mistakes, and thereby improvements. So I will count both submissions and resubmissions of homework towards this standard. A student who turns in 20 homework assignments or thoughtfully revised assignments will earn a 4 on this standard, 15 will earn a 3, and so on. I hope this will have the desired effect of giving students maximum flexibility and responsibility in their own learning, while also acknowledging the work and practice they do.

All of the rest of the standards, general and content, will also be graded out of 4 points, with the following interpretations: 1 – novice ability, 2 – basic ability, 3 – proficiency, 4 – mastery. (I’ve adapted this language from that used by several other SBG instructors). At the end of the semester, to guarantee an “A” in the class, a student must have reached “mastery” in at least 90% of the standards (that is, have 4s in 27 out of 30 standards), and have no grades below “proficiency”. To guarantee a “B”, she must have reached “proficiency” in at least 90% of the standards, and “basic ability” in the rest. A final grade of at least “C” is guaranteed by reaching “basic ability” in at least 90% of the standards.

Two other blog posts about standards in college-level math classes went up yesterday:

• Bret Benesh wrote about his near-final list of standards for calculus 1, and again explained his idea to have students identify for which standards they have demonstrated aptitude when they complete a test or quiz. I really like this idea, as it essentially builds metacognition into the assessment system. I will have to consider this for future semesters.
• Kate Owens posted her list of standards for calculus 2, which she has organized around a set of “Big Questions” that highlight the main themes of the course. This is particularly important in calculus 2, which can sometimes seem like a collection of disconnected topics. In an ensuing discussion on Twitter, it was pointed out that these kinds of Big Ideas are what can really stick with students, far beyond the details of what was covered.

After reading Kate’s post, I looked at my monolithic list of standards, and attempted to organize them into groups based on three big questions: “What does it mean to study change?” (concepts of calculus), “What are some methods for calculating change?” (computational tools), and “What are some situations in which it’s useful to measure change?” (applications). I was not particularly successful at sorting my standards into these categories, but I like the questions. I may ask the students how they would use the various standards to answer these questions. There are trade-offs in any method of developing a set of standards. I am grateful for these other instructors who are also working on changing how we think about grading and sharing their ideas.

* Jo Boaler’s online courses on “How to Learn Math” are currently open:
For teachers and parents until October 15 ($125)
For students until December 15 (free)

standards for analysis

Writing standards for a proof-based class is a different beast than for introductory calculus, or even probability. In my last post, I described a bit of the structure of the analysis class I’m teaching this fall: inquiry-based, primarily structured around group work, running on a weekly cycle of tackling a problem, agreeing on an approach, and presenting a solution to the class for discussion. My usual way of compiling standards—looking through the course content and breaking it into 20–30 skill sets of roughly equal importance—sort of falls apart here. Do I want students to be able to prove that every Cauchy sequence in the set of real numbers is convergent, and to explain what this implies about the completeness of the reals? Yes, but what I really want is for them to be able to assimilate new concepts and make sense of them by creating examples and fitting the definitions into proofs. Do I want them to be able to compute integrals with respect to both Lebesgue measure and singular Dirac measures? Yes, but what I really want is for them to see how these represent the interplay of mathematics and other sciences—how the exigencies of other fields of science led to the development of both the Lebesgue integral and the Dirac delta—and to feel part of a scientific community, both in and out of the classroom.

While considering these questions, I determined that there are six standards I want students to actively develop during the semester, and on which I want to be giving targeted feedback. These skills will be grounded in the content of the course, but they will also provide the benchmarks of success in mastering the content. Here they are:

1. Correct use of vocabulary and notation: Using mathematical terminology and symbols, especially those particular to analysis, correctly and appropriately.
2. Correct and convincing argumentation: Creating and recognizing complete proofs, with their various pieces presented in a logical order.
3. Clear written exposition: Organizing a paper for the benefit of the reader, making it easy to read and using proper English grammar.
4. Broad vision of the subject: Providing context in papers, including statements of solved problems, a guide to the structure of proofs, and connections with other ideas in the class (previous work or larger themes).
5. Effective verbal presentation: Using good speaking habits (e.g., speaking confidently, talking to the class and not to the board, being sensitive to the audience, handling questions well) to present mathematical content.
6. Collaboration and participation in discussion: Attending class regularly, engaging in discussion through questions and critical feedback, seeking ways to serve the overall community.

(As usual, I’m grateful to Bret Benesh and Theron Hitchman for helping me think through these at an early stage.) As I will acknowledge to my students, some of these standards depend to a certain extent on others. For example, it’s hard to make an effective presentation without mastering the vocabulary of the topic. But I believe these are distinguishable skills, all of which are important for students’ development as mathematicians. And I believe the students should be reflecting on their mastery of these skills as much as their mastery of analysis, and have the chance to show when they’ve improved.

My grading scheme for this class is somewhat of a compromise. I am keeping as many of the features of standards-based grading as I can—including scoring individual assignments by standards and providing opportunities for reassessment—but in order to take into account how well the content has been mastered, at the end of the semester I will weight and total points to determine a final grade. This last step is a kludge made necessary by the continued use of letter grades. If I had my druthers, I would leave the final assessment in terms of the students’ demonstrated mastery of the standards on the individual assignments, so that their focus would always be on improving in those areas rather than reaching a particular grade. I have tried to set this up in a way that, to quote T.J., “if you tried to ‘game the system’ to improve your grade, you would be doing exactly the kinds of things I wanted you to do, and improving your abilities as a mathematician.” (This suggests that we’re having to work against the current grading system to encourage students to grow in the ways we want. I suppose it’s a bit idealistic to believe that we can create a grading and reporting method that will provide both useful feedback to students and a helpful summary to those outside, but I digress.)

Of the standards I’ve listed, 1–4 are basically about writing and 5–6 are basically about active involvement. They will be handled separately in the grading scheme. Each student will write, as part of a group, eleven papers that state and solve a particular problem. These papers will be graded on the basis of standards 1–4, with each standard receiving either a 0 or a 1. After a paper has been graded, the groups will have the benefit of feedback from me and from their classmates, and they will revise, if necessary, until the paper merits at least 3 of the possible 4 points. This final version will be included in a document for the whole class to share. There will be a midterm and a final exam, as required by the college. Both will be take-home, and the individual problems on the exams will be graded according to the same standards as the papers. Following the midterm, students will have the chance to revise their solutions, as they do with the group papers.

Standards 5 and 6 will be graded over the whole semester. Each student will have approximately four chances to present in front of the class; although they will be presenting as part of a group, I will give individual presentation grades, again out of 4 points. The baseline will be 2 points. Grades of 3 or 4 will be achieved based on the quality of the presentation and adherence to the principles stated in the description of the standard. I’ll only consider the highest presentation grade at the end of the semester. For the participation grade, the baseline will again be 2 points, for regular attendance. (This is my first time giving an attendance grade. I generally believe college students should be free to decide for themselves whether coming to class is useful or not. In this case, however, the presence and participation of individual members is essential for the class to work, so I think this grade is justified.) Grades of 3 or 4 will be achieved based on involvement in class discussion, either during meetings or online in the class forum (where each week’s papers will be posted), and in general contributing to a supportive, scientific atmosphere. Since this grade is not given on any particular assignment, I will meet with students individually a couple of times during the semester to gauge their progress and experiences, and to discuss their level of participation.

Now, at the end of the semester, I want students’ work on the group papers and the exams to count about equally towards their final grade, and I want each of those to count about four times as much as their presentation and participation grades. So I will convert everything to a 40-point scale (16 possible points for papers, 16 for exams, 4 for presentation, and 4 for participation Edit: I’ve clarified these numbers in the comments). A letter grade of A will require at least 38 points, with no grades lower than 3 on any assignment (paper or exam problem) or standard (presentation and participation). A B will require at least 28 points, with no grades lower than 3. A C will require at least 18 points. This is as close as I can get to my usual way of assigning final grades: a 4 on 80% of standards (or 90%, depending on the class), with no grade below 3, and so on. It also follows relatively closely the French grading system based on 20 points, with 10 required for passing.

It’s not perfect, but that’s my current grading plan for this inquiry-based Introduction to Analysis course. Thoughts?

probability skills

Tomorrow I start teaching my first probability class. At 8:30 am. So that’s a thing. (I’ve done 8:30 classes before, and they’re fine. In fact, I suspect upper-class students will be more amenable to them than the first-years I’ve taught in the past.)

Probability is a little different for me. It has proofs, sure—it’s a math class, after all—but it feels more like a collection of techniques than the other, more theoretical, upper-level classes I’ve taught. Those techniques are unified by a general philosophy: We can understand the long-term behavior of random processes. So I’ll be incorporating some philosophical texts along with the standard fare.

Because the theory of probability is so steeped in problem-solving, it seems like a good candidate for inquiry-based learning. I chose a classroom with a conference-room design and lots of blackboards (pictures here), so that it feels more like a collaborative environment.

Although it took me a while to realize it, since I’m not as familiar with the material, the nature of the subject also lends itself to standards-based grading. Because of previous confusion I’ve encountered when using the term “standards”, for this class I’m calling them “core skills”. A list is below. (The arrangement of topics has been heavily influenced by our textbook, A First Course in Probability, by Sheldon Ross.)

I feel like this class is a big step forward for me. I’m really going to try to let go of controlling what goes on in class through lecture, and to let (hopefully deep) exploration happen through my choices of topics and questions.

List of core skills in probability

1. Communication – Contribute to class discussion. Present neat, original written work.
2. Combinatorial analysis – Apply counting principles, in particular models involving permutations, combinations, binomial and multinomial coefficients.
3. Axioms of probability – Use definitions of sample spaces, events, and probability, together with Boolean algebra, to prove propositions. Explain meaning of the axioms of probability.
4. Conditional probability – Find conditional probability of one event given another. Explain meaning of conditional probability. Use formulas involving conditional probabilities.
5. Bayes’ formula – Use Bayes’ formula to compute conditional probabilities. Find how odds of an event change with introduction of new data. Justify potentially counterintuitive results.
6. Independence of events – Determine whether two (or more) events are independent. Explain significance of independence.
7. Random variables – Find the probability mass (or density) function and cumulative distribution function of a random variable. Explain what a random variable is and why they are important.
8. Expectation – Find and interpret the expected value of a random variable.
9. Variance – Find and interpret the variance of a random variable.
10. Discrete random variables – Create models with discrete random variables, including:
    • Binomial distribution
    • Geometric distribution
    • Poisson distribution
    • Hypergeometric distribution
11. Continuous random variables – Create models with continuous random variables, including:
    • Uniform distribution
    • Exponential distribution
    • Normal distribution
    • Gamma distribution
12. Joint distribution – Find the joint distribution function or joint probability mass (or density) function of two random variables. Find marginal distributions of the two variables.
13. Independent random variables – Determine whether two variables are independent. Explain meaning of independent random variables. Use independence to compute expectation.
14. Sums of random variables – Justify and apply linearity of expectation. Find the distribution of a sum of independent random variables by using convolution.
15. Conditional distributions – Find the conditional mass (or density) function of one random variable given another. Use conditioning to compute expectations or probabilities.
16. Covariance and correlation – Find the covariance and correlation of a pair of random variables. Explain the meaning of covariance. Compute variance of a sum of random variables.
17. Inequalities and Laws of Large Numbers – Use the Markov and Chebyshev inequalities to approximate distributions. Explain the Weak Law and Strong Law of Large Numbers.
18. Central Limit Theorem – Explain the conditions under which the Central Limit Theorem holds. Apply the Central Limit Theorem to approximate sums of random variables.

SBG: what could have gone better

A couple of weeks ago, I posted about many great things that came out of my experiment with standards-based grading in multivariable calculus. I think the experiment was a success in that there was a lot more good than bad, and the bad things weren’t so bad. Nonetheless, things could have gone better, even in ways I don’t think the students realized, and I would be remiss not to mention them. Whence this post.

Arrangement of standards: There is a tough balance to achieve here, and I didn’t quite make it. The twenty-four standards included

• 7 “common standards” that could relate to any college-level math class
• 3 standards related to vectors and their geometry,
• 3 standards related to graphing and parametrization,
• 5 standards related to differentiation and its applications,
• 4 standards related to integration and its applications,
• 2 standards related to the classical theorems of vector calculus.

I wanted each standard to present a unified concept to be mastered. However, I didn’t want the number of standards to proliferate. I think the total number of standards is about right (anywhere in the range 20–30 would have worked), but the distribution of topics was a little off.

First, seven common standards is too many. Four or five would be more appropriate. Algebra and presentation are essential to treat separately. The others, while distinct in my mind before the class began, became somewhat muddled in their distinction during the semester, and some even overlapped a fair amount with the content-specific standards. The first six or seven standards scored on each homework assignment (often only six, because one was essentially “use of technology”, which rarely figured directly into the homework) became a blurry wash, occasionally used to try to indicate some general, but ill-defined, skill needed attention.

Second, I was too clever in collecting related topics, to the point that certain standards were only partially covered for several weeks at a time. For instance, “line integrals” included both arc length computation—covered in the first chapter—as well as integrals of vector fields—covered in chapter 4. This was perhaps the most egregious example. The topics covered by each standard should have been collected not only by commonality, but also chronologically.

Third, the relative importance of the standards, or at least the relative emphasis that was given to each during class, was not as balanced as I would have like. Setting up and evaluating double and triple integrals—a single standard on the syllabus—takes over a week of class time (although part of that time includes changing coordinates, which was a separate standard). Visualizing vector fields—the only standard that was only tested once, on the final exam—was dealt with sporadically in class. Is it as important to be able to interpret the visual information carried by a graphical vector field as it is to find integrals of several variables? Arguably, hence the separate and equal standards. Was that equality reflected in the amount of attention it was given during the semester? Again, not as much as I would have liked. Not sure this is a challenge of standards per se, but more of course design. Having the standards just highlights the inequity.

Finally, on this topic, even some of the content-specific standards overlapped more than I had intended. I mostly managed to avoid the obvious pitfalls: for instance, computing integrals and finding parametrizations were handled separately, so if a question asked students to find the surface area of a figure, say, and someone set up the wrong integral but computed it correctly from that point, they could get credit for integration but not for parametrization. But what exactly are the skills that go in to setting up an integral? There were standards for describing objects in 2 or 3 dimensions, as well as an “analysis” standard that, in part, required finding the domain of a function. When a double or triple integral is needed, one has to draw on one or more of these skills to find appropriate limits of integration. When it seemed to me like a problem could have been solved by several different approaches, does a failure to find any solution reflect a lack of mastery of all those skills? Hard call. No one’s scores suffered seriously from this ambiguity, but occasionally I found myself judging a surprising number of standards on the basis of one or two exercises.

Grading scale: Having a four-point scale for each standard worked well, on the whole. It was sufficiently refined to target both areas of success and areas needing work. However, the overall quality of work was so good that I had trouble distinguishing among the highest levels of performance. What should I do with a solution that reflects a clear understanding of the skills involved, but has one or two minor errors? Do those reflect some genuine misunderstanding, or simply a slip? How can I judge between a score of 3 (“generally good accuracy”) and 4 (“complete mastery”) in that case? I think I may have to move to a five-point scale, as described here, where 4 and 5 both indicate mastery, but 4 allows for small mistakes.

On the other hand, I think I could have been more exigent in what level of mastery should be reached across the spectrum. On the syllabus, I stated that attaining 4 on 80% of the standards with no scores lower than 3 was sufficient for an A, and I believe I could have raised that percentage to 90% to better reflect complete mastery of the course material. The end result of this is that perhaps a few final grades were more elevated than they might otherwise have been. But seriously—these students worked extremely hard, I am extremely proud of them and have full confidence in their calculus skills, and they deserve some recognition for working with me on this grading experiment and making it a success.

Assessments: I was exceedingly grateful to have a grader with whom I had worked before, and who I trusted to help me implement this SBG system as effectively as possible. I could not have made this first attempt work without her aid. Each week, she would mark the homework, making particular note of places that raised concern or showed exceptional mastery, and then we would meet together to assign scores. I don’t think this method is sustainable across terms. I need to shift to a model that depends less on explaining the grading system to a new assistant each semester, but also that will not vastly increase the amount of time I have to spend grading. (I don’t think having the only graded assignments be two midterm exams and the final is sufficient for me to trust those assessments, nor does it communicate with the students in the way I always hope SBG will.) This is perhaps the area I have to think most about revising as I move forward with SBG in future classes.

Re-assessments: About a fourth of the class took advantage of the opportunity to re-assess any standards. Not such a bad number, especially considering how well they were doing on the whole. But I feel more could have benefitted from this feature of SBG, had the process of reassessment been clearer, and had some of the above obstacles been removed. I do need to find a way to cut down on the time required for reassessment, however: I always tried to claim it would take 10–15 minutes, but often it was much longer than that. No student ever complained about the length of time, which arose both because I gave multiple chances to explain themselves and because of the relative complexity of the material. Nonetheless, I think retesting will have to be made more efficient for it to work in other classes.

Compiling scores: The students received regular updates on their scores in the form of score sheets attached to their homework and exams, but there was no established system by which they could see what their current scores on all the standards was. (Having had some troubles using our LMS in a much simpler grade book setting, I’m averse to the idea of using that or any other online reporting system.) Fortunately, I think this problem is easily solved. Most likely, I’ll handle it in the future by passing out sheets on which students can record their own scores, so that they don’t have to consult with me to find out their current standing. (This is a suggestion I got from Bret Benesh. I suspect some students were already doing this on their own.)

That’s probably not everything that needs improvement, but it’s what came to the forefront of my attention. I have some ideas, some listed above, on how to make my system better next time around. As I’m working on future syllabi, I’ll jot these ideas down and post them here.

SBG: what worked well

I’ve been lax all spring in blogging (other things, too, but that’s beside the point here), and now that the end of the semester has arrived, it’s time I settled down with a cup of coffee to share some of my thoughts on how standards-based grading went. I keep reading blog posts by other teachers and acquiring such cool ideas thereby (this blog brims with excitement about the possibilities for improvement SBG brings to both instruction and assessment; hat tip to Dan Meyer, who recently linked to it), but it is still the case that few teachers at the college/university level are writing about SBG in that context, so hopefully this will be a productive exercise. It’s a good thing qualifications aren’t a prerequisite for blogging, ’cause I ain’t got ’em. Which means this is at least as much about benefitting from the community as trying to contribute to it.

To make this a little more manageable, I’m going to deal with three topics in three (or more) separate posts: what worked well, what worked not so well, and how I plan to move forward.

The Backstory: When last we saw our intrepid blogger, he was heading off into the spring semester to teach multivariable calculus at a small liberal arts college in New England. Having spent several weeks thinking about how he might structure SBG in this class, he had settled on a 4-point system (where 0 represents “complete unfamiliarity” and 4 represents “complete mastery”) with 24 standards: 7 common standards and 17 content-specific standards (an early version of this list was posted here).

What came next: When I met with my class, I explained the system and why I was using it. Assessment should be about giving students the chance to demonstrate what they’ve learned, I said, and providing sufficient opportunity for them to show they’ve mastered the material during the course, even if it doesn’t happen in time for the first test on the material. A point-based system confounds this process. How many students really know what they got each “point” for? (How many of us teachers do?) And a point, once lost, cannot be regained, unless some system of “extra credit” is established, which just creates more work for everyone. I explained that the homework and tests would both create opportunities for them to demonstrate their understanding, and that there was no “weighting” of grades, just regularly-updated scores for the standards. A few expressed surprise, but overall they were accepting that this was how things would work. I explained that the process required honesty from all involved. For my part, I would give scores that I believed accurately reflected each student’s prowess with the various skills they were to learn. For theirs, since I was going to be assessing homework using the same system as the tests, they needed to present their own work each week. (From what I saw, this worked. Students worked together to tackle the problems, but they did not turn in assignments copied from each other. Had I not been at a private liberal arts college with a stringent honor code, I would definitely have had to find another way to handle this. Fortunately, my academic setting allowed me to try SBG this way without worrying about cheating.)

During the semester: We had weekly homework sets and two mid-semester exams. The students have just taken the final exam, and I’ll grade it over the weekend. The homework exercises were primarily taken from the textbook, Michael Corral’s Vector Calculus—available for free download here—and I also wrote some additional exercises to cover other material. (Side note, tangentially related: I chose this textbook because it seemed ridiculous to me to pay $150 for a book that covers material which is available for free almost everywhere. This book basically has the outline I wanted to use, and it has the additional benefit that the exercises are on the whole quite straightforward. I’m realizing that lots of books, and lots of instructors, like “clever” exercises that seem to students only distantly related to the material they’re learning. I’m often tempted that way myself. But if I’m going to assess standards rather than cleverness, a collection of direct applications is invaluable. More on this another time.) The tests were open-book and open-notes. While memorizing definitions, formulas, and theorems is an important step towards forming a coherent picture of the subject, I wanted to emphasize that in the Information Age one can use myriad tools to recall these facts, so that what’s really important is using them intelligently. (Tip: students are afraid of open-book tests, because they assume they’ll be harder. Does “more conceptual” equal “harder”? Possibly in their minds. They did well on the tests, however.)

In addition to the seven “common” standards, each homework covered between three and six other standards, so that many were assessed multiple times. None of the content-specific standards appeared on every assignment; most showed up 2–5 times, although some only once, and some only on the exams. Once a standard had been tested (not just appeared on homework), students could schedule appointments with me to reassess specific standards, up to two per week. To emphasize the importance of mastery, I told the students that I would guarantee an A for anyone who reached (and maintained) 4s in 80% of the standards, with no scores below 3; a B for anyone who reached 3s in 80% of the standards, with no scores below 2, and so on. Scores could be revised up or down, but to alleviate concerns that a fluke of a bad performance at the end of the semester would ruin their scores, I would average their highest and their latest score at the end of the semester.

Student response: When elicited, this was generally positive, which is the most important measure from my perspective. Several students said SBG reduced the stress of test-taking. Others liked how it affirmed their understanding in certain areas while pointing to areas that needed work. A handful took it as a personal challenge to reach all 4s by the end, even though having a couple of 3s wouldn’t change their grade. In the middle of the semester I used an online poll to get anonymous feedback. A couple complained that they didn’t know how their performance was compared with the rest of the class; I view this as part of the purpose of SBG (albeit a minor part)—the striving is against self, not in competition. One said she worked harder to master the material, but appreciated not having to worry about a single bad performance wrecking her grade. The consensus of more than half the students who responded was that SBG reflected their progress and communicated my expectations very well; other responses were at worst neutral. (I wish I had a comparison poll from my non-SBG classes to see if my expectations were being clearly communicated. But if I had done that, I probably would have been using standards anyway.) Even at this level (third-semester calculus), when one might think students’ feelings towards mathematics are firmly set, several students told me that they either had thought they were bad at math or didn’t like it, and now they’ve changed their minds.

My impressions: Mostly I have the sense that standards-based grading was freeing for the students. Far fewer worried about their grades than seems typical (though a few still did), knowing that the way to improve their final grade was the only sensible way: improving their understanding. I was glad to target my feedback, which was the main reason I started considering SBG to begin with. For example, most of the students were adept with algebra, but not all. Some had trouble moving between formulas and visual representations of graphs or objects. Some couldn’t quite grasp how to come up with parametrizations. No student, however, could come out saying “I’m not good at calculus.” They almost always knew which areas they struggled with, and by separating out the different skills, this method of assessment provided confirmation and encouragement at the same time. Each student could look at her scores and say, “Hey, I’m pretty good at a lot of this. I see an area where I’m having trouble, so I guess I’ll work on that.”

In the end, I have tried to be guided by the principle that it is not what I do, but what the students do that contributes the most to their learning. (I picked this up from somewhere, probably several places, and I’ll try at some point to elaborate on how else I applied it.) From that perspective, I would call SBG a success in this class. The participation and performance throughout the class was more uniform across all topics than I have ever seen before. By which I mean, each student knew she was responsible for a certain collection of skills, not just for an accumulation of points or a certain average letter grade, and so they all stepped up to learn all the skills. (Of course, this work ethic is characteristic of students at my school.)

Those are the upsides. In my next post (probably next week, after I’m done grading), I’ll discuss what didn’t go quite so well and why.

assessing standards

As promised, today I want to describe my plan for assessing the standards in my multivariable calculus class. I’ve pretty much settled on the “common standards” that I think would be appropriate for any intermediate college math class, and thanks to some feedback I’ve received since yesterday, I’m refining the list of “content-specific standards” for this class. (For some of the reasons I’m using standards-based grading in this class, see this post, or these slides by T. J. Hitchman from last week’s Joint Math Meetings.) As I see it, there are 4 issues to deal with in scoring standards:

• what scale to use;
• how to assess;
• how to re-assess;
• how to convert to a letter grade at the end of the semester.

I’m almost scared to bring up the last one, because it’s the issue that could unravel the whole process, but I’m certain my (highly driven and motivated students) will panic without it being addressed. If there are suggestions for other issues that should be ranked with these, please let me know. I’ll cover each of these briefly.

What scale I will use

I’ve seen several proposals, including the very simplest, a 2-point system for each standard. (To be fair, I think that works when the list of standards is more refined, so that very specific skills are treated separately and not clustered.) After thinking about what I believe will be the most useful to students, and based on my experience using a 3-point system, I’ve decided to score each standard out of a possible range of 0–4, with 0 indicating “complete unfamiliarity” and 4 indicating “complete mastery”. To aid the students in seeing what I expect at each level, I’ve written sentences they should be able to read and agree with when assessed at the various levels. This is another idea that I’ve borrowed from somewhere, but am having trouble finding at the moment. In my syllabus, I’m describing a standard as a set of closely related skills that represent a piece of knowledge towards mastering the class material, which should explain some of the language below.

1. “I have some idea of what this skill set and its vocabulary mean, but I don't really know how to use it.”
2. “I can complete basic exercises that involve these skills as long as I have some guidance.”
3. “I can use these skills in familiar situations with generally good accuracy.”
4. “I can use this skill set effectively and explain its significance. I can recognize when the skills are useful and apply them to both familiar and new situations.”

(I did not write a sentence for 0-level, as it would be hard for someone completely unfamiliar with a topic to muse on her understanding of it.)

How I will assess

In brief, there will be homework, two midterm exams, and a final exam. All of these will be assessed on the basis of individual standards, and each time a standard appears, its new score replaces the previous score.

I know the debate rages on about whether or not to grade homework, but because the learning time is compressed in a college class, and I do not get to see my students everyday, I think it’s important to have some way to encourage and recognize work done outside of class. That said, the homework grades will not be based on “completion”. Instead, they will provide an opportunity for students to set a “base-level” for their understanding. The report from each homework assignment will list the relevant standards and how the student’s work rates on those standards. This gives them immediate feedback, as well as a chance to see how prepared they are in advance of the exams. I suppose a student could just copy someone else’s work to inflate their scores, but I will explain that in that case their Presentation score (which is part of every assignment) will suffer; their work should be original.

Exams are larger collections of standards, integrated into a broader context. By the time a student gets to a test, she should have a good sense of which areas she will do well in, thanks to homework and earlier self-assessments. Part of the review for each test will include a list of the standards that have been covered to date and may be expected to appear. (This is another good reason for my standards to be a bit coarse, rather than drilling down to specific types of computations—it’s easier to guarantee that a test covers “parametrized curves” than “parametrizing lines”, “parametrizing circles”, “parametrizing spirals”, “checking for smooth points of a curve”, etc.) Again, I suppose a student could not have done any homework before the test and demonstrate total mastery of the material, but that outcome is not, in principle, outside of my goals for SBG.

How I will re-assess

This will be tricky to explain. For many students, tests have always been about how much they contribute to the final grade, rather than how much they say about the current level of understanding. I want to make clear that tests are important and useful only insofar as they create a rich opportunity for learning (through synthesizing the material) and showcasing one’s abilities. Whereas homework assessment is intended to establish a base level of understanding about a student’s ability from week to week, an exam provides a snapshot of her ability, and often a stressful one, at that. After the test, I want to give every student a chance to prove herself in the areas where she may have previously struggled. The experience of other teachers using SBG suggests that this not be done indiscriminately.

Thus, my policy (initially) will be to have students contact me to schedule reassessments for specific standards (during or outside of my usual office hours), at any point in the semester after a standard has been tested. This reassessment could take the form of either an oral examination or an expository presentation by the student. It is unlikely that another written assessment will be given, since I believe the obstacle is often precisely that written tests provoke anxiety. No standard can be reassessed more than once a week, and no more than three two standards can be reassessed in a week. The main point among these practical considerations is that if a student proves she has mastered a course standard, then she receives credit for doing so.

How I will convert to a final grade

This is the least important of the four issues, and yet it is the one that leaves the most lasting record. (In contrast, I hope that what leaves the most lasting overall effect is the knowledge and confidence the students gain.) I don’t want to encourage students to fiddle with a fixed formula, especially since this is my first time using SBG, but I do want to make it clear that mastery of standards is directly correlated with the final letter grade. So here’s what I’m starting with:

• In order to guarantee an A in the class, a student should attain 4s on at least 80% of the course standards and have no scores below 3.
• In order to guarantee a B in the class, a student should attain 3s on at least 80% of the course standards and have no scores below 2.
• In order to guarantee a C in the class, a student should attain 2s on at least 80% of the course standards.

This emphasizes that the goal is mastery. It is also commensurate with what one might expect of the scoring levels in any case: “mostly 4s” should look like an A/A-, “mostly 3s” should look like some form of B, etc.

The score that will be counted for each standard towards the final grade will be the average of the latest score and the highest score. That way earlier gains will not be wiped out by later retreats, but it is still important to keep up each set of skills. Because there are no opportunities for reassessment after the final exam, any prior standards that reappear on the final can only be raised by the scores on that test, not lowered.

And that’s it! That’s my plan for assessing the 20–25 standards that will finally form the basis for grading multivariable calculus this spring. Thoughts and advice are welcome.

standards for multivariable calculus, first pass

OK, it’s time to get real with this. In my last post, I explained some of my reasons for attempting to use SBG this spring and listed seven general standards for college-level mathematics classes. Now I’m listing the standards I have created specifically for multivariable calculus. There’s still time to tweak these, so I would certainly appreciate any feedback over the next few days (or, indeed, at any time!).

I have grouped the standards into five larger categories: “geometry of vectors”, “functions, curves, and surfaces”, “differentiation”, “integration”, and “classical theorems”. In some SBG implementations, the competencies listed below might be more finely sorted out, but I have come to believe that giving college students a somewhat broader classification will encourage them to guide their own education and to think holistically about the material. I have tried to sort these according to three basic principles:

• they cover roughly the same amount of course material;
• they are roughly of the same importance towards mastering the content;
• they can be more-or-less independently measured (although there are indisputably dependences among them).

During the semester, the content on which a standard is based may be introduced gradually over time. For this reason as well as the general expectation that skills should remain honed, many of the standards will be assessed several times. This is also one of my main sources of concern for confusion—what does it mean to have “mastered differentiation operators” at the level of computing partial derivatives and gradients, but not curl and divergence? If I were to distinguish these standards further simply because some parts are separated temporally, however, the number would increase two- or three-fold, making grading an intractable problem for me.

Despite these misgivings about the list itself, I feel it’s important to go ahead and publish it so that I and others can reflect on it. So here goes: (Update 1/18: Following some feedback and discussion, I have revised this list from its original form. Please ignore the parts that have been struck out.)

Geometry of vectors

• Operations – compute and interpret sum, scalar multiples, dot product, determinant, and cross product of vectors in the plane or in space
• Objects – describe points, lines, planes, spheres, and other surfaces using equations, vectors, set notation, or geometric objects of a different kind

Functions, curves, and surfaces

• Visualization – sketch or predict appearance of the graph of a function or curve based on a formula or other description; sketch or describe level sets of a function; use computer software to examine shapes of graphs
• Parametrization (added) – find parametrizations of lines, circles and other curves, as well as planes, spheres, and other surfaces (e.g., tori and graphs of functions)
• Analysis – find domain and range of functions of 1, 2, and 3 variables, and describe these using set notation or geometric terminology; determine continuity

Derivatives

• Operators – apply and interpret partial derivative, Jacobian, and gradient operators to functions; divergence and curl to vector fields
• Operations on functions (added) – compute and interpret partial derivatives and gradient of a function of 2 or 3 variables; apply and explain equality of mixed partial derivatives, including sufficient conditions for such equality to hold
• Operations on vector fields (added) – compute and interpret divergence and curl of a vector field
• Linearization – find tangent vectors, tangent lines, and tangent planes; use these to approximate curves and surfaces near a point
• Higher derivatives – apply and explain equality of mixed partial derivatives; use higher derivatives to collect data about shape of the graph of a function, including classifying critical points
• Optimization (added) – use higher derivatives to collect data about shape of the graph of a function; classify critical points; Lagrange multipliers
• Differential equations – interpret partial differential equations (such as the wave equation, the heat equation, and the Laplace equation) in terms of their solutions; verify that a given function is a solution to a PDE; use a computer to solve or approximately solve PDEs

Integrals

• Multiple integrals – accurately describe regions over which double or triple integrals are computed; perform calculations of double or triple integrals; apply and justify change-of-coordinate formulas; use computers to find integrals
• Line and surface integrals – effectively parametrize curves and surfaces, and use these use parametrizations of curves and surfaces to compute length, area, work, and flux integrals
• Applications – use double and triple integrals to solve problems in geometry and probability; use line and surface integrals to investigate physical phenomena

Classical theorems

• Integrability conditions – check conditions for a vector field to be irrotational or divergence-free (“closed” with respect to curl or divergence operators), and explain the meaning of these conditions; find potential functions
• Generalizations of FTC – explain the meaning and significance of Green’s Theorem, Divergence Theorem, and Stokes’ Theorem
• Applications – use Green’s Theorem, the Divergence Theorem, and Stokes’ Theorem to convert integrals between various forms
• Generalizations of FTC – explain the meaning and significance of Green’s Theorem, Divergence Theorem, and Stokes’ Theorem; use these to convert integrals between various forms

In my next post (probably tomorrow), I’ll explain how I plan to grade these, including what scale I’ll use, how assignments will be broken into their component standards, and how students may improve their score on an individual standard.

some common standards for college-level math

As I mentioned in my last post, this spring I will be using standards-based grading (SBG) in my multivariable calculus class. I won’t belabor what that means, since this is my first time using this method of assessment, and others have written far more expertly on the topic. In short, to me SBG means two things:

• refocusing the nature of grades from periodic goalposts to instructive feedback; and
• honing in on specific expectations so that students know where they are doing well and where they need improvement, from our perspective.

It’s often easy, for example, to become frustrated when a student does all the correct computations, but uses parentheses incorrectly. Or when he or she can differentiate polynomials but not factor them. When grading with points and percentages, we can try to leave helpful notes in the margins or at the end of an assignment, but the presence of THE NUMBER or THE LETTER at the top mostly overcomes our efforts to give informative guidance. Number and letter grades obscure the distinction between “generally understands the course material” and “understands this class well but struggles with prerequisites” and “follows set procedures neatly but has only surface understanding”. A set of standards separates out those cases and handles them differently.

After a stimulating conversation last month with T. J. Hitchman, Dana Ernst, and Jon Hasenbank (with some additional feedback from Bret Benesh), I began to rethink what should (or does, or could—there’s never been just one way to implement SBG) constitute a standard in a college/university math class. Up until that point, I had been planning to split up the course material into every possible type of computation or problem that I would expect a student to be able to complete at the end of the class. It became clear to me that this was probably too fine a gradation; it treats the material reductively rather than holistically, and moreover there’s almost no chance I could equitably test all of the pieces in a 14-week semester, with three class meetings per week.

At the time, I had already planned to have two standards appear on every assignment: algebra and presentation. These constantly trip up students at the introductory calculus level, and without SBG I always struggled to convey to certain students how much their performance would improve just by focusing on one or the other of these. During the aforementioned conversation, it was also proposed that problem solving could be its own standard. In some sense, it is an skill that is independent of the particular course material, but requires some kind of content to be implemented.

I began to conceive of two types of standards: those which would apply directly to the new material of the class, and those which would be expected of students in any early-to-intermediate level college math class. My model was inspired by Euclid’s division of axioms into “common notions”—applicable to any mathematical realm—and “postulates”—assumptions tailored to the study of geometry. Or, to use a liturgical metaphor, one might think of the “ordinaries” and “propers” of the mass; they are equally important, but the propers change at each service, while the ordinaries remain the same. The ordinaries provide a constant framework for interpreting and experiencing the propers, which focus on the day or season.

Below is a list of seven common standards I have devised, along with a brief description of what each entails. Next week I will post the content-specific standards for my multivariable calculus class.

• Algebra – accurately simplify, expand, and otherwise manipulate symbolic expressions involving variables and common functions
• Mathematical literacy – correctly use mathematical vocabulary, set notation, equality, and logical implication
• Technological literacy – use computers appropriately for computation, visualization, and research 
• Modeling – translate fluidly between verbal, symbolic, and graphical descriptions of both abstract and “real-world” objects
• Problem solving – determine what question or questions are relevant in a given situation and choose an appropriate strategy for answering the question(s)
• Estimation – anticipate the nature and/or size of solutions and evaluate whether a solution makes sense in the given situation
• Presentation – submit neat, organized, clearly written, and independently produced work, with appropriate context and a clear progression of ideas

These are skills I try to develop in all of my classes, but not always so explicitly. Adding estimation to the list, for instance, was inspired by the books Street-fighting Mathematics and Misteaks … and how to find them before the teacher does…, which emphasize what mathematicians often think of as “common sense” guesses as to what solution a problem will yield, as well as checks that a final answer makes sense. It promotes numeracy. It gives me something I can point to when I wish to explain to a student, “It’s less important that you made a sign error early in your calculation than that you failed to notice at the end that your car is going faster than the speed of light.” Modeling is, for some students, one of the hardest skills to learn. I’m partly using Dan Meyer’s notion of the “ladder of abstraction” to shape my thoughts in this area. Both modeling and estimation could be classified under problem solving, but are sufficiently large subcategories that I chose to separate them out and focus the problem solving standard more narrowly.

To some extent, these are standards for myself. Am I providing guidance on when and how to use computers? Am I demonstrating proper mathematical grammar? Am I distinguishing the (neat, organized) presentation of a solution from the (often messy) process of discovering it? I’m looking forward to this way of assessing and highlighting material.