A few nights ago I was working on an assessment and I spat this in my twitter machine:

I do this often as a way of talking to myself while working on things late in the evening. Often it's just to be funny, sometimes it's a little more serious, but I always I figure it's late and not a lot of people are reading. Not so much this time. It happens. There were a few reactions I want to address though, as my use of "points" there was misinterpreted in a couple ways.

Don't use them!

Ideally, yes. I feel you here. Unfortunately I have a gradebook I have to maintain with a minimum number of assignments per grading period, so I've got to do something.

Let the kids decide!

I tried this once before. In 2015 I implemented A/B/Not Yet grading in Calculus. We'd take an assessment, kids would look at the solution, and then rate themselves. Generally, kids were not adept at rating themselves. I had no good system for dealing with students who rated Not Yet, I was too busy with athletics to have any kind of viable after school system. Collecting the ratings was very time consuming and I was poor at communicating how to determine what should be what. A rubric you say? At that point this work saving system has now become more work than another system would be, so no thanks.

It was interesting experiment, but one I chose not to continue. Your experience may be different.

SBG!

I never never never assume someone is familiar with my teaching journey. These responses were expected (and welcome!) and I chose not to reply to them, because it'd be too easy to come across as that guy who is all "well I wrote the book on SBG blah blah blah..." because that's not a good look. But to those who suggested SBG, yes, I love it as a system and it works super great in a lot of contexts. I have used in Algebra 2, Pre-Cal, and College Algebra with great success. If you are interested in my history with the systems, I believe I have tagged the posts appropriately.

What I do these days...

In general, most classes work great for SBG. I have an SBG system in place with College Algebra and the kids like it. It's extremely similar to the system I came up with a long time ago. However, AP Calc has really never been SBG friendly in my opinion. Implementing a built-in retry system is really the problem. And with the speed you have to move with AP Calc, eventually in class assessments just become a burden. Last spring, AB Calc shifted entirely to free response based assessment because that's what we needed to do. It didn't work, but I still liked it and have some thoughts for this year. In general, with Calculus I will break stuff into a topic, assign some general value to the category, and give a handful of questions about that standard. The points vary, the kinds of questions vary, and there is no built-in retry. It's not really SBG. It's also not a test worth 100 points.

Here's the assessment I was working on when I tweeted:

Screen Shot 2018-09-03 at 11.13.28 PM.png

This particular assignment was for my BC group. The complaint was about how to weight the various sections based on the time it would take to complete them and the complexity. When I grade something like this I take an overall picture of the work. I check for correctness, offer comments, and give students a chance to discuss their work with others. Each one of these sections is an entry in the gradebook. But 1 point ≠ 1 correct problem, I take the whole body of work into account based on any trends in error I may see.

Maybe that clears things up, but maybe it doesn't. Non-traditional graders of the world I'm very with you.

Posted
AuthorJonathan Claydon

Wendy was looking for help on assessment the other day:

My thoughts on assessing in Calculus are ever changing. I attempted to adapt it to two-attempt SBG with a colleague, it didn't work super great. Then we tried an SBG-ish hybrid system. Then I went A/B/Not Yet, and now their assessments are graded on an A/B/Not Yet scale but I don't do a lot of the grading.

Gradebook

For the purposes of reporting, I keep track of grades. The assessments are about once a week, sometimes longer. It takes a while to cover enough unique material in Calculus for it to be worth assessing, part of the reason double SBG clunked. Each one has two or three sections. These assessments are purely for mechanical stuff. I cover all the phrasing and conceptual stuff through AP style benchmarks. These sections are recorded individually and students can earn an A (95), B (85), or Not Yet (0 or 50). All the sections added together are worth 50% of the grade. I shoot for 5 or 6 in a grading period (six weeks).

Practice

Early on I realized that little grades like this are a big pile of whatever. Since there are two and a half topics in Calculus we are constantly addressing the same things over and over, just refining our applications of them. We don't really have the full picture until April. I use these graded assessments as little checks along the way. What are we doing well? What could we do differently? What topics can students comfortably explain to one another?

The explanation part is what I want to get at. I incorporate a lot more discussion into assessment this year. We've done 5 so far and in each cases the students had a period of time where they could talk to each other about the task at hand. Sometimes the entire time, sometimes for only a few minutes. Then we'd discuss. Then I'd force them to go back and look at everything until they had a decent idea. No leaving questions blank or giving up. You aren't allowed to declare intellectual bankruptcy.

Grading is done by the students. I give them access to an answer key and they spent part of their time sifting through it. I ask for honesty in their ratings and I think for the most part I get it. Some students will ask for my opinion of their work before committing to a rating.

This is a time consuming process, a reason I minimize these assessments and stop doing them altogether at the end of January. Planning an assessment that is comprehensive, challenging, and completable in the time allotted is hard enough. Accounting for 10-15 minutes of discussion and 10-15 minutes of grading is equally difficult.

If you have a goal of assessing once a week or moving through a very long list of topics (like the way I do Pre-Cal), you will find this method rough. If you have a class that is pretty focused in scope and you have some flexibility in your time table, give it a shot.

The discussions are fascinating to listen to.

Posted
AuthorJonathan Claydon
2 CommentsPost a comment

Previously I examined results from my first A/B/Not Yet Calculus Assessment.

Now we've had a second one. It covered two new topics and the optional Limits retake I included because the class average was so low. Here are the numbers:

The retake had the intended effect. Students who opted in generally improved their rating. I had a few who performed the same (whether that was a B or NY previously). I learned a couple things from this. First, I will keep including optional retakes from time to time, but I'm not going to make it a guarantee. I worry about student dependence on those. It's been an issue with SBG experiments in AP classes at my school. Thus, the Asymptote category up there I'm going to leave it up to students to improve that on their own time. In the event I do I put an optional, in-class retake on something, I'm going to have students request it in advance. I didn't count, but the number of students who attempted the Limits section wasn't worth making all the copies. Now I think I'll keep them on smaller, separate slips and only make copies for those that request them.

Due to some weirdness in our upcoming schedule, I'm going to address the Asymptote issue through a little daily "test" like I do with parent functions and unit circle values.

I make solutions available, so I am limiting the feedback I give. I need to structure what happens when they get a test back a little better, but I think I'm getting them to buy in to using the solutions as a resource. I'm also getting better at being harsh when making the decision between a B and a NY.

I'm liking the set up so far. The next big step is the transition to a majority of AP material as our assessment base.

Posted
AuthorJonathan Claydon

In Calculus I struck out with an intention of using the bold A/B/Not Yet assessment strategy. Given the broad strokes used to grade AP Exams, it seemed like a good fit. And unlike the SBG system I use in Pre Cal, I don't necessarily have to be weighed down by mandatory second attempts. That can be time consuming. One of the problems I had last year in Calculus was burning class days on assessments for the sake of filling a gradebook when there were probably more important things to do.

I gave the first one a couple days ago. The test had three sections. Class 1 and Class 2 are equal in size (36 vs 35). Here's the test if you're interested. I enter A as 95%, B as 85%, and Not Yet as either 50% or 0% depending on effort.

Chunking content gives me detailed information much like SBG. But I'm not forced to include it next time. Part of A/B/Not Yet is to offer opportunities for retakes. At the beginning of the year I vaguely mentioned that retakes would be "a part of the process" with the intent of figuring that idea out later.

I think I know how it'll go now. If the class as a whole does poorly, that's a sign that I should address some things and offer it a second time. Here, I should address some things about Limits and give everyone another shot on Test 2. The other stuff seemed to take pretty well, so a retake on those will be up to the student to do on their own time.

What else should be done differently? Well, I want to foster more student discussion about what they were asked. I said on the first day that a bunch of C students does me no good. First task upon handing these back will be pulling up the solutions (posted to a class website) and discussing what happened. I think most of the issues that happened in the Limits section will get cleared up just by talking with a peer for 5 minutes.

Like SBG, the act of putting a rating will make the student focus on the rating. It's unavoidable. Though I'm hoping the discussions and the retake triggers will encourage them to sort out weaknesses, i.e. care about learning, so we have a stronger group for the more difficult second semester.

Posted
AuthorJonathan Claydon
2 CommentsPost a comment

Frank had a few thoughts to consider a couple weeks ago.

I got to thinking, do I overtest? Would the tests I give be considered high stakes? Having implemented a standards based system for several years, I have tons of data on the idea. I think the answer to the first question is "maybe" and the second is "no." Though I think my students disagree.

Raw Numbers

Some notes: SBG was introduced in Algebra II as a mid-year experiment; in 2011 and 2012 I gave a one time test of basic skills at the beginning of the year in Pre Cal before the SBG style tests began. In 2012 I gave fewer Pre Cal tests because, ironically, Texas toyed with the notion of increased standardized testing: 15 exams required to graduate high school. We administered benchmarks and real versions of all those tests that school year, in addition to the previous slate of tests still required for the class of 2013. That summer they retracted the plan after public outcry.

I allow about 40 minutes for each test. They happen every 7-10 calendar days. What I think is missing from Frank's numbers is a considering of that old stand by the quiz. Usually at least two quizzes accompany those three term tests, and take about 20 minutes each. Also, it is more common for schools here to have six terms per year. Estimate 15 tests and 12 quizzes per school year and you get about 14 hours, excluding the final.

My Opinion vs Student Opinion

If you ask my students, a lot of them would tell you we test a lot. In the case of testing successive Fridays, that's when many will moan "didn't we just have one?" If you ask the right followup questions, you can get them to see beyond the gut reaction. Do we have quizzes? No. Do you have more grades? Yes. Do you know more about how you're doing? Yes. Do you have homework on top of all of this? No.

I have had them interviewed before, year after year, lots of feel like they have a more specific idea about how they're doing. They can misfire on one part of an assessment but celebrate success on another and come out feeling like they learned. They can share heartbreak over coming oh so close to that elusive 4.

The Stakes

Are these test high stakes? I don't think so. My students get nervous about them, sure. They THINK it's life or death (thanks grade culture). Some of them really push themselves to get double 4s as much as possible. A lot are ready to have a shot at improving something that didn't go well previously. How do I know they aren't high stakes? I have watched the data for years. The standard 15 unit tests allows 15 attempts at 15 test grades. My students have over 80 attempts at 40 test grades. One misunderstood topic in the mix doesn't make a dent.

At the end of some grading terms, topic 7 or 8 might make a 1 point difference. Nobody fails a grading term for an isolated problem, it takes a series of miscues. At I've noticed the problem long before it's ever actually a problem.

I think a better question is: what are you getting from your 14 hours?

Posted
AuthorJonathan Claydon
4 CommentsPost a comment

Soon after I introduced notebooks, my classes started getting bigger. Testing started to become a challenge. Long ago I'd make them physically separate a bit, but now it's impossible. I tried the folder blinder thing. And still, the determined would try to deceive me. Different test versions have had the most success. But I'm trying to remove the motivations as much as possible.

In the spring of 2013 I said forget it, they put so much effort into these notebooks anyway, let's make them available on the test. Day one of school last year I mentioned this policy, with much excitement, naturally.

It felt appropriate in academic level classes. These students needed a little more support. I want to reinforce good habits like keeping a worthwhile notebook. It helped me prove a point when I could draw correlations between the state of someone's notebook and their performance. They usually weren't doing the classwork anyway.

Fast forward to now and I've got PreAP Pre Calculus. A lot of people hold up Pre AP as this sacred thing. We don't mess around in here, mounds of homework, and the highest of high stakes on tests. Again faced with students on top of one another I said you know, if I make enough versions (three each time) I'm ok just making them all open notes.

You might see that as a cop out. But it actually affords a lot of unique opportunities. For one, we don't review for tests. Straight up, no. I announce the date some days in advance and I might mention the list of topics. But there's no scripted review.

The content of my tests has changed as well. A lot of the sections model things we've done in class, but each and every time there's something in there that can't be copied.

And in the end, everyone in there did ridiculously well. Over half got an A for the semester. That might sound wrong and unnatural, but aren't the kids supposed to feel like they learn something rather than satisfy a normal bell-curve of grade distribution?

Posted
AuthorJonathan Claydon
2 CommentsPost a comment

I have never been challenged so much by students. I have never enjoyed a semester so thoroughly start to finish. And, I have never taught at such scale.

This semester was nuts. My students were amazing, and we haven't even gotten to the movies yet. So many times this year I had to step back and come to grips with how much of everything was going on. When we return for the spring, it will be an adjustment remembering what it's like to present to a noisy audience of 36.

I like playing with numbers. The sheer quantity of things got me thinking. Fiddling with Illustrator got me this (full size PDF):

Some personal observations:

  • No one tell first year me about this
  • I interact daily with about 15% of the junior and senior classes
  • I interact daily with about 9% of the school
  • There are more Pre-Cal students in 3 sections than I had in 4 last year
  • Some months ago I estimated I've taught 800 students, it's 759
  • 90 of them had me more than once
  • That number excludes about 200 who know me via athletics or otherwise
  • My Pre-Cal students are amazing
  • Calc leaves me cautiously optimistic

Exciting. Excited.

Posted
AuthorJonathan Claydon

Standards based grading has been a huge success for me. It's helping my students have conversations about success and becoming determined about showing improvement. Most of them have stopped talking about That Number. The number that I dislike. The one all the top kids in your class obsess over. 100. Ideally, it should all be about growth and feedback, but tradition dictates we give kids a number. They live and die by that number. See also: college, where sub-par teaching methods and the pursuit of 4.0 poisons just about everything.

AP culture feeds the beast. That GPA bonus is enticing. Surely extra credit must be offered. I NEED to have an A. If you can just give me two points mister, it's just two points!

I dislike it intensely. It takes a fantastic opportunity, exposure to high level curriculum, and turns it into a war for 100. How do you teach a group of Calculus kids, kids who have fought for 100 forever at this point, to forget about the stupid number and focus on learning for the sake of learning?

A few years ago I helped adapt my typical SBG approach to Calculus. Subdivide the class into topics, test each topic twice, honor the best score, and offer full replacement after school. Throw a six weeks exam in there to keep a little accountability. It didn't work. Sure, kids demonstrated improvement. But many blew off the initial attempt. TONS of them showed up after school to try and replace everything. All in pursuit of monkeys, er, 100.

This year? Reset button.

The Strategy

The first failure is trying to take Calculus and force it into nice little topics. After a two year experiment, it was pretty clear Calculus doesn't like being treated that way. It's a Big Ideas kind of class. It's primarily conceptual. Everything matters. You can't forget about September.

The second failure is removing too much accountability from the assessments. That after school thing was a crutch. Special rules had to be written.

The third failure was a disconnect between the assessment material and the design of the AP Exam. AP scores were terrible. Blowing off assessments probably didn't help. Seeing the format of the exam too late probably didn't help either.

What do we do?

Giant tests out of 100 are out of the question. Assessment needs to remain short and frequent. Reassessment should be available, but limited. Concepts matter. More accountability needs to exist. I should have the freedom to put anything and everything on them.

The Method

After a lot of discussion with a fellow Calculus teacher who wanted to do the same thing, we arrive at our current method. These are given once a week-ish.

It's a hybrid sort of thing. Each section is evaluated in a big picture sort of way. No nitpicking -1 or -2 junk, just feedback and an overall evaluation. The front page is skills. The top is for the deep catalog stuff. The bottom is fresh. The back is all conceptual. Written answers, thought questions, justifying answers and all that. Each is a separate entry in the gradebook. Kids still go right to the number, but it's not That Number. Only skills are eligible for a retry. Concepts are important. Six weeks exams are now mini-AP tests.

The Hook

Students should keep track. Students should get a pat on the back for success. There should be a goal unrelated to 100.

Since each test is a unique set of semi-permanent scores, the tracking is a little different. Each test is recorded on a line with the results from the three categories. If the total is 16+, you get a shiny, Calculus-only silver star.

It's the dumbest thing in the world. But an 18 year old will commit crimes for a sticker. And now that's how they talk. Did I get a star? I starred this one! I'm all ABOUT those stars! NOOOOOO 15, I HAD IT!!!!!!

Take a step back for a second. There are 20 points total. A 16 represents 80% (a comfortable 5 on the AP test, the reason it was chosen). A 15 is 75%. Think about that kid with the 15. What happens if they got a paper back with a 75? I probably can't do this. This is as hard as everyone said. Now I'm behind in math too.

When would a student with a 75 be ENCOURAGED by that result? Be willing to stop by after school to make sure that darn it they get a sticker next time? In the fight for 100, 75 is pretty unacceptable. And yet, my students have no problem with it, because the dreaded 100 has been abstracted away.

That's all I want. If you are motivated by getting As, ok. But I want you in tutorials because you're eager to learn, not because you're going to beg for some extra credit. And in class where the real evaluation isn't until May? Your number out of 100 is the least important number there is.

Posted
AuthorJonathan Claydon
2 CommentsPost a comment

In January 2011, I tried something crazy. It's now a local phenomenon.

I had a mind blowing meeting with a group of middle school teachers a couple weeks ago. A group ready to change just about everything they do regarding student work management and assessment. Not just ready, but determined.

I began to seriously look at my assessment practices almost four years ago. Really, it was a time concern. I spent my first year giving big unit tests and grading homework and all that stuff they say you're supposed to do and found it incompatible with the demands on my time from coaching. By accident, I wandered into SBG, notebooks and all that. Three years ago, I started sharing my successes with my math department. Two years ago, many of them were ready to try it.

And now, years after a secret experiment, SBG and notebooks is the majority practice at all levels of math in my school, Algebra 1 to Calculus. We've all implemented it differently, PreAP is excluded for the most part (though not mine), our notebook management systems are varied, and I will say that not everyone may experience the same altering side effects I did, but we made a change.

Even crazier, in just a handful of meetings, an entire middle school hopped on board because many of my colleagues had experienced such positive results and testified as much to them. At some distant point in the future, there's a kid who will graduate having spent their 6th - 12th math career judged by how they show growth.

Challenges remain:

  • SBG is no bandaid for bad lessons, learning to be interesting is a whole other hurdle
  • this middle school is in uncharted territory, with no one specifically on campus who has implemented this stuff before
  • our Algebra 1 team is trying SBG, notebooks, AND a tweaked curriculum, there will be kinks, but my role this year is their direct line of support
  • new teachers are always a tough sell, never getting the full philosophy behind the move or experiencing the classroom challenges that lead to this stuff in the first place
  • SBG and notebooks can't be forced as a matter of policy, it will guarantee half-hearted implementation, each teacher needs the eureka moment, and that may never come

We aren't perfect. It is not math department heaven on earth. Simple requiring a student to have a notebook does not mean you will have 100% engagement. Implementing SBG will not guarantee you have a 0% failure rate or that kids will achieve that understanding they've seemed to be missing.

The point of this is to say that change is possible. Change at scale is possible. And it happened organically. Our district did not mandate any of these policies. There is a ton of work to do. But the fact that we even have these discussions locally without anyone thinking I'm a crazy person is important.

If you have great practices that are working for you, you don't have to be the lone voice in the wilderness. Student results speak for themselves. Chances are there's a teacher in your district who wish things could be better but doesn't know where to start. Show them what to do. Share everything. And slowly, you can convince a lot of people to step back at their current methods and ask "why?"

Posted
AuthorJonathan Claydon

I'm a couple years into the Standards Based Grading switch and after a lot of observing, I'm adding some tiny touches to our procedures.

First, I added a new grade. I mused about it a year ago that in Pre Cal particularly there was a lot of nuance in the questions. In rare cases, rating work at 3 (60%) felt like a disservice to the effort displayed. I've introduced the 3+ this year, which is equivalent to 70%. Historically my system works with 3 + 3 = 3.5, and 4 + 4 = 5. Now I slid the 3+ to work as 3+ + 3+ = 4. I predict it will be pretty rare for a student to earn a pair of 3+s though. It's intended to be a rare situation. I rolled it into Algebra II as well just to keep my head straight while grading.

Second, an extra level of organization for the notebooks. Two of the most frequently asked questions are "do we have to keep the tests?" and "what do these numbers mean again?" I've always left keeping the tests up to the student, but stuffing them in a composition notebook often results in them disappearing over time. Both questions can be answered with a simple strip of sacrificed manilla folder. 

Enter test pockets: 

day10.jpg

Returned tests go in the pocket. I clean out my stash of manilla folders (former desk blinders to discourage cheating which didn't really work) and hopefully there's more clarity. 

Reactions from students I had last year ranged from "ooh, clever" to "ugh, NOW you think of this!" 

Posted
AuthorJonathan Claydon