Thursday, April 30, 2009

Writing Rubics and Evaluative Criticism

**Warning: I have a migraine, my shower only sometimes has hot water these days, and I'm freaking out about educational assessment...so, if there were ever a day to just keep on scrolling down the old RSS feed, this might be it. I'm just saying.**

The old-school way of grading school papers, tests and presentations was called "norm-based", and unless I wildly misjudge the Rose-coloured reader age demographic, most of us were graded that way. We're basically talking about the bell-curve, which means that the teacher reads all the papers and ranks them. Then a small top tier of papers gets A-plusses, a slightly larger tier gets As, with the graph belling out around the middle-range of C+/B-, and slimming down again for approximately as many Fs as As. Thus, if one class is a somehow sharper than another, or simply has a better teacher, in norm-based assessment, one A paper could be light-years better than another. It depends not on the work the student did, but on the competition.

Another, more current pedagogical theory suggests criterion-based assessment, in which a defined standard is linked to a each tier of grades. And if I understand correctly, there is no expectation of failure; every student is expected to achieve a minimum level of comprehension, and every teacher, to teach towards universal success. There is no "acceptable" number of Fs.

In order to understand what success looks like, teachers create or are provided with rubrics that describe specifically what student achievement will look like at *each level* of comprehension. For those of us who have marked as university teaching assistants, we usually have a basic idea of an A paper based on our own understanding of the material, and simply deduct marks for all the ways a student's paper fails to coincide with that A. Rubrics eliminate a lot of that subjective leeway by providing parameters for As, Bs, Cs, and so on (if you want to get technical about it, criterion-based learning and its accompanying rubrics often don't endorse letter grades, but some do, and sometimes teachers just write'em in anyway, and anyway that assessment structure is a bit beyond me to explain in my current state of migraine-induced malaise).

So, obviously, criterion-based rubrics offer a lot more structure and guidance for a teacher, while still leaving *some* room for subjective interpretation (what does "well-thought out" mean, for example? "some evidence of structure"). There is no substitute for a tough, smart, prepared and aware teacher; but rubrics offer support and keep you from going too far off the rails. Especially for someone like me, who is in the classroom without benefit of formal training, who is without the years of education and experience that would train my subjectivity to coincide with typical student achievement.

Except...I teach creative writing...isn't that *all* subjective? Can anyone ever really say what defines a "good" story? Should one even try, or does that just quash the students' experimental impulses? Shouldn't all creativity be encouraged?

???

I'm treading on sacred ground here, but I don't actually think that. I come from a family of opinionated people who mainly don't bother anyone with our opinions...unless we are fairly confident our insights are well-informed and might help. I don't venture advice very often, because, well, what do I know?

Except sometimes I think I do know about a few things. And sometimes, I think the person who might be on the receiving end of my opinion might actually hear what I'm saying and actually use it for the good. "Veer left or you are going to back into that pole," is a classic example, because sometimes the driver's side blindspot just hides the pole. Other than potential carwrecks, however, mainly I venture opinions on writing. I like to think I read deeply and carefully, and if a writer were interested in hearing what I thought, I might be able to tell things, both good and bad, that would give him or her a little insight into the work.

Mainly I keep quiet about my thoughts on my reading, because nobody asked me, because the writer is faraway and famous, because plenty of reviews have already been written, because I don't think anyone needs or wants to know what I think. When I'm not asked to read critically, I usually don't--I read happily, even if I do twig on things that aren't working from time to time.

However, I think my students are open to feedback--maybe more so than they ever will be again. And they expect my opinion and, mainly, respect it. I have trained them to answer the question, "What's a useless thing to say in workshop?" with,

"That sucks!"

"Or??"

"That's amazing."

"Why?"

"Because they don't help the writer improve the story."

Big opinions don't say anything that matters. A careful, detailed assessment of a work will improve it; a blanket compliment might encourage furture work, but does nothing for what's actually on the page at that moment.

It must be plenty obvious by now that I am knee-deep in marking short stories by grade 10s and 11s, that I have written an assessment rubric to help myself do this, and that I am plagued with doubts. Some good and dedicated students have worked hard on stories that miss the mark in a variety of ways, and I am surprised and sad to be writing that down in red ink (I'm switching to blue) on their pages. I wonder if my assessment of this one particular story will make students hesitant to write the next one, and that thought is terrifying--I want them to improve, but even more, I want them to *write.*

The thought that comforts me is that I am reading polished stories that I have seen in previous drafts and, in several cases, I can see exactly where my feedback and feedback from their peers has spurred the writer to new heights. That's thrilling, to see evidence of learning right there on the page. Some kids ignored me and found their own new heights, and some kids ignored me and stayed at the same level, but I do think most of my students are capable of hearing feedback and using it if it resonates.

And that makes me want to be stringent in my grading, to respect these young writers enough to ask for more than they've done so far and assume they are capable of doing it. I think they are; I hope I am.

There may be 10 or 12 things I could tell you / after that you're on your own
RR

3 comments:

Mark said...

Re the colour in which to mark: I always use green ink when evaluating children - it's the colour that means "move forward".

AMT said...

re: this whole messy mess, which as you know is a big part of my messy life... i will just say two small things. ok, three.

1. i am very impressed with you for caring this hard. no, seriously -- it takes SUCH energy, don't you find?

2. for what it's worth, my attitude wrt grades: if a class's grades are supposed to matter for something like getting into university, then you WANT them to be bell curvey. i mean, if your high school grades are supposed to act as gateways to university, then the question of whether somebody's writing puts them in the top 10% or 20% or 70% is kind of relevant. but having said that they are on a curve doesn't answer the question of where the *mean* is -- if you get a batch of geniuses, you put the mean at A- and then curve away.

if the grades don't serve some purpose like that, but are supposed to be a purely evaluative tool themselves -- then tell your students they are dumb, because they are, and then try to come up with rubrics that give you a lot of leeway, and make the real evaluation come from the comments. i realize this is not practical in high school, for lots of reasons.

3. ... i think an interesting thing to think about is to have two rubrics in assessing written work. one is the criteria on which the work is judged in isolation -- the things you mentioned above -- and then the other rubric is about the work in context. so: did they improve from last time? did they try to do what you suggested, even if they failed at it, or even made things worse? or did this student try something hard, did they take a risk, did they try to be genuinely original? ... you see what i am getting at.

one of the classes i teach sort of annually involves a writing component, and i decided i wanted two essay-like things from them. on the first one, i grade based almost only on the ideas themselves, but i tell them that the second one will also be graded on the writing, and then i write fairly harsh comments on their academic writing, fairly extensively. so then on the second one, if they are still writing insanely but really trying hard to improve i can give them points for that, and hopefully they will have gotten something out of it.

it's hard work, eh? really humbling and infuriating and affirming all at once. man, now i need a nap. too bad i'm at an airport.

Rebecca Rosenblum said...

That is interesting, both about moving the mean and the double-rubric. There are so many things I think I could have done better with this class, and so many things I'm dying to try next time. I really hope the program asks me back so I have a chance to put all these lessons into action!