On the matter of numbers

Mindful of other deadlines, I finally applied pressure to my felt-tipped pen while in transit, on a quiet Sunday morning subway-car. I felt unsuited, ill-prepared, to start writing. No notes to work from. Just a folder full of documents unrelated to this blog post. I did have, for better or for worse, an inky pen with a soft point (ballpoints are better for business) and the blank surface of a manila folder. I began drafting this contextual blog post for the “Writing About Numbers” faculty roundtable that Bill Ferns and I will co-run next week. I ended up with this: drawing out, crossing out, sketching again, a recurring discomfort I’ve had since grade school. The story of this recurring feeling is not particularly remarkable, one that is not so dissimilar from my impulse to avoid the freshly opened new word-processor document on my laptop screen (blankness). The story:  I am immediately stunned by numbers and, in defense, my mind triggers a blank.

scribble

This anecdote is a roundabout way of saying that the initial discomfort I sense when writing in a familiar language is, in some ways, akin to the perceived challenges I feel when encountering figures and languages that I am less literate in (i.e., numbers, data, French). It is, quite frankly, the discomfort–some blending of vulnerability and responsibility–that arises when one communicates while learning, thinking, processing. There is always recourse, though, to leave things blank or to remain silent.

* * *

slaveship

But what does writing, discomfort, and silence (blankness) have to do with numbers and data? I’ll try to explain by turning to a context, by relating my academic work in literary study to the subject of numbers. I study Atlantic slavery and its relationship to literary production. The archival materials and texts affiliated with the Atlantic slave trade have been read as documents that reveal the ways in which lives of the enslaved were reduced and dehumanized by violent abstraction. That is, ledgers, balance books, nautical journals and other accounts of the transatlantic slave trade converted captives into commodities, lives indexed by numbers and figures. Take for instance Stephanie Smallwood’s description, in Saltwater Slavery: A Middle Passage from Africa to American Diaspora (2007), of how ledgers rationalized the violent logic of the slave trade:

 “The ledger’s double-entry pages and the neat grid of the invoice gave purposeful shape to the story they told. Through their graphic simplicity and economy, invoices and ledgers effaced the personal histories that fueled the slaving economy. Containing only what could fit within the clean lines of their columns and rows, they reduced an enormous system of traffic in human commodities to a concise chronicle of quantitative ‘facts.’… Instruments such as these did their work, then, while concealing the messiness of history, erasing from view the politics that underlay the neat account keeping” (98).

In spite of the violent accountings of the slave trade, practitioners of the humanities–historians and literary scholars in particular–have been able to supply nuance, variation, and interpretation to realities that are gestured at but not revealed by the neatness of numbers, charts, and graphs. In the area of slavery studies, robust and incisive work has emerged from scholars who engage with and rethink the politics, ethics, and historical contexts that adjoin the quantitative facts and the administrative records of the slave trade. This is evidenced by recent scholarly gatherings, like “‘Against Recovery?’: Slavery, Freedom, and the Archive,” and digital projects, like Vincent Brown’s cartographic narrative of an eighteenth-century slave revolt.

To return to the question: what does writing, discomfort, and silence have to do with numbers and data?  Writing is a practice in working through the discomfort of learning whatever our subject of study might be. If there’s discomfort, I’ve told students who are silent or on the brink of giving up, it’s because learning is challenging and that thorny realities are involved in subjects we choose to study. Whether working on a formula, or analyzing a set of statistics, or deciphering the mind of Milton’s poetry, writing sets into motion a cycle of processing, self-assessing, and renewing material.

Because writing is a striving for the precise combination of words and signs that correspond to a thought and, simultaneously, an exercise that invites feelings of vulnerability and responsibility, it seems to me that writing is a practice of ethics and politics. In other words, through the process of writing, we reflect on the matter that characterizes whatever our study might be and, as a result, learn a bit more about the limits and the possibilities in what matters to us.

sievers

 Works Cited

Smallwood, Stephanie. Saltwater Slavery: A Middle Passage from Africa to American Diaspora. Cambridge: Harvard University Press,  2007.

Source of image #2 and #3: The Atlantic Slave Trade and Slave Life in the Americas: A Visual Record (Click on images for exact url address).

A resolution and a note on rubrics

I always wish I had a habit of writing down teaching realizations immediately after every class. I do it sometimes but not systematically, and I’ve lost some good ideas and observations as a result. Starting this semester, I resolve to annotate my syllabus after every class and keep a running document that includes notes ranging from things I noticed about my assignments to things that happened in class that confused students or stimulated a good conversation.

Even though I kept forgetting to write it down, I miraculously remembered one realization I had last semester about my final paper rubric, and so I’m making a note of it now as I plan for next semester. The categories for that rubric (for a compare/contrast paper) were: Thesis; Evidence and Quotation; Progress of Ideas and Paragraphs; Grammar and Spelling; and Clarity. I realized while grading the papers that I wished I had a category that assesses to what degree the students understood the texts. I had assumed that all the categories together would address that question, but I discovered that I wanted a category dedicated entirely to that question.

The rubric in question.

The rubric in question.

One of my students, for example, wrote a thesis that made some kind of coherent sense but depended on many misreadings of the texts. According to the way he was reading the texts, his thesis worked. But, his thesis was nonsense because he misunderstood the texts. I wanted to be able to applaud his understanding of what a thesis does (he had made a controversial, interesting argument that was text-specific and somewhat complex) but I didn’t feel I had enough room on my rubric to show him that his misreading of the text was a significant problem even though he had understood what I wanted from a thesis. I think I circled the “B” column for “Thesis” and the “C” or “D” column for “Evidence and Quotation” (even though he had used many quotes as evidence, quotes he misunderstood and therefore mishandled) but that didn’t seem to sufficiently describe the problem I found in his paper. I explained it in depth in my comments to him, but the circled assessment categories didn’t really match the comments closely enough.

So, next semester, in addition to the categories I already have on the rubric, I will add a category that assesses the degree to which a student has shown mastery of the text. A simple adjustment, and one of many I could make if I systematically noted my observations.

While I’m on the topic of rubrics, I would like to ask any readers for their feedback on a question I ask myself every time I make a new rubric. The categories on my rubrics aren’t weighted. No category officially counts for more or less of the total grade. I tell the students that if I had to choose, the “Thesis” and “Evidence and Quotation” categories count the most (without them, there’s no hope of getting a good grade) but do people assign actual numerical values to their categories (i.e. “Thesis” counts for 30% of the paper grade)? And if you do that, do you find it useful or too constraining? I kind of like the wiggle room that not assigning weights to each category gives me, and that’s why I continue to keep it unweighted, but sometimes I feel like I’m not being clear enough with my expectations. Any thoughts or experiences would be really appreciated!

On Time and Risk

If there’s not enough time, I could just cut to the chase: the scene is at risk without context.

***

As I write this segment of my blog entry, I’m on a train returning to New York City from a conference. Voices in the background unify into one murmur and whenever I look out the window with the silly hope of pausing on a frame, I see green foliage running, flashes of indecipherable station signs, a moment of cars going in the opposite direction. And now, I am reflecting on a roundtable discussion. My mind is a bit murky.

“Is there a war on the humanities?” This was the title of, and the question posed to, a roundtable discussion earlier this afternoon. While holding up a print version of The New York Times, the moderator began the session by referring to a recent scientific study on the social value of fiction. This prefaced the expressions of unease that later filled the room. The general sense of unease stemmed from the pressure for the humanities to define productivity in quantifiable, measurable, and instrumentalist terms. One of the speakers briefly discussed, I can’t recall his name at this moment, the pressing need to read for content, to browse for a reference. The value that was once placed on the practice of slow, immersive reading seems to be eroding.

***

Scenes are less meaningful without context. Because I taught classes scheduled in the evening and early morning, classes scheduled prior to and after the workday, I wonder often about the temporal contexts that affect scenes of learning and student performance. Like most CUNY students, my City College students frame their education around their work schedule and commute time. For each student, there’s a different set of stakes, a unique set of contexts, that shapes her/his performance. I can’t fully comprehend every set of stakes, but I appreciate when students attend class regularly with the desire to pause on an idea, in spite of temporal discomforts.

***

This student didn’t stick around after class to chat, nor was she a frequent office hours visitor. I remember the first time I read her writing. It was an essay where she brought together Aphra Behn’s Oroonoko and Italo Calvino’s essay, “Levels of Reality in Literature.” She’s a stunningly bold writer. She reappeared in my classroom a year after she wrote that paper. She registered late and arrived late to the second class of the semester. She maintained a taciturn presence. Writing assignments were turned in, hers were efficiently written; adequate. I couldn’t find the fearless voice.

Towards the end of the semester, she arrived to my office to complete an assigned recitation. She had to select lines from Paradise Lost, recite them from memory, and then discuss her understanding of those lines. Her boldness returned in a different form, through Satan’s soliloquy. She fumbled on one article but otherwise had delivered the lines perfectly, with verve. A discussion about those lines commenced, about Satan, about Milton’s experience with political defeat. I was impressed with her analysis of the lines and asked about her strategy for studying the lines. Tiredly she smiled at me, zipped up her jacket, and said that she had done it piecemeal. She made flashcards for her subway commutes and meditated on the lines during her shifts at work. “I’m working extra hours this semester,” she told me.

This made me wonder about the vocations that “[involve] both commitment and risk, boldness and vulnerability.

Fail

Failure is everywhere, we’re always talking about it, but we’re never willing to do it or encourage it.

Last week I ran a faculty development roundtable here at Baruch called “Invention in the Classroom.” Many interesting things came out of it, but one in particular has been sticking with me over the past couple days: the importance of failing, failing publicly and epically, including in the classroom.

After the roundtable, a few of us continued talking about failure, noting especially that our New York City public school students are brought up thinking that if they fail in an assignment or a test, they’re failing themselves, their parents, their teachers (whose careers are now increasingly linked to their students’ test scores), and their schools (which might even get shut down if they fail too epically, or even just a little). If our students are taught to always stick to the rules and never take a risk — to never fail — they are going to fail epically where it counts: in being inventive, inquisitive people.

At the college level, we often ask our students to “be creative” with an assignment. Here’s one example, from my course blog, of me asking that of mine:

Screen Shot 2013-03-22 at 4.03.54 PMAlmost none of them chose to write these extra posts (which I consider my own failure — one which I am attempting to work through and respond to in future assignments). In general, our students prefer clear-cut assignments that tell them exactly what the professor wants. I don’t blame them. I wanted that A, too, and I wasn’t expecting that I would get it through a semester of botched experiments.

There are a lot of ways to bring failure, and with it, creativity into the classroom, and I’d love to invite a discussion about it in the comments or elsewhere. In fact, I’m already thinking about a faculty roundtable for next semester specifically about failure.

But in the meantime, I wanted to end this post with one idea about how to bring failure into the classroom.

Model failure yourself. Or, to put it another way, model risk taking. Erica Kaufman, the faculty member and Schwartz Communication Fellow who helped us lead the roundtable, told us about asking her students to use technologies in assignments that she herself hadn’t mastered. (One such experiment was written about in the Ticker last week.) It takes guts to go to a room full of students and say (and this is an imaginative recreation of what she might have said): “I don’t know everything about how to use the 3-D printer/video editing software/animation software that I’m asking you to use, and I’m not sure exactly what we’re going to get out of this assignment, but I have a gut instinct that creating something physical that relates to your research will be instructive, and will ultimately help you figure out your argument.” It also pays off. By creating assignments that ask her students to fail again and again, hit a wall, and then by helping them to gather strength, look around them, and move beyond their failure, and by modeling a willingness to fail herself, Erica gets her students to consistently produce work that is stunning, mature, risky, and thoughtful.You can see one of her course blogs here. (And search her other course blogs from there.) It’s worth looking at her assignments, and considering how much risk they require of her students. Look at the “Our Blog!” page on the course site and you’ll see some of the things they wrote and made. The payoff seems obvious.

Did You Do the Reading?

If you’ve taught a College Now course, you know that inevitably, teaching in a program designed to give younger students a taste of college involves explicitly targeting a set of life skills in addition to course content.  College Now is a program through which NYC public high school students can take certain CUNY courses for college credit.  After class, I find myself helping students take the plunge into dialing the number of the tech help desk to troubleshoot a computer login problem, or walking students through the various ways they can find my email address if they’ve misplaced the course syllabus.

And although I generally teach the course the same way I do with undergrads, I do end up bulking up my systems of accountability and scaffolding of assignments.  I require students to do a little more to respond to weekly readings, break larger assignments down into smaller steps, etc.  These little changes have me thinking about how these kinds of accountability systems can be perceived as micromanaging or even condescending, but how if done effectively, they can vastly strengthen learning experiences in many educational contexts.

I’ve noticed I’m quick to assume that the idyllic Midwestern liberal arts college experience I had, in which most of my courses followed the model of “read something and come in and talk about it,” is the educational ideal.  And while I took many wonderful classes, some course titles come to mind from which I remember literally nothing.  As a teacher, I’m often mock (sort of) horrified when friends—people I view as successful, smart adults—dismissively reference all of the assigned readings they didn’t do in college.  But then I remember those long-forgotten books for classes outside my own major that I acquired but rarely opened.  What was in them?

In my last semester of graduate school coursework, I took a class outside of my discipline that turned out to have a tiny student enrollment.  I felt out of my element and awkwardly in the spotlight.  Rather than having to post a discussion question or the equivalent in response to each week’s reading, we were assigned to hand in a more thorough weekly summary/response in writing.  This was more accountability than I was used to in graduate school, and it was uncomfortable at first.  But oh how I read, wrote, spoke, and ultimately… remembered.  The same goes for knowledge I acquired while studying for recent comprehensive exams.  These structures of accountability unquestionably compelled me to learn more efficiently and effectively than I often have.

Although being a student (especially a graduate student) means being responsible for one’s own learning, teacher-imposed structures for recording and responding to course content have a huge impact on what kind of learning takes place.  Systems of holding students accountable for learning come in an infinite array of forms. They are obviously not only for College Now students.  This is hardly a new or unusual idea, but it’s an important one—one that I wish even some of my own teachers had chosen to take more seriously.

Competition Piece

In high school, I participated in a large-scale competitive festival of performances by high school drama clubs. This was not the beginning of my interest in theatre-making but it was a turning point for me. The production process was so intense that it was not until I had graduated college and moved to Poland to work with a professional experimental ensemble that I found something to match it.

My high school, Cambridge Rindge & Latin School, was a participant in the Massachusetts High School Drama Guild Festival, which we simply called “Festival.” I remember the competition rules exactly: Each high school sent a forty-minute production to compete. Five minutes were allowed for set-up and for strike. These time limits were strictly enforced and exceeding them meant disqualification. I remember practicing one year, over and over, to ensure the set-up of a fairly massive stage design in under five minutes. Putting up the set was as precisely choreographed as the show itself.

Comic and Tragic Masks: The MHSDG Logo

[Read more...]

Really Writing Writing Rubrics

I’m writing this post from the Landmark Ballroom, Salon 4, in the Renaissance Hotel in St. Louis, MO. Even more specifically, I just left a session at the Conference on College Composition and Communication titled, “Teaching Reading and Writing in New Media,” featuring presentations by Barclay Barrios, Richard E. Miller, and Cynthia Selfe. Their talks were titled: “A Funny Thing Happened on the Way to Digital Literacy…” (Barrios), “Learning by Doing: A Year of Thinking in Public” (Miller), and “A More Capacious Conception: Long-Form Scholarship in Digital Environments” (Selfe). For me, what resonated across the presentations was the idea that the way that writing is composed and presented (whether to the public, to a select group of friends, or to instructors/colleagues) is changing–gravitating rapidly towards what Miller refers to as an “eternal future on screen.”

For the past few weeks, I’ve been trying to find the language to write about something that happened in my Composition 2 class, a class that revolves around the theme, “Pay Attention: Or, What is our Brain on the Internet?” Thus far, we’ve spent the semester working through texts that focus on our relationship with the Internet—how the web changes the way we interact with the basic tenets of a traditional composition classroom—writing and reading and thinking. We began by reading Nicholas Carr, who clearly states, “What the Net seems to be doing is chipping away my capacity for concentration and contemplation. My mind now expects to take in information the way the Net distributes it: in a swiftly moving steam of particles. Once I was a scuba diver in the sea of words. Now I zip along the surface like a guy on a Jet Ski.” (“Is Google Making Us Stupid?”)

I’m not sure that I agree with Carr’s point, yet when faced with my first batch of papers to grade, I literally didn’t know what to do. After weeks of reading and writing together, and after weeks of conversation and thinking together, I was at a total loss. I had no idea what to do with this stack of twenty-eight, typed and printed (12pt. font/double-spaced), three to five page papers. I find myself thinking about how, in this class, we use technology, we blog, but we also write papers and every single student must purchase the required books for the class—the print books—no e-readers, we hold the books in our hands, the pages are turned and folded, and we explore a world where those things might no longer exist. It feels crucial to me that I do not fully abandon the formal typed paper, but what do I do with them? And, how do I approach reading this form of writing, when I am so accustomed to the experience of reading the student blogs—the low stakes posts they make on a regular basis. Has the internet done something to my assessing brain? Why now?

A few thoughts (in which I tip my hat to Cathy Davidson, whose Now You See It is required reading in this course, and whose NY Times response helped me think through the issue at hand:

  •  One of my priorities when creating essay assignments is that they should be either student-generated, or at least should propose a question that is open enough so that no two papers will deal with the same idea or argument.
  • If my modus operandi behind assignment design is to privilege student input and curiosities, why not do the same with my assessment process?
  • Baruch students almost always ask for a rubric so they can see “how” they are being evaluated–they seem accustomed to rubric-based grading, and might even “like” it.

After thinking through these things, I decided to see what would happen if I asked my students to design their own rubric, to give them a chance to really think through how they are graded and how they wish they were graded.

The above image is what the class decided to do after being asked to write about their experiences being graded on their writing. As a class, they decided to generated a list of terms that felt important and put it all up on the board. I tried to relinquish my authority to the class–and basically kept silent as they generated this list. I want to point to a few terms that I was interested in and surprised by–effort, creativity, compelling, arouse curiosity.

The class then decided to take their “key terms” and organize them by category–they then turned these categories into the prose that filled the boxes of the rubric.

Again, I was surprised and fascinated by the decisions the class made–particularly in the category of “writing style” and its link to “organization.”I felt as though the rubric the class wanted and created was far better than any rubric I’ve used in the past.

But, in the piece of process writing we did after creating the rubric, a number of students expressed frustration with the task–they wanted the teacher to tell them what to do, despite the fact that those who voiced displeasure were the same students who demanded that terms like “creativity” be included.

I’m still not totally sure what to make of this experience. But, I do wonder if the same issues raised in the panel discussion I attended (online writing, the “end of privacy” when all writing is public) might have something to do with the way the class handled the task at hand. Do students now value their own ability to create more than ever? Are students learning to take responsibility for what they write and  how they write it (since they are essentially producing more language than ever)? What does this mean for the world of assessment and high stakes assignments? And, what do we do as writing teachers?

 

Facebook: The Third ‘R’?

http://doctortext-info.blogspot.com/2009/08/techniques-for-custom-research-paper.html

How much writing did you do as a first semester undergraduate? 15 pages? 30? 22? 2?

How much should a first semester undergraduate write?

I’ve been thinking about the answer to that second question since I met with a student— I’ll call her Jane—in the midst of a routine day of individual appointments with Introduction to Theatre students. Immediately after I had made an introduction in her class in the early days of the semester, Jane emailed me seeking general feedback on her writing– she is a transfer student from another CUNY college, and is eager to take advantage of Baruch’s resources now that she’s here. Unlike the majority of students who utilize the services we offer when supporting THE1041C, Jane wasn’t panicked about a soon-to-be-due assignment, but wanted a kind of general consultation on her academic writing skills. I asked Jane to send me some samples of her written work, and she told me that so far, she only had blog assignments.

When we met, we spoke about her approach to these blog entries; it was clear that she had given them some thought, but her sentence structure was often confusing, and it took me repeated readings to fully grasp her meaning. In most of her blog entries, she was beating around the bush of her argument or main idea. This isn’t an uncommon problem; I face it all of the time in my own writing, and it is among the biggest issues that our students face.

Jane’s eagerness to write more was what was uncommon. As we talked, she peppered me with questions. How could she improve her writing? What should she be doing differently? What kinds of exercises would help her improve her writing on her own? I had never before had a student actively seeking additional written work, so I asked her about the assignments she had coming up in the semester. I discovered that Jane was not being asked to write very much at all. Out of four classes, her longest assignment was a four-page paper. After talking with her a bit more, a few questions kept popping up:

How do we negotiate the balance between boldly experimenting with new technology and maintaining certain (old) standards of rigor? This question comes out of the sheer lack of quantity (yes, not always quality, but important nonetheless) of writing that I saw this student being challenged with, thanks to word-capped Facebook and blog assignments. Often, adventurous faculty members are juggling many different assessment elements at once– course blogs, maybe a course wiki, too, and then oral presentations, low-stakes writing in class, plus quizzes and finals. Your syllabus is busting out before you’ve even gotten to factor in class participation. So it’s not hard to imagine that having students write  extended essays might be what gets lost in the shuffle.

How do we make the assignment diversity feel relevant, not random? Jane was a little self-conscious about her blog posts, confessing that she wasn’t sure of the expectations in terms of formality. But, as I gave her feedback on them, she also defended herself; these weren’t really evaluated, she explained, they were just graded on the basis of whether she had done them or not. She felt they were an after-thought, and so, that’s how she thought of them: after. (Click here for my own reflections on the challenges and triumphs of course blogging, here for a course blogger superstar story, and here for much more about the phenomenal Blogs@Baruch and profs who are using it to thrilling ends.)

Can we teach code-switching within online social networks? Jane was not assigned any papers in her Sociology course, either. The class has a Facebook wall, where they post pertinent links and have lively conversation about readings and class discussions—even the organizing logic of the course is debated on the Facebook page, which looked to me to be a healthy and vibrant online commons. Still, the Facebook page comments are either 250-300 words or 420 characters. Since Jane is likely using Facebook to communicate with her friends and contacts, too, how will this Sociology professor go about making the distinction between one mode of commenting and another?

Could Jane’s lack of high-stakes writing assignments have to do with work-avoidance on the part of her Instructors (and so what if it does)? Are Jane’s assignments—blog posts about 18th century acting techniques and Facebook comments in response to Sociology theory– examples of radical teaching, or just radical avoidance of the time-consuming task of reading through an 8-10 page (or 10-15 page) academic paper? None of her classes culminated with one of those. (In her Math class, Jane had no writing. In her Great Works class, the bulk of assignments were short—very short, 150 word assignments identifying a certain theme in the literature they were reading.) As is the norm within CUNY, half of Jane’s faculty is adjunct; adjuncts are generally only getting paid for one hour of work outside of their time in the classroom. A Facebook page can easily be monitored in one hour of work, so having students compose 420 characters at a pop could seem like a good way to minimize faculty labor while shaking up the tired old models, too. But there is a vast qualitative difference between infusing your syllabus with a diversity of learning objectives through multiple learning styles and creatively trying to avoid grading 10-page papers from 30+ students.

Are Jane’s assignments  preparing her for future employment challenges? The ability to communicate short, coherent messages is a fundamental expectation of many, many jobs. Just this year, at my “side gig,” I found myself parsing copy for a website, brochure, and even the 140 characters allotted for a web advertising button. These kinds of tasks will await Jane in every one of the fields she expressed interest in pursuing.

Still, these jobs will also expect the ability to sustain an argument (or inquiry into a topic or question)—exactly what is exercised in writing the long essay. Indeed, my friend who does just the kind of work Jane is interested in—communications for a policy organization—is called upon to write everything from one-page letters to the Mexican parliament to lengthy research reports on human rights abuses in Cuba. He is generally not the one tapped to write the blog posts or tweets for his organization, but someone else there is. So if we are giving students Facebook comments and blog posts as assignments, what kind of an evaluative standard should we use to ensure that they’re not just throw-away writings, but reach the kind of level that may one day be expected of them professionally?

I’m not advocating that we willy-nilly unleash a bevy of high-stakes writing assignments on our students, or mandate a standard number of pages of “academic writing” expected of each student. This post is appropriately full of questions, not answers. (And I would be remiss if I didn’t mention that Write-to-Learn strategies can and should be employed with incredible effectiveness.) And yet, it seems fairly clear that I saw something else happening in Jane’s coursework, and that something seems to be connected to a very worthy kind of experimentation on the part of her instructors. We can’t draw hard and fast conclusions from any one student’s anecdotal experience– and it is important also to mention that Jane was absolutely inspired by many of her classes and professors, and she was motivated to master their individual challenges. And yet, the question nags– what could explain this?– that an undergraduate could be writing so little? And what would you recommend to Jane?

Clear as Mud

Page A15 of the New York Times on March 7th looked suspiciously like a story from The Onion about the tangled mess that is teacher evaluation in New York City public schools. Winning the award for the most understated headline of the year, “Evaluating New York Teachers, Perhaps the Numbers Do Lie,” Michael Winerip tells the (predictably?) sad story of Stacey Isaacson, a 7th grade English and Social Studies teacher at the Lab school, described as “very dedicated,” “wonderful,” and “one of a kind,” by teachers, students, and principals alike.

So why, then, is poor Ms. Isaacson ranked in the 7th percentile of city teachers when it comes to student academic progress?

Because of this formula, designed to calculate a teacher’s value-added score by the Department of Education’s “accountability experts” (satirists, start your engines):

Click to view full size.


As someone who once taught for the NYC Department of Education and is also a product of it, I wasn’t really surprised that they had gotten it all wrong. I wasn’t even surprised to imagine that they would think such a formula could be an accurate method for tenure evaluation. They did, however, outdo themselves in the category of overall incoherence; not only did this tool strike me as wrong-headed, but it was also completely unintelligible. This is so unbelievably unhelpful a formula (ready-made for critique by visualization genius Edward Tufte), that no teacher could be expected to look at it and see her work (or her true challenges) reflected within it. Matrix-like in its complexity and opaque in its reasoning, it is a formula incapable of communicating what it is measuring or how a teacher might improve her practices based upon it. And from what I can tell, the variables are wonky, too.

It is not until the 16th paragraph of the article that Winerip summons the courage to try to explain the thing:

According to her “data report,” Isaacson’s students had a prior proficiency score of 3.57. “Her students were predicted to get a 3.69– based on the scores of comparable students around the city. Her students actually scored a 3.63. So Ms. Isaacson’s valued added in 3.63-3.69.” Simple enough, right? Wrong. The author– who knows he’s hit pay dirt with this one– goes on:

“These are not averages. For example, the department defines Ms. Isaacson’s 3.57 prior proficiency as ‘the average prior year proficiency rating of the students who contribute to a teacher’s value added score.”

Eh? And the calculation for her predicted score is based on 32 variables, which are plugged into a statistical model– the one that made me feel like I was, surely, reading The Onion.

Anyone reading this case study of Ms. Isaacson will naturally wonder a few things, like, “Wouldn’t it be fun to calculate what percentage of Joel Klein’s contract at Fox News Corporation represents Ms. Isaacson’s salary?” or, “Wouldn’t it be interesting to invite these statisticians to actually teach us this formula and how it works?” I frequently work on assessment at the Schwartz Institute, and it is also a built-in aspect of every course I teach. So I know that evaluating teaching and learning is a tricky thing indeed, a hall of mirrors in which you think you see the student reflected but often, you don’t.

I decided, then, to concoct my own formula, with my own variables, to evaluate the teaching that I do at Baruch in my capacity as a Fellow and an instructor of Communication Studies. What variables get in the way of student progress that cannot be accounted for after you have observed my class, read my syllabus, and tested my students for their proficiency level?

Click to view full size.

What if you really tried to articulate the variables that come into play when facing a group of students and a set of learning objectives?

Winerip explains that teachers are eligible for tenure based upon three categories: instructional practices (including observations), contribution to the school community, and student achievement (which is where the formula comes in). Now, I’ve never been much of a whiz at statistics, but maybe that’s okay. After all, if the communications people made the formulas, and the formula people made the communications, perhaps we’d all start getting somewhere?

So please—in the spirit of collaborative learning, improve upon my draft and post your own visual and/or variables in the comments section.

Testing As a Weapon

Photo credit: Robert King/Getty Images

A bill that will link individual teacher’s salaries to student performance and effectively destroy teacher job security was passed by the Florida state Senate last week.

Besides its obvious anti-unionism (pretty much business as usual for Florida politics), this bill will most likely serve to punish rather than help schools that are facing a number of difficult obstacles while rewarding those that are already relatively successful. It’s unfortunate that the war against public education in recent years is so often waged using the tools of accountability and evaluation, both concepts that might actually be put to good use. Even historian and former assistant secretary of education under Bush Diane Ravitch, a long-time supporter of standardized testing and No Child Left Behind, seems to be reversing her position on the matter in her newest book, The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education.

If the bill passes (as it most likely will given the comfortable Republican majority in the House and the willingness of Florida Governor Charlie Crist) notoriously inaccurate standardized testing outcomes will be used to evaluate teacher salaries and job security, essentially using one inaccurate form of evaluation as a foundation for another; however, it will also have a much more direct effect on learning. When Florida teachers begin “teaching to the test” in a desperate attempt to hold onto their jobs and a decent standard of living, it seems inevitable that the teaching of many important written and oral communication skills will quickly drop out of the curriculum.