http://doctortext-info.blogspot.com/2009/08/techniques-for-custom-research-paper.html

How much writing did you do as a first semester undergraduate? 15 pages? 30? 22? 2?

How much should a first semester undergraduate write?

I’ve been thinking about the answer to that second question since I met with a student— I’ll call her Jane—in the midst of a routine day of individual appointments with Introduction to Theatre students. Immediately after I had made an introduction in her class in the early days of the semester, Jane emailed me seeking general feedback on her writing– she is a transfer student from another CUNY college, and is eager to take advantage of Baruch’s resources now that she’s here. Unlike the majority of students who utilize the services we offer when supporting THE1041C, Jane wasn’t panicked about a soon-to-be-due assignment, but wanted a kind of general consultation on her academic writing skills. I asked Jane to send me some samples of her written work, and she told me that so far, she only had blog assignments.

When we met, we spoke about her approach to these blog entries; it was clear that she had given them some thought, but her sentence structure was often confusing, and it took me repeated readings to fully grasp her meaning. In most of her blog entries, she was beating around the bush of her argument or main idea. This isn’t an uncommon problem; I face it all of the time in my own writing, and it is among the biggest issues that our students face.

Jane’s eagerness to write more was what was uncommon. As we talked, she peppered me with questions. How could she improve her writing? What should she be doing differently? What kinds of exercises would help her improve her writing on her own? I had never before had a student actively seeking additional written work, so I asked her about the assignments she had coming up in the semester. I discovered that Jane was not being asked to write very much at all. Out of four classes, her longest assignment was a four-page paper. After talking with her a bit more, a few questions kept popping up:

How do we negotiate the balance between boldly experimenting with new technology and maintaining certain (old) standards of rigor? This question comes out of the sheer lack of quantity (yes, not always quality, but important nonetheless) of writing that I saw this student being challenged with, thanks to word-capped Facebook and blog assignments. Often, adventurous faculty members are juggling many different assessment elements at once– course blogs, maybe a course wiki, too, and then oral presentations, low-stakes writing in class, plus quizzes and finals. Your syllabus is busting out before you’ve even gotten to factor in class participation. So it’s not hard to imagine that having students write  extended essays might be what gets lost in the shuffle.

How do we make the assignment diversity feel relevant, not random? Jane was a little self-conscious about her blog posts, confessing that she wasn’t sure of the expectations in terms of formality. But, as I gave her feedback on them, she also defended herself; these weren’t really evaluated, she explained, they were just graded on the basis of whether she had done them or not. She felt they were an after-thought, and so, that’s how she thought of them: after. (Click here for my own reflections on the challenges and triumphs of course blogging, here for a course blogger superstar story, and here for much more about the phenomenal Blogs@Baruch and profs who are using it to thrilling ends.)

Can we teach code-switching within online social networks? Jane was not assigned any papers in her Sociology course, either. The class has a Facebook wall, where they post pertinent links and have lively conversation about readings and class discussions—even the organizing logic of the course is debated on the Facebook page, which looked to me to be a healthy and vibrant online commons. Still, the Facebook page comments are either 250-300 words or 420 characters. Since Jane is likely using Facebook to communicate with her friends and contacts, too, how will this Sociology professor go about making the distinction between one mode of commenting and another?

Could Jane’s lack of high-stakes writing assignments have to do with work-avoidance on the part of her Instructors (and so what if it does)? Are Jane’s assignments—blog posts about 18^th century acting techniques and Facebook comments in response to Sociology theory– examples of radical teaching, or just radical avoidance of the time-consuming task of reading through an 8-10 page (or 10-15 page) academic paper? None of her classes culminated with one of those. (In her Math class, Jane had no writing. In her Great Works class, the bulk of assignments were short—very short, 150 word assignments identifying a certain theme in the literature they were reading.) As is the norm within CUNY, half of Jane’s faculty is adjunct; adjuncts are generally only getting paid for one hour of work outside of their time in the classroom. A Facebook page can easily be monitored in one hour of work, so having students compose 420 characters at a pop could seem like a good way to minimize faculty labor while shaking up the tired old models, too. But there is a vast qualitative difference between infusing your syllabus with a diversity of learning objectives through multiple learning styles and creatively trying to avoid grading 10-page papers from 30+ students.

Are Jane’s assignments  preparing her for future employment challenges? The ability to communicate short, coherent messages is a fundamental expectation of many, many jobs. Just this year, at my “side gig,” I found myself parsing copy for a website, brochure, and even the 140 characters allotted for a web advertising button. These kinds of tasks will await Jane in every one of the fields she expressed interest in pursuing.

Still, these jobs will also expect the ability to sustain an argument (or inquiry into a topic or question)—exactly what is exercised in writing the long essay. Indeed, my friend who does just the kind of work Jane is interested in—communications for a policy organization—is called upon to write everything from one-page letters to the Mexican parliament to lengthy research reports on human rights abuses in Cuba. He is generally not the one tapped to write the blog posts or tweets for his organization, but someone else there is. So if we are giving students Facebook comments and blog posts as assignments, what kind of an evaluative standard should we use to ensure that they’re not just throw-away writings, but reach the kind of level that may one day be expected of them professionally?

I’m not advocating that we willy-nilly unleash a bevy of high-stakes writing assignments on our students, or mandate a standard number of pages of “academic writing” expected of each student. This post is appropriately full of questions, not answers. (And I would be remiss if I didn’t mention that Write-to-Learn strategies can and should be employed with incredible effectiveness.) And yet, it seems fairly clear that I saw something else happening in Jane’s coursework, and that something seems to be connected to a very worthy kind of experimentation on the part of her instructors. We can’t draw hard and fast conclusions from any one student’s anecdotal experience– and it is important also to mention that Jane was absolutely inspired by many of her classes and professors, and she was motivated to master their individual challenges. And yet, the question nags– what could explain this?– that an undergraduate could be writing so little? And what would you recommend to Jane?

So why, then, is poor Ms. Isaacson ranked in the 7th percentile of city teachers when it comes to student academic progress?

Because of this formula, designed to calculate a teacher’s value-added score by the Department of Education’s “accountability experts” (satirists, start your engines):

Click to view full size.

As someone who once taught for the NYC Department of Education and is also a product of it, I wasn’t really surprised that they had gotten it all wrong. I wasn’t even surprised to imagine that they would think such a formula could be an accurate method for tenure evaluation. They did, however, outdo themselves in the category of overall incoherence; not only did this tool strike me as wrong-headed, but it was also completely unintelligible. This is so unbelievably unhelpful a formula (ready-made for critique by visualization genius Edward Tufte), that no teacher could be expected to look at it and see her work (or her true challenges) reflected within it. Matrix-like in its complexity and opaque in its reasoning, it is a formula incapable of communicating what it is measuring or how a teacher might improve her practices based upon it. And from what I can tell, the variables are wonky, too.

It is not until the 16th paragraph of the article that Winerip summons the courage to try to explain the thing:

According to her “data report,” Isaacson’s students had a prior proficiency score of 3.57. “Her students were predicted to get a 3.69– based on the scores of comparable students around the city. Her students actually scored a 3.63. So Ms. Isaacson’s valued added in 3.63-3.69.” Simple enough, right? Wrong. The author– who knows he’s hit pay dirt with this one– goes on:

“These are not averages. For example, the department defines Ms. Isaacson’s 3.57 prior proficiency as ‘the average prior year proficiency rating of the students who contribute to a teacher’s value added score.”

Eh? And the calculation for her predicted score is based on 32 variables, which are plugged into a statistical model– the one that made me feel like I was, surely, reading The Onion.

Anyone reading this case study of Ms. Isaacson will naturally wonder a few things, like, “Wouldn’t it be fun to calculate what percentage of Joel Klein’s contract at Fox News Corporation represents Ms. Isaacson’s salary?” or, “Wouldn’t it be interesting to invite these statisticians to actually teach us this formula and how it works?” I frequently work on assessment at the Schwartz Institute, and it is also a built-in aspect of every course I teach. So I know that evaluating teaching and learning is a tricky thing indeed, a hall of mirrors in which you think you see the student reflected but often, you don’t.

I decided, then, to concoct my own formula, with my own variables, to evaluate the teaching that I do at Baruch in my capacity as a Fellow and an instructor of Communication Studies. What variables get in the way of student progress that cannot be accounted for after you have observed my class, read my syllabus, and tested my students for their proficiency level?

Click to view full size.

What if you really tried to articulate the variables that come into play when facing a group of students and a set of learning objectives?

Winerip explains that teachers are eligible for tenure based upon three categories: instructional practices (including observations), contribution to the school community, and student achievement (which is where the formula comes in). Now, I’ve never been much of a whiz at statistics, but maybe that’s okay. After all, if the communications people made the formulas, and the formula people made the communications, perhaps we’d all start getting somewhere?

So please—in the spirit of collaborative learning, improve upon my draft and post your own visual and/or variables in the comments section.

Photo credit: Robert King/Getty Images

A bill that will link individual teacher’s salaries to student performance and effectively destroy teacher job security was passed by the Florida state Senate last week.

Besides its obvious anti-unionism (pretty much business as usual for Florida politics), this bill will most likely serve to punish rather than help schools that are facing a number of difficult obstacles while rewarding those that are already relatively successful. It’s unfortunate that the war against public education in recent years is so often waged using the tools of accountability and evaluation, both concepts that might actually be put to good use. Even historian and former assistant secretary of education under Bush Diane Ravitch, a long-time supporter of standardized testing and No Child Left Behind, seems to be reversing her position on the matter in her newest book, The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education.

If the bill passes (as it most likely will given the comfortable Republican majority in the House and the willingness of Florida Governor Charlie Crist) notoriously inaccurate standardized testing outcomes will be used to evaluate teacher salaries and job security, essentially using one inaccurate form of evaluation as a foundation for another; however, it will also have a much more direct effect on learning. When Florida teachers begin “teaching to the test” in a desperate attempt to hold onto their jobs and a decent standard of living, it seems inevitable that the teaching of many important written and oral communication skills will quickly drop out of the curriculum.

photo credit: Sara. Nel

After doing several workshops for students planning to take the CUNY Proficiency Exam (CPE) I’ve been thinking about some fairly basic questions about standardized testing that are nonetheless important ones. I’ve come to realize (as have other Fellows at Baruch) that one of the most important functions of these workshops is to alleviate student anxiety. While some students do not seem to worry too much about the exam, many (some of them excellent students) become rather anxious especially in regards to the time constraints. This raises a number of questions for me regarding the effectiveness of this form of assessment. Are we really setting up a situation that accurately measures student performance of these skills given the stress of the testing situation? According to this article, we aren’t.

As health blogger Laurie Pawlik-Kienlen points out, “Scientists have long known that long-term stress impairs brain cell communication, but they’re just now learning that even short-term stress – such as a few hours of anxiety – can negatively affect cognitive skills.” Pawlik-Kienlen cites research from the University of California (Irvine) School of Medicine as well as the Laboratory of Stress Research at Douglas Hospital Research Center to make this point. Given this negative affect of stress on memory it would seem that we are setting up students for failure. Of course, it could be argued that the anxiety-producing test situation is preparation for stress soon to be experienced by students in the work world. If this were the case, why wouldn’t we coach students on ways to manage this type of stress early in their educational careers? In general I understand the need for assessment of student learning; however, I wonder if it isn’t time for us to start thinking about some different ways of accomplishing this goal outside of the traditional timed exam.

I finally figured out what to write about for Cacophony! Following the advice of my colleagues at the Bernard L. Schwartz Communication Institute, the best way to approach this was to write about something I am familiar with in the context of my work.  As a professor myself, I set specific guidelines and objectives when giving assignments to my students in order to avoid writer’s block because of the openness of possibilities. I don’t want to curtail, however: Cacophony’s open posting policy makes it versatile and unique.

I hope this post gives some basic guidelines for anyone out there interested in organizing debates as a classroom assignment.  The topics of the debates I am coaching are in the 12th Edition of the Management and Society textbook issued by the Department of Management at Baruch College. But you can device your own and have students do a little research to defend their positions.

The first step is to assign students to groups and divide the groups into PRO and CON sides of a given topic.   Then, provide precise instructions about the format of the debate.  For example, one format consist of a ten minute opening presentation, followed by a five minute period for rebuttal, and three minutes for conclusions, going back and forth between the PRO and CON side.  Ten minutes for the PRO, Ten for the CON; five minutes for the PRO, five for the CON; and three minutes for the PRO, and three for the CON. You can make them longer depending on the number of participants and the time available.

Make sure students understand that the objective is to persuade the audience that their point of view (in the debate) is the most valid: they need to make arguments.

In the beginning, they should introduce themselves, the issue, the point they are defending and any terms that might be unfamiliar or that might take a particular meaning in the context of the debate.  For example, in a debate that deals with whether genetically modified foods should be labeled, it is necessary to know from the beginning what constitutes a genetically modified food product.

Encourage them to read the materials a couple of times (in the management course I coach these are organized in chapters), even the reading for the opposite team.  In that way they can figure out a strategy to organize their presentation as well as anticipate the points are going to be brought up against their arguments.  It’s also important for students to practice their entire presentation out loud so they have an idea of time management as they become familiar with public speaking.  In terms of oral presentation skills,  you should emphasize to the debaters that they should not read, and should maintain eye contact with the audience,  which is a non-verbal way of engaging their attention.  Index cards are an acceptable way of keeping track of the order of the arguments they will stress, but in order to avoid reading too much from them,  suggest they write bullet points, rather than entire sentences.

If they are using numerical data such as statistics and/or percentages, remind your students that if they are hard to understand, the audience will just glaze over them.  Quantitative data should be easy to read and understand and should make a strong point.  If they are quoting textbooks or the internet, make sure they cite valid sources and not just random articles (especially online),  and that they have those sources (author’s names particularly) readily available during the debate, in case someone asks.

Time does not have to be equally split, but all students in a team must participate.  Have students dress professionally (although this is not a strict requirement).  Attire is a non verbal language that reveals many things, and it is difficult to find credible someone wearing an oversized sweater whose sleeves are longer than the arms. Lastly, remind students to keep their language appropriate and to keep their composure.   Debates can get heated,  but for as much as a Jerry Springer fight will definitely engage the audience, the loudest people are usually revealing insecurity.

The end of each debate could be marked by an open Q&A period where the audience can participate and ask questions or comments to the presenters.  Here you can explain how the topic is still current and give an informal assessment of the students’ participation.

Rather than focusing on all of the problems and tensions around these issues within some (not all) universities, I thought I might mention a few basic elements often emphasized by community-based organizations:

First, assessment should be truly collaborative or it can quickly become extremely divisive. Transparency seems really important here. Asking for all kinds of information about someone’s classroom, students, and teaching without being clear about how that information will be used can be a great way to alienate faculty members.

This leads to the second point, which is that assessment should serve as a means of improving the overall quality of education in a particular department or discipline or university rather than as a policing mechanism. While it’s important to be aware of areas that need improvement, highlighting best practices is equally, if not more, important.

Finally it seems important to start and finish with the people actually doing the work, in this case, faculty members teaching writing and using writing as a teaching tool. Being aware of the needs of these folks allows the assessment to be more than charts and graphs. This way the information gleaned from this assessment project can be put to practical use. This is also a good incentive for faculty members to cooperate and provide useful data. It can even make it possible to enlist their help more directly. While faculty and administration often have different priorities, they don’t have to conflict. I think both groups have some stake in assessment and, if designed and implemented properly, it can help both meet their goals.

A few weeks ago, I logged on to VOCAT for the first time, and as watched the video of a student’s rehearsal for their presentation, I was surprised to hear my own voice. I was sitting near the camera, and focused on the students as they went through their Powerpoint slides. Maybe because the camera was pointed towards them, at the time of the recording I was unaware that I was also recording myself.

And this made for a kind of unexpected self-assessment, along with the student assessment I was prepared to do. I’ve often wondered if my voice is too low, if I repeat myself too much, if what I’m saying makes any sense, if what I’m saying is more helpful than confusing to my students. And I realized, listening to myself talk to a student on the VOCAT video, that I’ve spent six years of graduate school trying to get better at absorbing what I read, and better at writing clearly. But I haven’t put any sustained or rigorous effort into getting better at speaking.

For me, the VOCAT incident, the unexpected switch of the assessment tool back on the assessor, made me realize how alone I have felt with this part of teaching. The first day of your adjunct job: the door shuts behind you, it is just you and students. A professor visits my class for one session during the semester, sometimes they don’t stay for the whole class. Their written assessment is usually generous and they’ve all talked with me after the class to offer encouragement and the wisdom of their experience. But, you know, the rest of the time, it is just you in there. Talking and talking. Wondering if the students are falling asleep because they’ve just eaten lunch, or is it the lulling drone of my voice? I know there are books and articles out there I could be reading on how to effectively engage a class. And I’ve sat in on other professor’s classes to see what I pick up from the way they engage a class. George Shulman at NYU Gallatin showed me how effective it is to value every student’s contribution, repeating it, rephrasing it, writing it on the board. Heidi Kruger at the New School held me spellbound with her intense, low whisper. Sekou Sundiata at the New School moved around the class like we were the orchestra and he was conducting us.

But, what works for me, and for my students, on this particular subject? I hadn’t really focused on that so much. Which is weird, given how, you know, important oral communication skills are in teaching.  Should the VOCAT assessment tool be turned on teachers? Well, I wouldn’t volunteer. But, when confronted with it, I thought it showed me some things that I should be aware of.

This brings me to the connection between writing and speaking. At the recent WAC conference, several people brought up the fact that writing often, in different forms, helps people become better writers. Speaking about writing also improves writing.  We talk about students ‘finding their own voice.’ One impediment to that might be that students are reading authors whose voices are quite different than their own. Often when I’m working with students on their presentation, I’ll ask them to summarize or draw a conclusion from their research. They articulate clear, original, logically organized claims aloud. But, when it comes to the formal work, they leave this out. Why? The answer I’ve heard more than once was, “But, that is just my opinion.”

What I want students to do, what I’ve heard other teachers say they want students to do, is enter a conversation with the authors they cite. What I’ve seen happen too often is a student articulating their own view, then summarizing an author’s view, using the author’s own style. How can we yoke them together?

One possible way might be to value thought when it is articulated aloud, not just in print. And one way to do this might be to film it, to actually turn the light and focus on recording speaking a thought, the way writing records a thought.

At the WAC meeting, Thomas Meechum and Karen Gregory’s documentary about the writing process in professor Michele Pacht’s class showed students responding to questions about their opinions about graffiti. I wondered if the heightened attention of the camera on the spoken thoughts helped the students to value their thoughts enough to commit them to print. I wonder if I should review the recording of my voice, talking to my students, as many times as I am reviewing the drafts of my dissertation proposal. I kind of think I should.

The institution of grading students on an A through F scale has done a horrible disservice to education. It has falsely given the impression to generations of students that the teacher or the professor has some ultimate authority over the value of their work, as if their own assessment of what they were doing was somehow secondary. The result of this institution is a division among most students into two groups — a group motivated by competition and the drive for the teacher’s approval, and a group lacking in motivation with little interest in the teacher’s assessment. What is missing all too often among students in both of these groups is the sense that their education is their own.

I have found several methods of correcting this problem that work within the extant system. By far the best of these methods is to ask students to write self-evaluations. All teachers who have ever taught a graded course know that students approach them to apologize for not having completed an assignment — the proverbial “my dog ate my homework” moment. The self-evaluation taps into the students’ innate authority over their work which is too often evident only in their apologies. If you ask students to write about how they have approached the assignments of the class and you ask them to write about their own perceptions of their strengths and weaknesses, they very quickly begin to realize their own agency in the learning process and to begin take responsibility for their own education.

Of course the best thing, I think, would simply be to do away with grades and grading altogether. I know that for many people this suggestion amounts to advocating “mere anarchy.” Without the carrot and stick, there would be no motivation anywhere among students, no assessment, no accountability. It’s true that in all likelihood, the students who come to me to rehearse their powerpoint presentation are not motivated purely by their own desire to improve. Their presentations are graded and they want to get a good grade. Well, perhaps this is true. But in a time when the movement for standards has taken over every level of education, I find some comfort in recollecting a different ideal.

VOCAT is a teaching and assessment web application. It is the fruit of a collaboration between the Schwartz Institute and mad genius code-poets at Cast Iron Coding, Zach Davis and Lucas Thurston. It is still very much in development (perpetually so) but is already in use in introductory speech communication and theater courses as well as a number of assessment projects. Our career center used it effectively a few semesters ago as well. To date, approximately 3200 Baruch students have used the tool.

VOCAT was developed in recognition of the principle that careful, guided review of video recordings of their oral presentations (or of any performance, for that matter) can be remarkably effective for aiding students in becoming confident, purposeful and effective speakers. It serves as a means for instructors to easily provide feedback on student presentations. It enables students to access videos of their performances as well as instructor feedback and to respond to both. It likewise aggregates recorded presentations and instructor feedback for each user and offers an informative snapshot of a student’s work and progress over the course of a given term or even an entire academic career. Presentations can be scored live, as students perform, or asynchronously once the videos have been uploaded. (Our turnaround time at this stage is between one and seven days depending on how many sections are using the tool at once — once some of the key steps happen server-side, turnaround time will not be as much of a concern.) Built on the open source TYPO3 content management system, it is a flexible, extensible and scalable web application that can be used at once as a teaching tool and as a means of data collection for research or other assessment purposes. (Screenshots are available here. I am also happy to share demo login info with anyone who would like to take a look — please email me at mikhail [dot] gershovich [at] baruch [dot] cuny [dot] edu.)

While VOCAT is quite feature-rich at this early stage, especially when it comes to reporting, data export, and rubric creation, we are always thinking about ways in which the tool can be made more robust and flexible. Currently, we are playing around with adding a group manager feature for group presentations, tagging for non-numeric assessment, moving from QT to Flash video, video annotation, as well as server-side video processing and in-line video and audio recording. We are also considering allowing users to choose to enable social functionality to take advantage of web 2.0 tools for sharing and commenting on one another’s work. And since, at its core, VOCAT is a tool for aggregating and responding to anything that can be uploaded, we’re thinking about other uses to which it could be put. It could easily, for example, be adapted for writing assessment. And someone once suggested that it could be useful for teaching bedside manner for medical students. Adapting VOCAT for these purposes is hardly a big deal.

The platform on which VOCAT is built is open source but the tool itself is not yet open. Right now, it is Baruch’s alone. Whether it should stay that way is a question much discussed around here. Here at the Institute we face several critical issues around open education, not the least of which is conflicting views on student access of Blogs@Baruch. In regards to VOCAT, however, the one thing constantly on my mind is the tension between an internal drive to share the tool as an open-source web application and build a community around it (there are no shortage of interested parties) and the pressures (or maybe a pernicious institutional common sense) that seem to compel us to keep VOCAT proprietary and use it to generate as much revenue as possible. I have heard arguments that VOCAT should be Baruch’s alone — that we should charge for its use and seek private funding for its deployment and development. This is a business school, after all, and I’m sure promoting and marketing VOCAT could be an interesting project for an upper division Marketing course.

Yet, I am inclined to believe that VOCAT should be shared freely and widely with other institutions and that other developers should be encouraged to develop for it. A great many more students would benefit and development would certainly be accelerated as more and more schools add features they need that could then be adopted for use here. Were VOCAT open, in other words, it would evolve quickly and probably in ways we haven’t even imagined. And that is very exiting.

In the coming months, I hope to continue to present on VOCAT and to gain insights into the roles it can play in communication intensive courses or in a communication-focused curriculum of any sort. More importantly, I would like to move towards opening it up and will work with our developers on the features and functionality that facilitate sharing. I hope also to draw upon the tremendous expertise of my friends and colleagues involved in the open education movement and learn from those who have worked with and developed various open source tools for teaching and learning. Listening to others’ ideas for VOCAT has been invaluable to thinking through what this web app could ostensibly do with the right sort of development. could be and how to best realize its full potential as a teaching tool — both in terms of deployment, training, and development.

I am currently working with the Great Works of Literature faculty at Baruch on an assessment for the Great Works course. The faculty is interested in evaluating the learning goals for the course. The first step was to talk with the faculty about what they teach, how they teach it and what they feel about it. These are always great discussions and I believe fundamental to making a good assessment. At one point a faculty member stated that reading was a central part of the course and that she was, among other things, teaching in-depth reading. I was quite struck by this thought of reading and how the Great Works of Literature course taught students to engage with different texts and make inferences to the world around them through reading.

David Frost and I thought a lot about how to incorporate reading as part of the assessment process and how to design a prompt to fit with the specifics of reading for a Great Works of Literature course. The obvious was to ask students to read a short text and respond to it. But the difficulty was to find a reading experience that mirrored what goes on in the course.  Reading literature from multicultural environments and then exploring the relationship between the different genres and cultures is an essential part of the course.  But this is not easy to assess, and as most faculty say, even to grade.

For this assessment we are going to try an experiment; a pool of short texts will be available for the faculty to choose for the prompt. The students will be reading pre and post prompt texts that might be different in author or culture but the same in length and complexity and genre. The texts will also relate to authors or literary periods that the students studied during the semester.

The hope is students will be reading, and responding to the reading, in the same way they do in the course. The second hope is we will be able to draw out meaningful information about the students experience in the course as well as any increase in comprehension and knowledge. Everyone involved in this assessment is pretty excited about this experiment and its creative use of texts for the prompt.

I am too, as I hold in my breath to see if it really works.

I was reminded today that I once drafted a set of learning goals for writing and speaking at the undergraduate level for a project headed up by our office of advisement and orientation. While these goals implicitly inform the curricular support and development work of the institute, they have not been codified beyond the document I created in 2006 (before I learned about Bloom’s taxonomy). These goals have not seen the light of day beyond their very limited original context. With that, I thought I’d post them for discussion. Take a look and let us know if you find these useful and/or whether you’d recommend revisions. Here we go:

By the end of their undergraduate experience students should be able to:

• comfortably pose pertinent questions to faculty both in and out of class
• demonstrate proficiency in a number of everyday written genres (email, letter, etc.)
• demonstrate sensitivity to audience in oral and written communication – write and speak in a manner appropriate to audience – articulate similarities and differences in addressing different audiences (email to peer vs. email to faculty, conversation with parent vs. conversation with prospective employer)
• demonstrate awareness that all communication is purposeful – each individual communication is meant to accomplish a particular goal or set of goals – sensitivity to purpose
• grasp rhetorical purpose of own written work (what is this paper, email, memo, etc. meant to accomplish? What do I need it to do? What should it accomplish?)
• articulate how they might go about accomplishing purpose of given communication (in order to accomplish X in my email to my professor, I need to make clear that Y and establish Z before making the argument that A)
• work responsibly and productively as a member of a group – to communicate appropriately with all group members
• comfortably speak before an audience – impromptu and prepared presentations
• articulate own understanding of how they can become better communicators (what do I need to work on to become a better writer/speaker?)

Discuss.

1. It’s not difficult. Considering the gong show of Blackboard’s tech problems this semester, it was almost comical how smoothly the blog functioned. A handful of students ran into some problems accessing it at certain computers, but often I found that problems encountered by students were frequently due more to lack of time and preparation on their part than any issue with the blog itself.

2. Don’t be conservative! I was. As one of my students told me at the end of the semester, “the blog was just there.” It wasn’t as dynamic as it could have been, in part because I didn’t use it to capture anything in progress. Students cut and pasted their work onto the blog, and then made the requisite comment on a post, creating a static space outside of the classroom, not a particularly engaging one. While it was satisfying to see this vast collection of interesting video clips assembled in one place—along with frequently cogent, in-depth analyses of them—I see now that I used the blog to solve a problem (that of my midterm assignment) rather than tailoring it for uses that would really suit the nature of the blog. Recent conversations with my students and others have highlighted a range of ways that it could be used in an Introductory Speech course– sharing audio files or outlines of student speech drafts that could be revised as the “audience” comments. On a related note, the public forum really does elicit strong work. When students feel the watchful eyes of their peers, the bar is set somewhere different. This makes my mouth water for the possibilities of the course blog—like facilitating peer review, for example—that I didn’t explore.

3. Be forewarned: out of sight, out of mind. In part due to #2 above, the blog can feel like that side dish you ordered but weren’t quite hungry for. It’s easy to lose track of the blog, and its implementation should be planned with an eye towards avoiding this. Usually, the material nature of grading compels you to eventually plop down on a long train ride and hit it out of the park. With the blog, not so easy. I had good intentions—I wanted to comment on posts frequently, but commenting is time-consuming, especially if students are posting 40-minute inauguration speeches. This in turn leaves less time to evaluate the work for grading purposes. From the student side, they were assigned a date for one post; once students posted, they didn’t have a strong incentive to return, which would leave me begging them to “visit the blog!” when I myself was embarrassingly behind on reading their old posts.

4. Students might be less excited about instructional technology than you are. (…How to get them more excited is part of the task.) Take ‘tagging,’ for example—it was harder than I might have imagined getting the ‘tagging’ to happen. Some assume that the ‘Sidekick generation’ will tag as if it were natural as breathing. Not so– every nineteen-year-old might know how to search YouTube, but they’re not all writing Facebook applications or even their own blogs. Making some class time available to teach students the rhyme and reason behind some aspects of the blog is arguably essential, and yet somehow easy to overlook.

The Com1010 Public Speaking Award Goes To...

5. Students love Pacino. As in past semesters, his speeches were cited with a remarkable frequency, rivaled only by Randy Pausch. This is perhaps not a surprise, since the first hit from googling “inspirational speech” is Pacino’s “peace by inches” monologue from Any Given Sunday, but still. City Hall has a less predictable—and arguably far better—dramatic monologue that I’m glad one of my students spread around.

I’ll end here with a question. As Luke articulated so well in his WordCampEd post, these open source technologies are blessedly DIY. But I can’t help feeling a little protective of the adjunct in this discussion– don’t adjuncts “do it themselves” enough? Can the full potential of Instructional Technology really be unleashed with the real limitations of the adjunct labor force operating in higher education? I’m in a distinctly lucky position as a dual-hatted Communications Fellow and adjunct; working with people jazzed and knowledgeable about these technologies has taught me tremendous amounts about how to use it and why. But how will Jane Q. Adjunct learn about the potential of a course blog, after tearing her hair out over Blackboard for months and missing the departmental meeting that announced a later workshop about blogs, all time she’s not paid for? How will Jane Q. Adjunct get excited about the potential of these tools, and why will she motivate to prioritize the time required to integrate them thoughtfully and productively in her course?

Each student holds one of the thin little clickers and answers the questions you placed in your Power Point slides. You can see the results immediately (or hide them from the class if you choose).

We were apprehensive about having to learn new software and then adjusting it to work with a Power Point presentation and a workshop we have been working on for months already. But it worked very well. The IT resources tech support person was happy to train us, it took barley half and hour. A little experimenting later and we were able to figure out how to make it work for us. It was as easy as creating additional slides to add to our Power Point. But the benefits were clear: we were able to ask students to respond to questions which then allowed us to introduce a related element of the workshop, or helped us explain a point we were making, or, at the end of the session, we were able to ask student to asses the workshop: what they learned, found useful, found challenging. After the session, with a click of a button, we printed out a report with percentage and graphical representation of the answers (see the fragment of it at the picture attached to this post). We designed very simple “yes” and “no” questions but the possibilities are endless.

The added bonus is that the box of clickers for students is brought to the classroom and then taken away after the class is over, by an IT person. You don’t even have to pick it up. Hopefully, some of our Institute’s PCs will have the Turning Point installed. You can also try it on your home PC. Give me a holler if you need help figuring it out.

Maybe it is because of their responsiveness to me that I become a softee when it comes to the evaluation of their performance. Luckily I do not have to grade them, but I talk with their professor about how they did, and, more often than not, I find myself taking their side. I want the professor to be more generous, more understanding of how nerve-wracking a presentation can be, more embracing of the students’ individual skills and needs, etc, etc. On the professor’s side, I am facing a set of extremely well organized grading scale that breaks down final grades to the smallest percentage. This is how grading should really look like, I tell myself, envying the social sciences for their apparent efficieny that messy humanities people, let alone literature buffs like me, tend to miss. Yet, I feel like a coward when the professor mentions a student’s way of being too “soft-spoken” and I let it go saying only that her “softness” comes from her cultural training as a Japanese woman. (Apologies if this comes across as relying, yet again, on stereotypes about Asian women. Obviously not all Japanese women are low-key, but I just finished reading Kyoko Mori’s memoir, Polite Lies, and I think I got at least a better appreciation of Japanese cultural normativity than I had before reading the book.) Evidently, the professor, who has earned all my respect for his superbly organized way of doing his job, cannot let himself bogged down by my remark since he has to evaluate the final product, but I am left with a sense of failure.  I wish I had a way of giving more time and space to the process, of being able to assist each student individually while I do not run around myself trying to finish up my dissertation. In my dream-world, I use a grading rubric that includes “cultural baggage” as a big bonus point because I know how heavy it gets at times and how important it is to keep carrying it on in spite of all.

Now the MTA wants to know exactly how and why we New Yorkers get around the city. When I first saw the advertisement for the survey I was skeptical. I couldn’t help but wonder if they were actually going to take our feedback seriously or if this was just a public relations move to get us all to feel a little more hopeful that better commuting days are ahead. When I got home, I went online to learn more about the survey. As it turns out, the MTA has contracted an outside firm, Nustats, to gather this information for them. Somehow, the fact that they are investing money to do this made me feel a little more confident that the MTA is actually making an attempt at genuine communication with its customers.  However, I found two things rather peculiar: The MTA does not actually mention this current survey on their web page; I actually had to google “MTA Survey” to find it. Also, the survey is not being made available to the public via the internet. In order to participate you need to either download a paper form from a PDF file or call a toll free number and take the survey over the phone. I’m curious about those choices. I’m also curious about the $500 prize they are giving out weekly to one survey participant who is to be chosen at random from a drawing. If you’re interested, go to:

http://www.nustats.com/mta/

As the presidential election approaches I find myself thinking a lot about communication between institutions and individuals and wondering how much weight does the individual voice carry. But also, how important is it that individuals feel their voices are being heard? Will the chance for $500 entice subway riders to actually pick up the phone or download the file and participate in this survey? How sincerely does the MTA actually want us to?

The inclination to read a huge Victorian novel, the capacity to untangle a metaphor in a line of verse, the desire to study and emulate a distant historical figure, the urge to ponder a concept such as Heidegger’s ontic-ontological difference over and over and around and around until it breaks through as a transformative insight — those dispositions melt away with every 100 hours of browsing, blogging, IMing, Twittering, and Facebooking.

This brings up a lot of interesting questions as educators are increasingly trying to incorporate some of these technologies into the classroom and publishers are pushing textbook content into more profitable eBooks.  Are we actually helping students by doing all of this?  Some initial studies of middle and high school students suggest that technology-intensive curricula do not improve student achievement.

Bauerlein has many interesting points in the article and makes a good case for “unplugging” some aspects of teaching and learning.  However, in my opinion, the question of whether or not technology in general improves/impairs student learning is not that interesting.  Instead, we should be focusing our assessments on understanding which technologies can be usefully employed in which aspects of the curricula.  Finding pedagogical fit for relevant technologies seems to be what we are striving towards at BLSCI.  Thus, as an institute, we undoubtedly have much to contribute to this important discussion.

As Figure 1 and 2 below show, BLSCI fellows support faculty teaching a number of distinct Communication Intensive Courses (CICs) across a variety of disciplines. As Figure 2 shows, the largest representation of faculty teaching CICs is in departments that have traditionally placed a heavy emphasis on both written and oral communication, such as English, Modern Languages, Marketing, Management, Performing Arts, Sociology and Anthropology. However, the institute has also supported CICs in departments that have not traditionally incorporated communication intensive elements into their curricula, such as Accountancy, Natural Sciences (e.g., biology, chemistry, and environmental sciences), and Computer Information Systems.

[click to enlarge] 

[click to enlarge] 

When we look at these data and hear about all of the great work going on at the Institute during our staff meetings, what we often don’t take into consideration is the amount of expansion that has taken place over the past ten years. As Figure 3 demonstrates, the number of faculty supported by BLSCI has steadily increased, reaching a peak of 144 last year. The number of faculty currently teaching CICs is nearly three times what it was ten years ago. Despite some minor fluctuations, the number of sections of CICs has also increased dramatically. Specifically, as shown in Figure 4, the number of sections of CICs offered last year is nearly five times as many as there were in 1997.

[click to enlarge] 

[click to enlarge] 

There is no doubt this kind of institutional growth contributed to BLSCI’s being awarded the Hesburgh award. However, the most interesting growth going on at the institute is arguably what happens on a more micro level among students, faculty, and fellows throughout continued mentorship and collaboration. Although we all get to observe this in our individual work, it’s often hard to demonstrate this kind of growth across the institute. As we keep on thinking about and celebrating growth at BLSCI we continue to think about ways to assess it. It’s my hope that this post will spark some ideas among readers on how we might approach this kind of assessment next semester.

Assessment, assessment, and assessment; this seems to be all I am hearing these days. Editors at academic journals inform me that if I have an article on assessment, they would be happy to publish it… Even my daughter’s 2nd grade teacher had a meeting to explain to the parents how our 7 year olds would be assessed over the school year. In my new position as Deputy Director of the Bernard L. Schwartz Communication Institute, I am currently overseeing two major assessments: one of the Institute’s programs and another of student writing in CICs. Surprisingly I find myself enjoying it immensely.

As assessment seems to be the 21st century companion to the education field I guess it is time to just jump in. I find myself fascinated and passionate about looking at and assessing students’ writing over a ten-year period and all of its infinite possibilities. However there is a small part of me that feels there is a fine line that is frequently crossed in assessment. It is when educational institutions, or even individual educators, over-invest in the assessment of whether students attain pre-established learning goals to demonstrate that students have learned. When student outcomes, in relation to pre-set learning goals, are the main goal of an assessment, outcomes can easily become the dominant product of education rather than the messy but profound experience of learning itself, which does not always produce a clear outcome. And if this becomes the assessment norm, to measure outcomes rather than transformative experience, than education runs the risk of merely accumulating material and compromising its fundamental role. This is a line I feel is dangerous for any educator to cross.

Ken Buckman, professor of Philosophy at the University of Texas, wrote in the fall issue of Thought and Action that, “the primary character of education is its transformative influence…” Therefore, experiencing the process of learning is as important if not more than the final goal of having attained an acceptable outcome or learning goal. Yet this is one of the hardest areas of learning to assess: the transformative experience.

However, there might be a way of examining student learning as a process and not an outcome. Follow the same students for several years and see what parts of their learning process have become integrated into the way they talk or write. In a sub-sample set of the written diagnostic data we are now examining the same set of students writing over a period of 3-4 years. Their vocabulary, their expectations and the complexity with which they express themselves can be analyzed as well as how this has been transformed over time. This data will not tell us whether the students did well on the final exam or whether they were able to write an “A” term paper. But we might be able to note when and if the student was going through a transformative process between the times they started writing in their first CIC course and the 3rd or 4th CIC.

There is no doubt that any educational institution needs to be able to demonstrate that its students on an aggregate level are reaching the goals and outcomes that have been put forth. However, students should also be studied as distinctive learners with unique goals and experiences. As for my part, I can’t get enough of knowing that my students might have been transformed.

When looking at the data, on average, students start the semester with scores on both the expectations and writing quality variables in the “middle” range (scores around 3). When we then looked to see if students’ scores significantly improved over the course of one semester in a CIC, there were no major findings. This was because many students started the semester out scoring high (scores from 4 to 5) on many of the variables, and thus did not have any room for improvement (as measured by the diagnostic scoring criteria).

However, when we looked at students who scored in the “low” to “middle” range on all of the variables (thanks Diana for this suggestion!) we observed statistically significant increases from the beginning to the end of the semester on all variables. These increases were consistent across disciplines and schools as well. The figures below illustrate the changes we observed in the data separately for the Weissman and Zicklin Schools.

[click to enlarge]

[click to enlarge]

Although these results are based only on a subset of the data we’ve been able to clean, match, and analyze (~ 5,000 students), they nonetheless illustrate that the work of BLSCI in creating and implementing CICs seems to be paying off for students across the board. Although most probably knew or were able to sense this already, it’s always great to have “hard data.” We would love to hear readers’ thoughts on these findings and how you see these data stacking up next to the work you’re doing with students in your own classes. Also, as always, any general thoughts and/or questions on the assessment data are welcome.