Feed on

Dirty Little Secrets

We just completed our finals here at the University of Rochester. It is probably the most frustrating time for me as a Professor. No, it’s not because I do not like grading. It’s probably closer to the opposite — grading finals gives me a terrific chance to see how well our kids write, how well they are absorbing the material and probably most important for me is to observe how they understand things and articulate themselves. Finally, I actually love writing finals each year because how often do you get a group of bright students captive for 3 hours to read what you have written?

My goal in writing a final is not simply evaluative in the strict sense, but also to give students a lesson that they might not have learned had they just come to class, or done the readings, or worked on some practice problems. A good final exam will have some questions that shed light on current issues, introduce students to new insights, and give kids a variety of ways to demonstrate that they can apply the tools of your course to the real world — in other words, to demonstrate that there is a reason aside from mental stimulation to be taking a particular course.

I teach very large classes, which makes it a lot harder to do what I want to do, but I still take the approach that my classes are 20 people in size rather than 200. So what is so frustrating about my final exam period? Here are a couple of examples, and none of these are discussed at the department level, college level, university level, in grad school as we prepare to become professors, etc. I am not sure there are answers to these things, but I thought you’d like to know some folks do try to think about these things.

  1. What exactly is an “A”? I certainly have never been told as a student nor as a professor what the awarding of an A is supposed to be recognizing. Is it simply to report the person with the best grasp of the material? If that is the case, if I were teaching Principles of Economics and Milton Friedman came in and took my class, he would get an A without ever reading a single page or attending a single lecture. Maybe that is what we want, but maybe not. Is it supposed to be awarding “value added?” So we try to mentally figure out how much “stuff” (what stuff? that’s another source of frustration and confusion) a student knows coming into the course and compare that to the “stuff” they know on the way out. Even if we agree on “what stuff” matters we are no closer to knowing what an “A” should be. How much more stuff does the student need to know for it to be an A? And should this be the same whether we are talking about an introductory level class, an upper level class or something in between? Is it the same whether someone is taking Economics as a hobby or whether it is their major? Should the amount of time and work dedicated to the class matter? Or should we coldly look at essay and exam performance? I will not tell you what I think of all of this, and what choices I ultimately make, but the list of questions is actually much longer than the few I pose above.
  2. When I grade, I do so anonymously. I only look at the numbers and the student id’s. I do not know the identity of a student until I go to submit the letter grade into the college’s grading system. I do this in order to prevent me from playing favorites or to do the opposite. This is probably the hardest policy to commit myself to. I am sympathetic to arguments on both side of this, and to be honest do not know whether my current policy makes sense. I think in a large class I am forced to do this, and my smallest class at UR has been 79 students, my largest over 300.
  3. My most serious issue with finals, grading and the like is a version of the “grade inflation” problem. Based on some extensive thinking about (1), I have come up with a few metrics for what I think a “good” student who takes each of my classes ought to know. I am especially wedded to this idea for my Intro Econ course. We can blog this at length in the future, but here is an example of what I mean. We will be awarding 165 or so Economics degrees at graduation this weekend. When someone gets a degree with our stamp on it, and indirectly therefore MY stamp of approval, I think there are things a student ought to have retained with them through college and ought to retain through their lives. But here is  wager I would surely take at even odds: over 50% of the kids who will be receiving such pieces of paper from us could not answer well a battery of 5 to 10 questions that anyone who calls themselves an economist ought to be able to answer. What might some of those questions look like? Well, they are not deep theoretical ones, or field specific questions, but rather general economic questions such as, “What important role do prices play in an economic “system”” or perhaps “where to prices come from” or “characterize how living standards have changed throughout history and vary across countries and say something about the many theories that have been advanced to explain these” and …

    In each particular class we have the same problem. For example, in my Money and Banking class this semester (about 80 students), my “raw” unadjusted class average was a 74%. That is a low C in my book. According to my “unadjusted” rubric, i.e. the one I articulate in my syllabus, an A is 94% or better, a B is 83% up to but not including 88% and so forth. And I write my paper topics, quizzes and exams so that I expect that someone who gets 85% of the points on them is a “B” student … a B student to me is someone who can give the material right back to me very similar to how I delivered it to them. In order to get an A, I expect the student to be able to synthesize the different course material, to be able to teach it to other people, and perhaps to analyze it in a way that I did not present — apply the material to many different conditions than were presented in class or the readings. How do you do that? We’ll talk about it one day. And a C student is, to me, someone that “kinda gets it” and gives the material back to me with some holes and gaps, and while being able to answer some direct and targeted questions, does not synthesize the course material at all, and does not demonstrate any ability to draw connections out of the context of the class. In my view, my class average was truly a C (it was the median as well).

    Under that original grading system, only 3 out of my 80 students genuinely achieved an A in my course. And these students ought to be recognized and commended. However, after a little “pressure” is applied from outside sources to adjust my grading from these levels, I end up “having” to award 8 grades of an A (way too high in my view), and my new class median is just about a B, with a mean of a little above a B-. (In my required core courses I am “allowed” to be a little more rigorous in my adherence to my standards). How would you feel if at the very pinnacle of the teaching year you were “asked” to compromise the very core of what you believe your students should be evaluated on? How do you think that serves my three “real” A students? And how do you think the other 5 “A” students feel? In fact, knowing this, a student in the class probably has no idea whether their grade was seriously earned or is something of a mirage. That must suck. In fact, that’s how I felt when I went to college and basically about any award that I have won in my intellectual career. I really do think I am the world’s tallest academic dwarf in the settings I find myself in.

There’s so much more to say regarding evaluation, pressure, grading inflation and the like, so I’ll stop here for now. But we’ll leave you with some more thoughts that I may address soon. What about the number of students that earn low C grades or below? Are there students passing classes that have not demonstrated an ability to comprehend the material? What is common about these students? How important is class attendance? What about differentials across departments? What about differentials between upper and lower level courses? What about policies of awarding internships a full FOUR credits of academic credit, and now even for a grade? What about allowing Independent Studies? And what about students with learning disabilities? What students are more likely to complain about grades (good ones or bad ones or ones in between and when are they likely to complain?) Should all faculty be asked to work with undergraduates? Should departments have a uniform grading policy or should they allow professors perfect autonomy in how they evaluate students? And controlling for various factors, what sorts of things do students use to make decisions about which courses to take. I can tell you beyond a shadow of a doubt that the more rigorous I make my classes, and the more “true” to my grading standards that I adhere, the far fewer students I end up enrolling. There is no question in my mind that if students knew that my median grade was a B+, and if they had to do, at most, a midterm and maybe a final paper, and just sort of be a good sport about the class to not fail, I would be the world leader in class size.  And believe me, my life would be a hell of a lot more “pleasurable” if I did that, especially if that midterm was recycled each year and especially if it were a multiple guess test.

Any one of these questions is worthy of a full essay. But I hope you can at least start to understand why this time of year is agonizing. Don’t get me wrong, the professor life is just grand, it would be silly to suggest otherwise. But I am not intending this to be a “whinefest” I am merely trying to help you guys understand that I am both cognizant of the problems of grading, and also party to it, even though I am probably one of the “hardest” graders here on campus.

OK, I am stopping. The quality of this post really is only about a C. When I think I put up a B or an A in the future I will let you know. I would only grade 2 of my last 25 posts with an A (at most).

6 Responses to “Dirty Little Secrets”

  1. blink says:

    I agree with your characterization of A, B, and C students. I think the real problem is instrument reliability: 3 hour tests do best to separate B-and-better students from the rest. Yes, writing excellent questions can yield information about the “A” qualities you mention, but with very high variance.

    Think of grades as coming with a confidence interval attached. One student is 86 (+/- 6), another is 91 (+/- 4), etc. When you are asked to reconsider your grade distribution, perhaps the request is really to ask of each student, “Can I reject the null hypothesis that this student is an ‘A’ student?” My advice: Hold the line on the “B” and be generous whenever you are unsure about the student’s true ability. If you want rigorous grades throughout the distribution, increase the number of measurements to reduce the standard errors.

  2. Greg says:

    Is it supposed to be awarding “value added?”
    -Just bomb the diagnostic! It’s an easy A!

  3. Harry says:

    I taught Lehigh students, mostly seniors and juniors, for three years as an adjunct — pardon if I already told you that, Mike.

    Grading was always a problem. I would start off by telling my students to work hard, that is, to spend a little more than an hour or two per class to deliver the best they could, and that their final grade would reflect how far they had come to mastering whatever I was telling them to master. You can guess the end of the story.

    The subject was business communications, a subjective beast if there ever was one, and my credential was from the school of hard knocks, which meant you got fired if you screwed up when meeting with the guy who wrote the check, when you explained how you made all these millions of dollars for his company, which wrote him his check.

    I think it is great when one can teach a few students a few valuable things, even if one happened to do that with about as much intention as a squirrel planting an acorn in a good place.

    You have done well so far, Wintercow, and you should be proud not just of the stars you highlighted at AHI, but a bunch more.

    Thanks for a great essay on the complexities of fair grading in our university system.

  4. Harry says:

    I have read that Chuck Schumer got 800’s on his boards. Senator Schumer had a distinguished academic career, and an enviable political career after that.

    What I am wondering is how Chuck would score today taking Wintercow’s money and banking class, assuming some of the questions were objective and not one a bullshit artist could not artfully answer.

  5. Rod says:

    I went to college in the olden days, when the main academic policy was to weed out as many students as possible in the first two years by means of “basic requirements.” My friends at Amherst had even harder basic requirements then: the same introduction to physics course the physics majors took, and a year’s course in calculus.

    Grade inflation had also not struck my college back then. Only 20 out of 340 freshmen made the Dean’s List, the qualifications for which were a B average and no grade below B-.

    While half of my class ended up being history majors, half started out as pre-meds, which meant that over 160 students were enrolled in Chemistry 101-102. The chemistry department had only seven professors/instructors, so it was the department’s mission to cut down the number of students taking higher-level courses like Organic and Physical Chemistry to a minimum. The way they did that was to base two thirds of one’s grade on three-question exams. Unlike high school chemistry (or chemistry at lesser colleges) there was no “part credit” if the answer itself was not correct according to slide-rule accuracy. Thus one got a 100, 66,33 or zero percent on any given test. What a shockeroo that was to me and my classmates, most of which had been at the top of their high school classes and had scored high on the college boards. Labs counted for a third of one’s grade, but there, too, the answer had to be right, and after that the lab assistant would subtract for this and that. Then the semester exam counted a third of one’s semester grade. I worked my backside off to overcome a 33 and a couple of 66’s and thought I had died and gone to heaven when I got a B- for the year. And I have to say I neglected work in other courses that came easier to me because I had to worry about chemistry all the time.

    Maybe it’s just as well I did not become a chemist or a medical doctor, however. Getting the right answer does not seem now to be an unreasonable demand. In the outside world, the wrong answer gets you fired. I also wonder how well I might have done in college if I had joined the Army, gotten a real job, and then had gone back to college. Also in my freshman year, one of my physics classmates was John Banghart, who had flunked out his freshman year and who had spent three years in the Marines before returning to my college. I remember John walking into my physics class one day, dressed in a suit, and taking a typewritten lab report out of his briefcase and flopping it on the professor’s desk. The rest of us had handed in handwritten lab reports in spiral notebooks. “There goes the curve,” we all thought to ourselves.” John also put himself through college, launching a “sandwich man” business that had other students delivering baskets of sandwiches and doughnuts to the dorms at night. The deliveries were always on time at the same hour every evening: we salivated like Pavlov’s dogs. At any rate, John Banghart showed us all what kind of motivation one gets when one is paying the tuition one’s self and when one knows how easy college is compared to the Marine Corps.

    Many of those former pre-meds chose other majors by their junior year. This is not to say that there were any easy courses or majors to pick from. But in order to get into law or grad school, it was the first order of business to pick a major where you got A’s. Class rank was everything.

    My senior year, the college offered a pass-fail option, and I took what was considered the hardest course in the English department, American Lit, just for fun. I got a flat A. There are two possibilities: either I had not discovered early enough the really easy department in the college, or I actually learned something. Too bad I did not take Money and Banking for fun — I might have developed a marketable skill.

    The year after I left, the college did away with most of the basic requirements and allowed students to choose their own major. No, Experimental Sex/Mixology was not a permitted combined major, but they did allow students to invent touchy-feely courses. The next year, they not only admitted women but did away with all the rules for when and where women should be after 1 a.m. I visited the campus that fall, and my old room in the fraternity had been painted black, with fluorescent cave paintings on the wall. In order to accommodate a large number of habitual heavy marijuana users, the grading standards were also relaxed so that few students would flunk out and be subject to the draft. Peace, man.

    Today, the college has very high admissions standards for women and lesser standards for men. Apparently they need more men for the sports teams to balance out the Title IX women. Of course, Educational Testing Service has dumbed down the college boards, and lots of high schools are easy with A’s, so it’s hard to tell what those admissions standards mean relative to the olden days.

    That’s why it’s good to read your dissertation on grades at the U of R. You give your students every opportunity to show you what they know and how well they think.

    Oh, BTW, the valedictorian of our class was an economics major, and he works for the government.

  6. […] Blog Archive » Dirty Little Secrets […]

Leave a Reply