Inside Public Education: Data, Schmata

In October (see “” post), I criticized the use of weighted grades as counterproductive in producing well-rounded, well-adjusted students.  For those of you who don’t remember, weighting grades is the practice of adding an extra grade point in advanced classes, when a B is counted as an A on a student’s GPA.  In response to that post, one of my colleagues (“Anonymous”) suggested that only a fool would rely on my kind of reasoning:  Anonymous wrote me,  “In this district we have a saying when it comes to making school policy: ‘Trust in God, everybody else bring data.’ You have a lot of anecdotes and opinions but no data. The data that weighted grades are a good thing for kids is overwhelming. Anyone interested in how this issue affects what is most important—our kids—should do a little ‘real’ research.” 

Well, despite the condescending tone—I must be either incompetent or lazy not to have found reams of “overwhelming” data—and the emotional appeal—how could anyone question something that’s good for kids?—I think there’s something really important (and wrong) in this comment.  The whole “research-based” best practices movement requires much more skepticism than we are currently affording it.

Let’s get some background out of the way:  No, Anonymous didn’t offer me any specific evidence, links or studies to back the claim of the overwhelming data deluge.  I did some investigating and didn’t come up with much, certainly nothing overwhelmingly positive when it comes to weighted grades.  But I’m sure that there is some data out there which would support Anonymous’ claim, at least partially.

The problem is our world has gotten so complicated and information clogged that we really do need to assess how we use all the data inundating us, as well as question the very premises that go into creating those “facts.”  This is especially true when we are trying to determine that which is “best” for our students.  What measures will we use?  Standardized tests?  Salary in subsequent years?  Number of arrests?  Career choices?  Attitudinal surveys?  Golden Globe awards?  Blood samples? 

Hold on there—we haven’t even determined what we mean when we say “best.”  Anonymous rolled out the heavy artillery in suggesting the data supports that which is best for the kids, but again, didn’t even hint at what that “best” entails:  achievement in college, test scores, grades, happiness and/or success are all possibilities, I suppose.  On second thought, scratch those last two: What do happiness and success have to do with data?  We wouldn’t want to muddy the water with those non-data-driven concepts, regardless of how important they are to our very existence. 

So our first data snag is using factual information to support subjective abstractions.  Consider this hypothetical example:  There’s no question that we would be healthier if we closed all fast food restaurants, banned potato chips from our grocery stores and sold soda only to adults over the age of 40.  But would this data-driven purge (we are the fattest country in the world, after all) be better?  Yes, you say?  Wow, you’re more hardcore than you look! 

OK, then let’s move to the next phase of our “research-based,” best-practices health crusade.  What if we required community weigh-ins for all Americans?  And supposing your weight didn’t meet the ideals for your height, age and gender (all very scientifically determined, using acres and acres of data).  Would it then be OK for the Fitness Police to force you to wear a doughnut-looking O (for “Obese”) on your clothes and haul your cellulite-riddled carcass out every morning for a five-mile run with the other Os?  You’d probably have a problem with that scenario being considered a “best” practice, despite all the data flowing from those activities that would clearly show America’s health was improving. Your problem would have nothing to do with the data, but would come from such subjective, valuative ideas like “freedom,” “liberty,” and “rights.”  The error the data-driven camp makes is to assume we have reached consensus on the abstractions the data is supposed to support. 

With schools, those assumptions have significant implications for our products, who happen to be human beings.  What kind of person do we want our educational system to produce?  This is part of the most important—and non-factual—question for all of humanity:  What sort of person should we strive to be?  Morals, principles, goodness, sins, truth, evil, right, wrong and honesty are just a few of the myriad totally subjective issues that go into the essence of what it means to be human. 

But data can’t really help us determine how we’ve done in those areas, so we look for connections through probabilities.  If you are a church member, statistics suggest you break the law less than non-members.  We then try to make the leap from that objective piece of data to an opinion on the morality of church members.  Now, is that hypocritical, cheating, manipulative, insider-stock-trading church member a better person than the courteous, family-centered, generous, convenience-store-robbing non-church member?  I’m not sure how you would come to a factual determination of that question, unless you simply ignore all the traits we can’t objectively measure and focus only on facts:  One guy breaks the law and the other one doesn’t; therefore, the one who doesn’t is morally superior.  Thus, church members are better people than non-members.  That example shows you how data adherents can use the objective to prove the subjective, regardless how tenuous the reasoning that leads to the conclusion is.

When it comes to weighted grades, I know of no data that even attempts to link grades with subjective things.  Anonymous is probably referring to a positive correlation between weighted grades and standardized test scores, more college-prep courses taken, graduation rates, acceptance into the “best” (darn those subjective things!) colleges and grades in college.  In short, other grade-related data that offer little to help us understand how weighted grades interact with the humans who use them and what they do to some of the less measurable characteristics of those humans.  Instead we are told that if weighted grades help convince kids to take Advanced Placement courses and Advanced Placement courses are good, then weighted grades are good for kids. 

What that simplistic logic doesn’t take into account is the pressure some of those AP students endure to suffer through classes they don’t want to take.  That data won’t show how miserable some of those kids are throughout their high school careers.  That data can’t determine if the non-weighted courses those students shunned because they felt compelled to take AP courses would have changed their lives, given them a new direction, or just made them a little more rounded.  And hardly anyone questions the fundamental assumption that Advanced Placement courses are better than regular courses.  Well, there are a couple of us who do.

Some might think it would be good if we could eliminate all subjective things from the schools so we could focus solely on the factual, but we can’t.  I would argue it would be horrible if we did, but that’s a moot point because humans won’t give up their emotional, creative, independent, questioning ways just because we’d feel more comfortable if we could reduce everything to a collection of data-driven conclusions.  The danger, though, is that more and more school leaders and government edicts are pushing us toward an exclusively data-driven future.  Advanced Placement tests and the PSAEs dominate the schools more each year, crushing any consideration of those vital-yet-immeasurable subjective areas.

Nor do I advocate the elimination of all that seductive, potentially misleading data, either.  We need to look at how all the variables that go into the complexity of a school system interact; objective data certainly has a place in that analysis.  The question, of course, is how to gain perspective on data’s role—right now it seems that data’s truly on a roll and nobody wants to consider anything else.  If I can’t “prove” that weighted grades cause more problems than they solve, I shouldn’t criticize weighted grades based on my experience in how they work. 

But what do I do with my belief that weighted grades hurt the system?  How do I rationalize away my experience, which includes the following story?  Once there was a girl desperate for the weighted-A of my honors class who knew what my in-class essay test prompt would be, wrote out the essay the night before, pretended to write it during class, surreptitiously switched the pretend essay with the one already written, and turned in the prepared essay as one she’d just written.   Yes, yes, Anonymous, I know:  This is one of those anecdotes I throw around that isn’t data based, much less provable, but it’s not the only one I’ve got.  There are the dozens of honors students who have plagiarized on research projects over the years.  There’s the cheating on tests, the cynicism about learning, the tainted, grade-grubbing attitude that permeates some honors classrooms. Whoops—I’ve gotten away from anecdotes and moved even further afield into personal observations.  But, I’m not going to apologize for that; I think my experiential analysis is as valuable as Anonymous’ data.

My final complaint about data’s place in schools has to do with the standards to which we hold our students.  The most nefarious aspect of data-driven discussions is that the pieces of data have become an end unto themselves.  Instead of having a discussion on what high school students should be able to accomplish upon graduation, what “excellent” means in relation to the skills associated with English or what level of “roundedness” we should strive to instill in our students through required courses, which makes use of data for support; we now bring data into play right from the start, eliminating all differing opinions before they’ve even been expressed.  Grades and standardized test scores seem to be the only way we even attempt to measure student learning.

Nobody makes the case that a shop class would be a great thing for that honors kid to take, that he should be in situations where he isn’t the top student in the room, that he needs to feel a little unsure in at least one class he takes during high school, that he might interact with students he would otherwise never meet, or that he might gain a better understanding of all the different kinds of intelligence there are when he can’t easily excel at building a spice rack.  Before I can raise a single point, though, the data mongers roll out their facts and figures that prove only objective things like grades, college acceptance and number of parental complaints, ending all discussion right there. 

Anonymous seems to think that simply throwing down the gauntlet that there’s data out there should be enough to discredit my argument that there are real negatives to the practice of rewarding kids, already possessing more ability, with extra credit on their grade point average.  I’m not claiming that my points should silence Anonymous’ data; why is it okay for Anonymous’ alleged data to trump my analysis that isn’t based on that data?  Keep in mind that what data we consider “important” or “persuasive” is also an opinion, and how we collect and measure that data is another subjective process.  We’ll take on the dreaded standardized tests another time, but they are one more aspect of data which has spiraled out of control in our schools.

We should be on guard when supposedly factual, scientific data is used to dismiss qualified, experienced opinions.  That Anonymous rejects my analysis without responding to any of my specific points is just as faulty as my totally ignoring Anonymous’ data.  To make the discussions on these subjective areas worthwhile, we can’t rely solely on numbers and facts.  Data-driven information matters, certainly, but so does the subjective analysis that only people can generate.