The Data has a Problem, Part 2

Behold, my beloved and buried 2014 Philadelphia Eagles.

eagles

This season was more of gut-puncher than most. We had breakouts from Jeremy Maclin and Darren Sproles; Fletcher Cox became maybe the best not-name-JJ-Watt-lineman; the utter and total collapse of our secondary; Coach Chip Kelly being the lovable-genius-weirdo we’ve been waiting for. The team had talent, the offense showed flashes, and even when losing the team was utterly memorable.

Were the 2014-15 Eagles good enough? No. But rankings don’t tell us “why” or “how” or “who”. If you took this information as a sign we need to clean house or fire the coach, you’re either on Sports Radio or you’re insane, or both. It’s simply not enough data.

Behold, my hard-won 2014 Keystone Data

kyestone2

 

Four. Four data points. Pennysylvania spent upwards of $400 million on Keystone testing, not including the work-hours of preparation, training, administration, organization, and evaluation. For four *&!%ing data points. The Pennsylvania Common Core1 has 23 reading standards (in addition to writing standards that are not assessed), that here are reduced to four:

1). Reading  Fiction

2). Analyzing Fiction

3). Reading Non-Fiction

4). Analyzing Non-fiction.

So looking at this data, I can tell that my 100% free-lunch eligible students aren’t that good at reading. Which we could learn by listening to them read or by checking their PSAT scores. We could read Johnathan Kozol’s Savage Inequalities, published in 1991, explaining in terribly painful detail how urban children were falling behind. We could ask them. I love my kids, and if they know they are loved they will be brutally honest about…everything.

Yup. That’s it. Four data points that do not tell us “why” or “how or “who”. If you took this information as a sign you need to fire all the teachers you’re either paid to say things or you’re insane, or both.

But wait, there’s less! I could try to decipher exactly why a student got a question right or wrong by looking at the questions. That would be tedious but worthwhile, especially to share with my students. Nope – the Keystone Test is something of a state secret. In fact, English teachers are discouraged from proctoring the English exam because we might “game” a system that seems primed to be gamed. The odd implication here is that our math teachers, now reduced to “active monitoring”, can’t read.

The other problem with a secret test is we don’t know if it’s a good test. We might have a “Pineapples don’t wear sleeves” moment and never know it2.  Even worse: The scoring of these tests is left to employees with minimal training or qualification. Your child’s Algebraic constructed response could be graded by someone who couldn’t successfully complete the problem.

And sure, the fact that we don’t do it right doesn’t mean we can’t do it right. But why would anyone in power do anything to change this? The Big 3 testing companies are swimming in money. The data is showing failure, which gives actors rights to do terrible things. That the assessments seem neither valid or humane is trivial compared to the other, greener incentives.

It doesn’t have to be this way.

The purpose of assessment, standardized or other, is not to announce that kids in Radnor read better than kids in Olney. Teachers, all teachers, can and do use data to inform instruction. This looks like pre-tests or retests. It’s those of us experimenting with Standards Based Grading or Kahn Academy. If the majority of the class fails a test, we do our best to re-teach, re-test, and figure it out3.

Testing should be part of the learning process, rather than the end of learning. Unless the Keystones are the testing, in which learning literally ends while the school gags on the terrible policy. “Is testing worth it?” requires we measure the value of testing. Look at the four data points culled from a secret test graded by people who are compensated less than a Starbucks barista that tells us exactly what we’ve known for decades.

What exactly is this “worth”?


  1. no, it’s not that terrible 

  2. to be completely fair, the same criticism can and should be applied to classroom tests. Local does not equal quality 

  3. this is a serious problem in math, where the standards are too many and too deep 

The Data Has a Problem, Part I

I want to apologize for being cranky this week.

I spend a lot of time trying to make my classroom welcoming. I have my contact information in four places. I bought comfortable chairs. I decorate with seemingly important quotes and song lyrics, I post student work, I use bright colors.  I shake hands and give high-fives at my doorway. I’ve developed a reputation for preaching, in a secular way of course, the gospel of being in class, on time1.

I greet a lot of students by name; I heckle and get heckled. It’s love.

My Principal greets everyone at the door. Our staff is outstanding at building lasting, meaningful relationships. On a typical day, you can show up at 3:30 and see kids helping, tutoring, or just…there. They don’t want to leave.

But this week, I tear down their work or, for certain immovables, cover the walls with opaque trash-can lining. I have to talk in hushed tones. When I greet my kids, I need to quickly get them in their seat, with a number 2 pencil. I need to have pure certainty that they have no cellular devices.

From @misskatrina

From @misskatrina

Any student who passed the test doesn’t have to come to school, and is, in fact, encouraged not to. You’re smart, your reward is escaping us.

Any student who failed a test has to come in, but can get an early dismissal. You’re a failure, but maybe this time will be different. Either way, you’re out before noon. You don’t want to stay here, right?

All the other students come in late. When we try to remind them that the most important thing in the world is their education, it’s always disconcerting that we can shut down something that important for a whole week.

For the next 6 school days we will be the model of modernity, efficiency. The message is loud and clear and gruesome.

Our civil institutions have utterly failed my students. The penal system has taken away their parents. The recreational centers are packed and underfunded. Ambulances and firetrucks come late, the stores sit behind bullet-proof glass and when they take themselves to the free clinic they have to wait all day. The Mayor is always on television when he wants to curse at them and nowhere to be found when their school closes. The only group more reviled than the Department of Human Services are the Police.

This is the baggage my kids bring into school. But we welcome them. We convince them that this is place worth being in, a place that can help them. We’re different.

But for the next 6 days, for the sole purpose of gathering data, all that will be stripped away. We’re another institution, empowered by the state and backed by its threat of violence, compelling them to do something against their will. For the data.

So yeah, I’m cranky.

Very Reasonable People are understanding. They smile and talk about how the tests are awful and really shouldn’t be and it’s wonderful2 how much I care.  But, and here’s how Reasonable they are, we need some objective ways to measure schools, students, and teachers.

 Let’s say I agree, going even further to saying I believe data can inform and improve schools. I reject the knee-jerk reaction against anything standardized. I’m a big fan of seat belts and the metric system.

I still support the Opt Out movement or anyone else yelling “shut it down”.  The Data has a problem. It’s not longer a question of whether or not the tests are worth it – based on the resources spent and misery imposed, the answer is clearly “no”. The data isn’t useful (part 2), the application is as malicious as it is capricious (part 3). The sheer misery brought unto places that are supposed to be sanctuaries is too much.

This is now a salvage operation. Can the Data Machine be saved from itself?

I’m not sure. But don’t ask me this week, you won’t like my answer.


  1. “You can’t say TGIF if you don’t have a job, and you won’t have a job without succeeding in the classroom” 

  2. kindly read that in your most condescending tone