Sunday, August 2, 2009

MEA contractual “firewall” blocks assessment data

Interesting article from the left coast:

LA Times: Obama chides California for not using test scores to evaluate teachers (07/25/09)

It spotlights a bizarre and well-entrenched failing of the education system: the reluctance – even resistance – towards the use of data when evaluating teachers.

This debate has been escalated by a new federal program called “Race to the Top”, which is described in The Economic Times:

WASHINGTON: President Barack Obama has announced a race for $4.35 billion in federal grants to improve academic achievement and reverse a decline in American public schools to meet increasing competition from countries like India and China.

"In an economy where knowledge is the most valuable commodity a person and a country have to offer, the best jobs will go to the best educated, whether they live in the United States, or India, or China," Obama said announcing the competition Friday.

"In a world where countries that out-educate us today will out-compete us tomorrow, the future belongs to the nation that best educates its people," he said in an address at the Department of Education. "We have talked about it for decades but we know that we have not made the progress we need to make."

Dubbed the "Race to the Top," the competition aims to ease limits on charter schools, which receive public funding but generally are exempt from some state or local rules and regulations, link teacher pay to student achievement and move toward common US academic standards.


A good discussion can be found in this education trade magazine article:

Eduweek: 'Race to Top' Guidelines Stress Use of Test Data (07/23/09)

The use of data in decision-making is a major component of this effort, and it’s going to be interesting to see how this concept plays. It’s a common sense concept, but one that is counter-culture in education, and is clearly opposed by teacher unions.

Some (many?) educators are opposed to the use of data analysis, instead believing the data comparison is not fair because each student is unique. Another common argument is that students may know the material, but simply not be able to demonstrate mastery in a structured test environment.

I don’t want to be completely dismissive of those concerns, but they fundamentally suggest that the teacher alone should the sole judge of whether the teacher has been successful, ultimately making them accountable to, well, only themselves.

The “Race to the Top” will force discussion of this topic because it includes grant money – the mothers milk in education – and it’s guidelines will likely exclude states like California and New York, which bar the use of data in evaluating teachers.

While Michigan (and most other states) don't specifically bar the use of data in evaluations, the real roadblocks are established at the local level.

In Rochester, for example, meaningful data is scarce, especially at the secondary level. Teachers don’t use common assessments, meaning that there is no common measuring tool within the district, or even with a building. The effort and achievement required to earn a “B” in one teacher’s class might’ve gotten a student an “A” or a “C” with another teacher. This hardly seems fair, and it inherently prevents effective and objective comparison.

There is a district-level effort in Rochester to move towards common assessments, but it’s reportedly meeting strong resistance. Like most things in education, it’s moving at a glacial pace. The progress is not clear, and the outcome is far from inevitable. It calls into question how serious the district really is about the use of data in decision-making.

It’s sad news, given the potential benefits.

At a minimum, it would insure that achievement in a class would be measured by common standards, and not the unique standards of an individual teacher. An “A” in a particular class would mean the same thing within the district, regardless of which high school or which teacher bestowed it.

Furthermore, common assessments would provide building leaders and district leaders with a macro view of curriculum success, and provide insight on where the district should concentrate efforts on improving instructional resources or professional development.

Consider an example where six teachers of varying experience levels, in different buildings, administer a common assessment, and most of the students tested fail to demonstrate mastery of a particular concept. Those results
would serve as an indicator that the district curriculum specialists need to better support the teachers by focusing attention on that particular concept.

But apparently those benefits are outweighed by the risk that the data might reflect poorly on certain teachers.

In that same example, consider if five out of six teachers consistently achieve comparable results, and the remaining teacher consistently performs below far below those other five. This data alone would not suggest that the one teacher is ineffective, but it would certainly serve as a red flag.

Having common assessments – and common achievement data – would inherently establish achievement norms for teachers, and is presumably one of the primary reasons it’s unacceptable to teacher unions.

Whatever the reason, this common sense use of data goes against the grain of education norms, and opposition to the use of data remains a significant plank in the MEA platform.

In Rochester, the union has a built-in “firewall” in the teacher contract designed to “protect” teachers from data. Section 8.21 of the contract states, “If teachers who teach the same course administer common assessments, there shall be no comparisons of classroom assessment results reflected on any teacher’s evaluation.” Furthermore, in a “Memo of Understanding” addendum, the concept is expanded beyond evaluations when it goes on to stipulate, “There shall be no comparison of classroom assessment results reflected on any teacher’s evaluation or provided to any parents.”

Unions contend, as summarized by AFT president Randi Weingarten, that student test-score data should be used primarily for informative and instructional purposes. In other words, we want to specifically avoid using measurable data to set expectations and hold teachers accountable.

I cannot imagine that philosophy being acceptable in any other profession.

Rather than confront this head on, some suggest a more pragmatic approach. They argue that teachers feel threatened by data, and that introduction of data into their evaluations will only strengthen opposition. They maintain that data needs to become integrated into the education psyche in a more natural, evolutionary way.

That argument may have some merit, but on the other hand it’s reasonable to wonder if this is simply too much coddling. How long are we supposed to allow for the evolution?

Teachers need to realize that this is not some sort of “attack” on their profession, but is instead meant to help instructional leaders and teachers alike identify what is working, and what is not. And yes, it should also help to weed out those teachers who are simply not effective.

The current evaluation system is far too subjective and superficial.

- Mike.

As a side note, the LA Times article cites the Data Quality Campaign, which maintains an interesting site here: http://dataqualitycampaign.org. I could not find any state ranking data, but I did read an interesting piece on the benefits of linking teacher and student data, found here.




Obama chides California for not using test scores to evaluate teachers
At stake are billions in federal stimulus funds to be allocated in 'Race to the Top' grants. Schwarzenegger says state law will be amended if necessary to comply.

By Jason Song and Jason Felch

July 25, 2009

President Obama singled out California on Friday for failing to use education data to distinguish poor teachers from good ones, a situation that his administration said must change for the state to receive competitive, federal school dollars.

Obama's comments echo recent criticisms by his Education secretary, Arne Duncan, who warned that states that bar the use of student test scores to evaluate teachers, as California does, are risking those funds. In an announcement Friday at the Education Department in Washington, Obama and Duncan said the "Race to the Top" awards will be allocated to school districts that institute reforms using data-driven analysis, among other things.

"You cannot ignore facts," Obama said. "That is why any state that makes it unlawful to link student progress to teacher evaluations will have to change its ways."

The remarks escalate a disagreement between the Obama administration and California education leaders. While a 2006 law prohibits the use of student test scores to evaluate teachers on a state level, it does not mention local districts, where state officials say pupil data can be used to judge instructors. A handful of districts currently are doing that; L.A. Unified is not.

Gov. Arnold Schwarzenegger said Friday he would push to amend state law if necessary.

"We will seek any reforms or changes to the law deemed necessary, including changes to our data system laws, to ensure California is eligible to compete" for federal funds, Schwarzenegger said in a statement.

California's top education officials sent the Obama administration a letter earlier this month saying no changes were needed to state law and that any attempt to modify it could distract from reform efforts, but the administration has not responded.

Obama's speech could also mark the beginning of a protracted fight with teachers unions, which have resisted some of the reforms advocated by the administration, including performance pay and data-driven teacher evaluation.

The state's teachers unions have already voiced their opposition to such a move. When the 2006 law was drafted, teachers unions insisted that it include an amendment saying: "Data in the system may not be used . . . for purposes of pay, promotion, sanction, or personnel evaluation of an individual teacher or group of teachers, or of any other employment related decisions related to individual teachers."

Obama and Duncan made their position clear. "This competition will not be based on politics, ideology, or the preferences of a particular interest group," Obama said. "Instead, it will be based on the simple principle: whether a state is ready to do what works."

"Race to the Top" applicants must show progress in four key areas to compete for the $4.35 billion: adopting rigorous academic standards, recruiting and retaining talented educators, turning around chronically low-performing schools, and building data systems to track student and teacher effectiveness. But Obama also pointed out that teachers should not be judged solely on student test scores.

Seven states have already lifted restrictions on public charter schools to better compete for the funds, the Associated Press reported Friday. Other states, such as Colorado and Massachusetts, are trumpeting their recent progress on issues like merit pay and higher educational standards, which they believe will give them an inside track to secure the federal dollars.

Federal officials have said that California legislators do not have to necessarily revise current law. Instead, the attorney general could certify that the state law is not a barrier to teacher accountability.

But some California education officials questioned whether it would be possible to comply with the administration's demands.

California ranks 41st among states in collecting and using data to evaluate teachers, according to a 2008 survey by the Data Quality Campaign, a nonprofit based in Austin, Texas.

"There is . . . [a] possibility nobody will apply" for the funds, said California Deputy Supt. for Public Instruction Rick Miller, who stressed that state leaders share the Obama administration's goals. "They're asking for fundamental changes in all sorts of areas, and you have to commit to all of it by October. . . . That's a heavy lift."

The draft guidelines for the federal funding released Friday are open for public comment for 30 days. States are required to submit applications by October for the first round of grants.

The money is a portion of the roughly $100-billion educational stimulus package approved by Congress. But much of that money is expected to be used by districts to make up for state budget cuts.

Jason.song@latimes.com

Kristina Sherry in the Washington bureau contributed to this story

20 comments:

Brother Ed said...

What? No one from the party faithful which typically trolls here to denounce the NEA's Acolyte-in-Chief for his apparent heresy?
Must be August.

Anonymous said...

Figures it is in Section 8. :)

Contract is up any YOU are in a position to make a difference.

Let's do it!

Bill said...

Most teachers I know aren't opposed to using data. In fact, many of us use it--and have it used to evaluate us--until it's coming out our ears.

The rub (usually) is that teachers feel threatened because teachers are left out of the process--and furthermore righty blogs tend to fuel that suspicion with repeated teacher-whipping posts.

Any sort of data collection that has its rubric construction and implementation done by persons outside the classroom (or those who spend 3 hours a month at a board meeting and occasionally blog and post links) void of teacher input is doomed to suspicion and failure.

You don't do much in your blogging to reassure teachers you understand them, Mike, or truly want any teacher buy-in concerning evaluation and data collecting. Sorry, that is fact based on repeated rhetorical analysis of your blog entries. Your political stripes are readily apparent; your contempt is readily apparent.

That is an interesting caption in your district's master agreement. Truth is, if I were a union member in your district and had board members like you posting what you post, I'd damn-well sure want some protections in there for me so that you couldn't run off half-cocked on some surface data.

I don't know that your district's language in the norm or not. It's been my experience that teachers have and do, and are subjected to, lots of data analysis.

And many of us aren't opposed--if we're treated like experts and equal partners in the process. After all, we are the ones spending all the time in the classroom and have the advanced degrees in teaching. Right?

Bill said...

Your turn, Brother Ed.

Go.

Sister Check Writer said...

C’mon Bill (your name suits you):

The issue here isn’t teacher opposition to, or familiarity with, “using data” and you know it.

The focus of the article Mr. Reno has thoughtfully posted here is the teacher unions’ disappointing pushback against data-driven teacher accountability.

This is the stuff that makes principled civilians crazy, Bill.

Why should teachers be exempt from metric analyses of their effectiveness in public service? The rest of the planet survives and thrives with such scrutiny, yet the public should accept the union’s reconstituted swill that “student test-score data should be used primarily for informative and instructional purposes"?

Even Messrs Obama & Duncan won’t sell that one…not until those job approval numbers dip a bit lower, anyway.

As for your “repeated rhetorical analysis” of Mr. Reno’s posts and your unfounded concerns that he may not “understand” local teachers, please allow me to ease your angst.

The best reassurance and show of support ANY teacher could ask for is a school board member who works tirelessly – minus the publicly-funded benefits of generous salary, bennies, longevity bonus & defined pension -for the long-term solvency and excellence of the publicly-owned system which employs us.

That school board member in these parts is Mike Reno.

We like Mike!

Bill said...

Meh, a lot of people run for school boards with axes to grind. Mike seems to be no different. Fortunately there are those of us who really do understand what goes on in a classroom getting involved to counter such nonsense.

Bill said...

i've seen data skewers in action. first hand. if there's a policy administration wants to pursue, they'll interpret the data to justify that pursuit. in relation to that, i've seen several groups have several different interpretations of the SAME data.

make the union, teachers, whomever to be the boogeymen all you want. guess what? they're not going away. mike's blog here will do nothing for that. instead, we should all be looking for ways to work together--instead of scoring cheap political points.

what's going at town hall meetings right now is the classic example of everything that is wrong with the republican party. and that mindset seems to trickle down to even school board members.

mind numbing.

then you have others who want to use the mea and teachers as a convenient stepping stone to higher political aspirations (cough, kyle olson). that's scummy any way you slice it.


...although now he's on to acorn so it looks like he's soaked the teacher thing for the political capital he could and has moved on. perhaps someday in the future he'll have a real job, one where he can shower at night and come clean and make jesus proud of him.

Brother Ed said...

Oh my…

What a nasty, little stain you've left here, Bill.

A liberal helping of hypocrisy topped off with some religious bigotry just for the hell of it.

Do you offer up these lessons in your classroom?

How sad.

Bill said...

No, I don't offer up these lessons in my classroom. Know why? I don't offer my political and religious views to students on my professional time or while wearing my teacher hat--and I don't blog about them while wearing those hats, either.

I'm on my own time here. Last I checked it wasn't me making a blog and using my title as a school board member or a teacher to espouse my views on politics and rant against entire segments of the population.

Your friends are doing that. And Mike is only one of many on the Right doing that.

If you want to get back to the conversation started, go re-read my first post and let's go for it. I've sat on our school's assessment teams, and I'd love to hear about your experiences and expertise in the classroom.

And I say that sincerely.

Sister Check Writer said...

Missing the point as usual, Bill.

Your pretentious vitriol against an exemplary school board trustee whose contributions to our schools and community you clearly know NOTHING about are an embarrassment.

Maybe you should revisit that hypocrisy & bigotry issue -- on your own time of course – rather than mucking up the space Mr. Reno has so graciously provided here for thoughtful discourse.

Brother Ed said...

Bing!

Bill said...

Really? I was just accused by "Brother Ed" (whatever that handle means) of indoctrinating my students. Again: I'm not running a pubic blog as a teacher or as a trustee and throwing out my political views for all the world to see--and trashing large segments of the school demographic in doing so.

Get over yourself already. Today, please.

Bill said...

And I'm happy to discuss the issue presented in this thread. Thing is, I doubt any of you have any real interest, as I'm a union member (ugh) and a teacher and believe in both.

But I've laid my credentials out there and am willing to productively discuss this.

Ed and any others, let's go. Let's discuss the issue.

Bill said...

Common assessments (exit exams) are common in higher ed (at least in my experience having taught at a 4-year school and now a community college). We, for example, just completed a humanities assessment that covered courses ranging from cultural anthropology to American Literature. Now: that is very challenging to do in this particular area--nonetheless faculty, staff, and administrators work together (as equals) to continually revise and improve the assessment process.

It's probably much more cut-and-dry in an area like math (and in particular, an exclusive, narrow sub-set of that broad category).

But here are some challenges that need to be critically examined by all stakeholders in such testing:

--What do the test results "prove"? For example, say six courses in the same general area do the same exit test and there are a couple of those classes whose composite score is in the B range, one in the A range, and one in the C range, and two in the D/lower range.

What you have in the text results is not an answer--but the first step in a long journey. Why? Student population of each course. Unless students were put into those courses with equal numbers of the same type of student (background, gpas, etc.) there is no conclusive way to use the surface data to prove or disprove proper instruction is going on. In other words, the class with the A average very well might have been a "super" class where the lotto balls of population populated that course with above-average students who were predispositioned to do well. Just the opposite might apply to the lower-end course.

With random population of a course and "open enrollment," the surface data cannot be used to pass judgment on the teacher or her methods without further investigation that requires earnest work and effort.

This, of course, doesn't mean we shouldn't have outcome-based work or tests. But standardized exams are pedagogically complex to pull off with any sort accuracy and integrity in the results. Lots of detailed research has pointed this out.

My main gripe is that this very worthy topic of debate has been simplified and politicized for cheap surface soundbites and the bashing of teacher unions. I fail to see how that's "putting kids first," either.

Sister Check Writer said...

Sorry Bill, but I don't see the basis for your charge that Bro Ed accused you of anything.He called you on your ugly slur against some guy's religious beliefs.

Bigotry is bigotry and just because you think it's OK to trash those who seek moral guidance through Christ's teachings, doesn't make it right...no matter how fashionable.Bigotry is unbecoming of any rational human being, much less a teacher.

Brother Ed said...

Let's get a grip here, Bill...

Mike's blog intro clearly states "The personal observations of an elected school board member who is dedicated to education reform in Michigan."

So I'm not sure what your beef is with the First Amendment guaranteed to each of us by our founding fathers and the men and women who have died to protect them, but it's YOU who needs to "get over yourself"

I'll edu-gab data-based teacher performance appraisals with you here, providing you comply with the following conditions:

1. You apologize to Mike Reno for your baseless attacks on a genuinely exemplary school board trustee, and

2. You cease using this blog for attacks on people's religious and political affiliations.

So what do say Bill?

Are you teacher enough to clean up the messes you've made here?

Bill said...

Of course there his personal opinions. Of course he's running this blog while using his trustee title in the blog. Again, Two things put together that I won't and will not do.

Nowhere did I say he doesn't have a right to say those things. I pointed out how woefully counterproductive it is.

Pretty simple.

I figured, though, that you'd dig up reasons not to engage me in a debate on this issue. You're obviously in over your head.

Good day.

Bill said...

And I've been a Christian since 1982. My wife is a conservative, born-again Christian. But she doesn't spend her time blogging and taking cheap shots at my profession and she's smart enough to know she doesn't know all she needs to know about my line of work.

Good advice for some of you to follow.

Brother Ed said...

No surprises.

But in the spirit of our common faith, let me leave you with this:

Humility is the secret of fellowship, and pride the secret of division.

Keep the faith.

Bill said...

And posting publicly on subjects that you know half-assed is foolish (I think that's in the Bible somewhere, though I'm paraphrasing ;)).

As you obviously have zero interest in having a debate about the issue in this thread, I'll leave you with some advice: take the plank out of your own eye first ;)

Cheers.