Harvard Study Supports Teacher Testing as Partial Tool: My Doubts

Big Study Links Good Teachers to Lasting Gain – NYTimes.com.

This article of the New York Times appeared in January and I had it saved to write about and then fell down my column of drafts. However it is still very timely. This is the very study debunked by the New York Times when it released the data of 18,000 city teachers a few days ago.

Don’t take me wrong: I also believe that a good teacher is better than a mediocre one. The issue at contention here is the means of measuring. I do not believe in standardized tests. Their usefulness/cost ratio is extremely low in my opinion.

The New Jersey Department of Education has just contracted yesterday with Rutgers University to dissect a new teacher evaluation system being tried out in 10 school districts across the state.

Acting Education Commissioner Chris Cerf says the findings by a review team from the Rutgers Graduate School of Education will be used to guide implementation of the new system in the 2013-14 school year.

Unlike the proposed New Jersey study, the Harvard Study (HS) was carried out by a team of economists; not educators. It encompasses data from a period of over 20 years. It matches data from classrooms to the same students later on in life, through their income tax returns. That raises my first suspicion: The data, by its sheer size and span of time, is unverifiable. The end of the study is abundant in data compilation which has been statistically processed. Overall, I fail to find the rigor expected in a scientific paper. The number of assumptions made and liberties taken with the data throughout the study is astounding. The study is highly manipulative (not in the popular sense but in the scientific sense of molding data to fit a theory). They also revise the work of others. Although the study claims to be empirical, it is in fact highly theoretical and it reveals so much so at the very conclusion – which I copied.

Controlling for numerous factors, including students’ backgrounds, the researchers found that the value-added scores consistently identified some teachers as better than others, even if individual teachers’ value-added scores varied from year to year. Nonetheless, many other factors which occur in classroom are ignored by the researchers.

One sentence: “If you leave a low value-added teacher in your school for 10 years, rather than replacing him with an average teacher, you are hypothetically talking about $2.5 million in lost income,” said Professor Friedman, one of the coauthors.

That “hypothetically” bothers me greatly. The difference between hypothesis and hyperbola is not large. Nonetheless, I am absolutely sure that a good teacher has a positive impact on students. The issue here – I stress again – is whether any model of standard student test can be used, not in theory but in the real classroom, to measure teacher effectiveness and more importantly, whether such testing is worth the cost in both time and money.

After identifying excellent, average and poor teachers, the economists then set out to look at their students over the long term, analyzing information on earnings, college matriculation rates, the age they had children, and where they ended up living.

They found a direct proportionality between good teachers and successful adults. The entire study is below  in PDF:

http://obs.rc.fas.harvard.edu/chetty/value_added.pdf

Here is the fundamental question that the Harvard Study claims to answer:

Does Value-Added Accurately Measure Teacher Quality?
“Recent studies by Kane and Staiger (2008) and Rothstein (2010) among others have reached connecting conclusions about whether VA estimates are biased by student sorting (i.e., whether Assumption 1 in Section 2.2 holds). In this section, we revisit this debate by presenting new tests for bias in VA estimates.”

Another point that the Harvard Study claims to correct:

“This is the reason that Rothstein (2010) …contends that “…fth grade teachers, whose students have had above average
fourth grade gains, have systematically lower estimated value-added scores than teachers whose students underperformed in
the prior year.””

That was another discovery among the 18,000 reports published last Sunday in the NYT. In other words, a teacher who has excellent students in one year, will necessarily under-perform in the next.

BTW, this was page 21 of the HS.

This is a second entry:

“One important caveat to these calculations is that they assume that teacher effectiveness does not vary with classroom characteristics. Our estimates of VA only identify the component of teacher quality that is orthogonal to lagged test scores and the other characteristics that we control for to account for sorting. That is, teachers are evaluated relative to the average quality of teachers with similar students, not relative to the population. Thus, while we can predict the effects of selecting teachers among those assigned to a sub-population of similar students, we cannot predict the impacts of policies that reassign teachers to randomly selected classrooms from the population (Rubin, Stuart, and Zanutto 2004). This is a limitation in all existing value-added measures of teacher quality and could have signi…cant implications for their use if teaching quality interacts heavily with student attributes. Lockwood and McCa¤rey (2009) argue that such interactions are small relative to the overall variation in teacher VA. In addition, our estimates based on teaching staff changes suggest that VA is relatively stable as teachers switch to different grades or schools.”

“Nevertheless, further work is needed on this issue if a policymaker is considering reassigning teachers across classrooms and seeks a global ranking of their relative quality.”

“7 Conclusion
This paper has presented evidence that existing value-added measures are informative about teachers ’long-term impacts. However, two important issues must be resolved before one can determine whether VA should be used to evaluate teachers. First, using VA measures in high-stakes evaluations could induce responses such as teaching to the test or cheating, eroding the signal in VA measures. This question can be addressed by testing whether VA measures from a high stakes testing environment provide as good of a proxy for long-term impacts as they do in our data. If not, one may need to develop metrics that are more robust to such responses, as in Barlevy and Neal (2012). Districts may also be able to use data on the persistence of test score gains to identify test manipulation, as in Jacob and Levitt (2003), and thereby develop a more robust estimate of VA. Second, one must weigh the cost of errors in personnel decisions against the mean of the benefits from improving teacher value-added. We quantifi…ed mean earnings gains from selecting teachers on VA but did not quantify the costs imposed on teachers or schools from the turnover generated by such policies.”

“As we noted above, even in the low-stakes regime we study, some teachers in the upper tail of the VA distribution have test score impacts consistent with test manipulation. If such behavior becomes more prevalent when VA is actually used to evaluate teachers, the predictive content of VA as a measure of true teacher quality could be compromised.”

This ends page 50 of the HS. I suggest you go through the entire study, and form you own conclusions.

Testing in Europe does count for much more than it does in the United States. Students are sent to a variety of schools after taking a test around the end of what would be our middle school grades. Those that aren’t found to have the academic drive and ability are not sent forward to an academic high school. They are sent to trade school or guided to apprenticeships. We may be the only developed nation that strives to place every child in an academic high school. Even some of our former county vocational schools have left their original tracks and become places with medical academies and such.

Do we have a much bigger problem with our education system than the few poor teachers? Trying to educate everyone in higher level math classes holds back those that can excel. State testing costs millions of dollars and only tests two, though critical subjects, language arts and math. A lot of high-power people with vested economic interests and political connections are tied into this and reaping the millions.

Advertisements

Bill Aiming at Forceful Municipal Consolidation Is a Trojan Horse

Bill offers N.J. towns a choice: Share services or lose equivalent state aid | NJ.com.

Senator Sweeney, the main sponsor of S2, is right in that government overall is taking too much money from the people. Where he misses the bus is in the remedy he proposes: Nothing less than taking away state aid to towns – that is out money – if taxpayers refuse to consolidate services at a local level – that is an euphemism for reducing services and laying off workers while keeping all the political bureaucracy untouched.

To top this inequity of a legislative bill, the senator proposes to eliminate Civil Service Law in municipal services – that is the same as creating the most welcoming atmosphere possible for political patronage and nepotism.

Municipal consolidation should never be forced upon the public and what happens when two towns merge under the current property tax-propelled system of government is that one municipality ends up paying for services benefiting the other.

On the other hand, in a political system based on income tax alone, where towns’ budgets are provided by the state income tax and are proportional to population, town A with budget X merges with town B, having budget Y, and the following year the new town AB receives budget X+Y. The income tax based system facilitates mergers but does not impose such on any municipality. It just makes merging desirable because any savings would be extra cash that the new AB municipality would have available.

Where we must move to reduce government first is in the redundant layers of it. I have identified as such county governments, local and county boards of education, and a large proportion of independent authorities and commissions.

Civil Service should never be gutted because it is the best guarantee that: 1. Public employment will be open to all the people; 2. Examinations will tend to identify the most qualified applicants; and 3. Once politicians can not place their friends and relatives in public jobs, they will only create positions when they are really needed.

Sweeney’s’ move is nothing but a desperate attempt to save the political class while merging the municipal workers, with possible layoffs, and eliminating civil service to create more patronage positions. I believe this act is caused in part by my agenda of reforming government and Sweeney knows that 2013 is getting closer. The bill puts all the squeeze on the people and services and leaves the useless part of government intact. That is the modus operandi of both republicans and democrats: Always protecting the political hacks at the expense of teachers, cops, firemen, maintenance workers, etc.

From here to November 2013 we will see the most spectacular proposals from the two parties in power, trying to save their loot and their perks. The political establishment will be very eager to attempt to confuse the public and to preempt the structural reforms I propose .

Muslim-Watch Would Continue Under my Administration

White House contributed millons of dollars to pay for NYPD surveillance of Muslim neighborhoods | NJ.com.

The difference between my opponents and I in November 2013 is that I will be honest about it and they will be sneaky.

Unlike section 1021 of the NDAA 2012 which – may I remind all? – authorises the president of the United States to arrest any American for any length of time without charge or reason, the surveillance of muslim citizens and organizations has been a passive surveillance . Unlike NDAA section 1021, passive surveillance does not violate the constitutional rights of anyone.

The president will be trying to evade out the charge that he is aiding the effort. Governor Christie already weaseled out saying that he forgot or did not recall knowing anything – he was just a humble U.S. District Attorney for New Jersey when this happened; why should he know anything about anything? Mayor Booker is now… I really do not know what mayor Booker is doing.

But I can say what I will do if I am elected governor of New Jersey in 2013. I can say that I will be very vocal in seeking the repeal of section 1021 NDAA 2012. I can also guarantee everybody that the passive surveillance of muslims or any individuals who could constitute a recruiting pool for enemies of the state will continue until the day when Al Qaeda warriors start wearing uniforms, abide by the Geneva Conventions, or we are entirely out of the Middle East, have learned to mind our own business (I can’t wait for that day) and the so-called War on Terror is declared over.

To add a quasi personal comment here: I do not care what religion people practice. In fact, I do not practice any myself. Philosophy and science guide my life; not religion. I do not care about races either. Being of Spanish descent, I am certain to have both Roman and Moorish blood in my veins, something that is entirely fine with me.

Reports of 18,000 NYC Teachers Are Released

City Teacher Data Reports Are Released – SchoolBook.

The study involves almost 18,000 teachers in all 5 boroughs of NYC. To put in in a few words: There was no evident relationship found between teacher “quality” and student achievement. Bad teachers (according to the tests results) were teaching in both high achievement and failing schools. Similarly, good teachers, graded in a similar manner, taught everywhere as well.  The tests used a value added system where previous years data was used as a cumulative baseline to determine the progress. There was a correcting factor for race, poverty, etc.

The margin of error of the ratings is so wide that in any serious scientific research, the data would be discarded as useless and un-supporting of the theory to be proven. Some teachers were evaluated with as few as 10 students. Some teachers only taught the students the subject for a portion of the year. Only 35% of the math and 53% of English findings met confidence limits (that is what percentile means in this case). Nonetheless the teachers’ names have been publicized with tag of value now.

The data is so inundated with randomness that it is essentially useless except for the most biased researcher.

Regardless of such lack of reliability, the report has been made public after the teachers union exhausted all its legal avenues to prevent the disclosure. Despite of the warnings of the Chancellor of Schools of NYC that the results should be taken with a grain of salt, there is no doubt that this will have a devastating effect on teachers, on their morale, and on the inclination of current college students to pursue a career in education. It is also evident that the data will be used to justify further privatization of public education.

Pointing to the latter is the fact that the push to release the individual rankings began in August 2010, when New York City education officials contacted the reporters who most closely cover the city’s public schools and encouraged them to submit Freedom of Information Act requests for the teachers’ rankings.

Of the activities that I contemplated in late 2009 for my post-retirement years, one was to teach chemistry in NYC where the need for math and science teachers is always high, notably outside of Manhattan. The work would involve teaching and at the same time following a fast-track certification process through New York University. With a son living in Manhattan, spending a few days of the week in NYC was not a problem.

With the current rumblings, there is no way I would set a foot in a classroom in NYC or anywhere for that matter.

Nonetheless the NYT writes in Teacher Quality Widely Diffused, NYC Ratings Indicate – NYTimes.com  today (page 2) that:  “However, the teacher data reports tended to be highly correlated to the schools’ grades. Last year, 79 percent of high-performing math teachers worked in “A” or “B” schools, according to the Education Department. But there was no relationship between a school’s demographics and its number of high- or low-performing teachers: 26 percent of math teachers serving the poorest of students had high scores, as did 27 percent of teachers of the wealthiest.”

But how were the A’s and B’s determined? Were they determined through the same standardized tests included in the published data? In such a case it is obvious that there would be a correlation of teacher quality and grades because the A’ and B’ were the data used to evaluate the teachers in the first place. It is like if I tell you that X equals Y because Y equals X; I have not proven anything – independently of X and Y.

Or were the A’s and B’s determined through other means, say, the regular final examinations at the end of the academic years, when the exams are not standardized?

The entire enterprise of education reform has such an increasing stench of sham that I can not help but to be more and more suspicious of hidden designs and its ultimate consequences.

My Administration Vis-a-Vis Public Sector Labor Unions

Governor Christie spent his first two years in office maligning public employees. A number of persons have asked me about what my posture would be regarding the public sector and I have provided them with it. However, I reached the point when posting an article on the subject makes more sense than repeating the same formula over and over again.

I was president of a local union once. If elected governor, I will be the management: It is a drastic role-reversal.

A brief description of the Public Labor Law In New Jersey:

The Public Employment Relations Commission (PERC) is the supreme arbiter in public labor relations matters – more or less in the same fashion as the National Labor Relations Board (NLRB) is for the private sector. New Jersey Law for the public sector has the provision of No Strike/No Lockout. In contractual disputes, the path is Negotiations – Fact Finding – Mediation – Arbitration. Contrary to what governor Christie and even the press have said, the power of the arbitrators is limited by the framing of the issue. For example, if the dispute is over wages, the arbitrator can not include in his/her award anything other than money. If the arbitrator exceeds his/her scope, any of the parties – or both in a joint petition – can ask PERC to overturn the award.

Fact Finding is a survey that both sides do of prevalent conditions relevant to the dispute in similar organizations or local unions to gather supporting data prior to mediation or arbitration.

Because mediators are often more expensive than arbitrators, in many cases the mediation step is omitted.

In the solution of disciplinary and certain promotional disputes, Civil Service Law may also play a role. Civil Service Law supersedes the arbitrator’s award. I believe it is the same case with teacher’s tenure although I am not familiar with tenure rules.

In January 2014, I would ask the New Jersey Legislature to introduce a law to bring about a popular referendum consolidating all police forces of New Jersey into the Police of New Jersey – a force with central command and data base. Similarly, I would ask for bringing all school districts under State control in the form a New Jersey Board of Education and the New Jersey Department of Education. Local police and schools would not be disturbed. It would be only the command and administration, respectively, that would change.

NOTE: The Police of New Jersey would be under the Department of Law and Safety which may be renamed Department of the Interior and consolidated with the N.J. Department of Corrections.

As the consolidation takes place, the State of New Jersey would automatically adopt all the existing labor contracts. The State would become the employer negotiating with several dozen local unions, all with different wage scales. There is nothing abnormal or new here. It happens all the time and even within a same local union there are two tiers of employees at times.

I would then ask the DOI and DOE setting a dividing line which could be the median or the average – whichever is lower – of existing wages and our negotiating position every time a contract came for renewal would be determined by whether the wage scale open for negotiations is above or below that line. Our policy would be to bring the pay scales closer and Fact Finding would be our best tool in achieving that goal. FF would only help those unions below the line.

Regarding Health Benefits of public workers: I believe the current reform law calls for a return of this issue to the negotiable list of items in 2014. Under my administration, it shall remain a negotiable issue although, as with wages, we will strive to achieve uniformity throughout New Jersey.

Tax Proposal Aims to Reward Domestic Manufacturers

Tax Break for Manufacturers Offset Elsewhere – NYTimes.com.

This mirrors a portion of my tax reform plan for New Jersey: Shifting tax liability from in-state employers and investors and increasing taxes on dividend and capital gains that do not originate in the state nor are earned from in-state activity.

Missing from the Obama plan is any measure to increase aggregate demand. Without an increase in demand, employers will not have much reason to expand domestically even if the tax code favors such expansion. Only greater demand leads to greater supply and consequently, more employment. The only exception to this rule, and it is a temporary exception, is when supply has been disrupted by an extraordinary event and shortages have disrupted the economic balance. In such an event, demand is artificially high. But that is not the case here. What applies here is that aggregate demand is directly proportional to the standards of living of the majority. In other words, most people have to make more  money.

Liquidity achieved through leverage fueled an artificial expansion that has brought us to the current crisis. That is why monetary easing has not yielded the fruits that the Federal Reserve anticipated. The liquidity has to be genuine. People have to truly own the disposable cash.

But I have come to believe that both parties in power are rather unimaginative when it comes to policy or they are hindered by their principal priority which is to stay in power. Party before nation seems to be the slogan: Or party before New Jersey.

“It is time to stop rewarding businesses that ship jobs overseas, and start rewarding companies that create jobs right here in America,” Mr. Obama said in a statementannouncing the plan.

Geez, it only took 30 years. And Obama’s plan may not be bold enough.

Of course governor Christie is terminally deaf to this type of call. He is marching blindly through the already beaten path of an ineffectual tax cut in the face of a dismal fiscal picture.

There is little chance that the Obama proposal will see life this year, with a congress divided and the lurking election. However, at least it signals that the administration is focusing on what I call “Geographic Taxation” – meaning, where you put you money makes a difference at tax time.

Neither the White House nor the republicans, who quickly responded to the plan, offered many details. The republicans responded that the plan should not be a campaign instrument but a mechanism to stimulate growth and the creation of jobs. There is no shortage of people figuring out what the problems are. In fact, I find no less than 100 articles every day describing the difficulties that America faces and offering general descriptions of what must be done.

But a recovery plan that does not describe the smallest nut and bolt of its mechanism, detailing the how’s, when’s, and where’s, does not deserve the name of “plan”.

I tend to believe that the recovery of New Jersey, or America for that matter, will involve either a reform of great complexity or a very long time of stagnation, similar to Japan’s lost decade. In our case it may be much longer than a decade because we lack some elements of the Japanese economy – theirs is more exports-oriented and most of the Japanese creditors are themselves Japanese. Most of the interest that the Japanese government pays for its sovereign debt stays in Japan. That is not the U.S. case and New Jersey has this bloated government whose political class devours everything; where the Japanese have ethics, we exhibit greed and selfishness. With the two political parties hugging power and petrified at the prospect of far-reaching reforms, chances are that is it the long tern sluggishness which will become the avenue of our future.

Christie N.J. Budget Proposal: I’ll Have One of What He’s Having

Gov. Chris Christie budget speech full text | NJ.com.

This a proposed budget; not the final budget. An over-optimistic proposal is not a capital sin but the main problem is that if this become the budget and the revenue does not meet the glossy expectations, of course the governor will rather sacrifice something else  than his pet-tax-cut experiment – not for any economical reason but for purely political and propagandistic purposes. And here are two further points:

First: The question if the governor is throwing this in to influence the VP selection in the event that Romney becomes the republican candidate in 2012. That would be the most unconscionable act but nothing surprises me. And anyway, I still believe it would be a futile sacrifice of New Jersey’s interests: I think that Romney – if he wins the nomination at all – most likely will choose a more conservative VP from the South or Midwest. A Northeastern ticket may not do very well in November considering that the bulk of the republican base is not in the Northeast.

Second: Every imponderable is against the prospect of a fast recovery: High consumer leverage, possible conflict with Iran, possible U.S. intervention in Syria (Obama may turn more hawkish if he sees re-election in doubt), China’s economy slowing down, political gridlock in Washington, more problems with the EU crisis  (Greece is not out of the woods and Portugal is beginning to show signs of distress again), New Jersey’s highest unemployment in the Northeast, wage stagnation, and the list goes on – all on the negative side.

The Straits of America – Nouriel Roubini – Project Syndicate.

I frankly can’t imagine how the administration arrived at this rosy forecast. As in “When Harry Met Sally”
I’ll have one of what he’s having.

Then, for the sake of argument, we must arrive at the best scenario – that everything goes well – and thus face the question: What does this tax cut accomplish – from the economic point of view? My answer is: Nothing, and the same goes for the democratic alternative. Both proposals are political; not economical measures. They are too small, would be applied in lieu of drastic structural reforms, and as they have been proposed are spread out – because the fat State of New Jersey is unable to do any better – so the input of cash in the economy is diluted to insignificance.

But we will see what comes out as final product at the end of June. By then, the French presidential election will be over and we may have a socialist government in France which may close the chapter of euro rescue. Or all hell may have broken lose in the Persian Gulf. And even if neither happens, we should be addressing our outstanding obligations before they compound beyond reach.