Will it be possible to test public school students to improve the educational system?87
A Proposal for A New Direction in Public School Student Testing
The “No Child Left Behind” idea has probably spun out its last ideological talking point. The ill fated program’s most destructive quality was that it provided way too much fertile ground for ideological “talk show” nit picking from both sides. It will die quietly among the corpses of other momentary political educational “snake oil” cure-alls dating back at least as far as the Eisenhower years.
All these programs died from the same pair of plagues. The first was that none of them could breathe itself into life without funding. Whatever funding was proposed was gradually eliminated by questionable Congressional opinions about results. The second was that none of these legislative efforts could present proof of effectivity adequate for its own political protection. There were never any credible, unassailable test results to substantiate progress which could survive the Congressional looting operations.
Now, President Obama has, predictably, propped education up as one the legs of his three legged stool. He has even dared to trot out the seemingly good idea of merit pay for good teachers, a traditional bipartisan "meat-puppet" fun factory with a history of utterly destructive results. No one in Congress from either side of the aisle can agree on a concrete approach for testing public school students or school teachers.
The truly new feature of the debate this time around comes not from the studied educators but from the headlines of the newspapers still surviving that industry’s head long slide into the irrelevance of its neo-con news manipulation. The United States, in a performance record eerily similar to our health care disaster, spends the most money for the worst out comes (36th in the industrial world in most areas) from public education. Today, we require neither a wizened old educational expert or a tax craving politician to tell us that the public schools are a mess. The Chinese, Scandinavians, Europeans and Asians are running laps around our graduates.
That is what our testing vacuum has led us into as a nation which was originally famous for its innovative public education. Facts on the ground always trump expert educational opinions if the media presents them to the American people. These current educational “facts on the ground” are all around us in a troubling, unavoidable way, yes, just like a herd of homesick wildebeests in the parlor.
So far we have managed to fill in the gaps left by the shortage of our own educated workers by siphoning off the best from Eastern Europe, India, China and other places where education is still being taken seriously. However, with the waning purchasing power of the new “Paulson” dollars, that solution may well be ending. In recent years we have focused most of our attention on exporting colonial, greedy aberrations of our export style of free market capitalism and crooked elections and the rest of the world has returned the favor by supplying the best and brightest of its chemists, doctors, mathematicians, physicists and programmers to run our country.
The dilemma with educational testing has a few issues in common with the wildebeest problem. Just as was the case with medieval martyrs who were to be torn apart by bulls attached to each arm and leg, the educational testing quandary is characterized by its own four, self-serving flesh tearing bulls. We will consider a definition of precisely who those parties might be in modern terms, then we will look at a possible -- and delightfully simple -- solution to the whole affair of sensible educational testing.
Flesh Tearing Bulls Type One: The dogmatic, ideological arsonists.
This bunch is always willing to rant about big government intervention in the educational system. The threat they harness so effectively is to suddenly arise during elections with their prepackaged inflammatory rhetoric. They are able to convert any outcome, good, bad, or indifferent, related to changes in the educational system to a creeping, liberal contamination of traditional American values, those would be values such as illiteracy. More importantly, they seem to be consistently able to do this at election time.
Consequently, these ideological arsonists, already suspiciously under educated, are able to impose their “fire up the crowd” generalities at just the most strategic moment, term and after term, repopulating the legislative bodies of the House and Senate with more talking point zombies.
Flesh Tearing Bulls Type Two: The "free market" educational profiteers.
By sustaining their influence with copious campaign contributions, this crowd is able to sell text books and tests of incomprehensibly low quality to school boards, state legislators and governors. The secret of accomplishing this travesty year after year lies in the careful marketing of the content of these books of theirs and the emphasis implied by the questions in the tests they provide.
This means that such obsolete concepts as a 6,000 year old earth, tax and spend liberals, historically justified wars of aggression and the like can be injected into public school text books on a “pay as you play” basis. The procurement offices of state and local governments purchasing these text books and tests are only too happy to get an opportunity to “reinforce societal norms” with the both the content of these shabby, confusing standardized books and the rather shadowy “right answers” which correspond to the shabby, confusing standardized tests.
Flesh Tearing Bulls Type Three: The Ensconced Educational Credentialists
This bunch seeks primarily glory, but very often find tenured job security a close second. If they are able to insinuate their consistently ineffective educational ideas into the testing system, their credentials as absolutely indispensable, educational geniuses are sustained. This may seem silly, but these Type Three Bulls actually labor constantly in the hope of complicating things so profoundly that the test questions they might approve have twisted, tormented baggage so circumnavigatory that graduate logic students would simply surrender when they read them.
The effect of this priority on typical public school students is to train their young minds to the elaborate "agenda dodging" tactics necessary to survive the test questions at the expense of what most tax payers would consider legitimate educational materials. Having materially failed at designing effective educational systems, these “parasites of complexity” have adopted a last, desperate ambition to keep all educational policy issues -- and test questions -- so complex that only they can interpret the results, an interpretation which, not surprisingly, usually suggests educational successes based on their plans.
Flesh Tearing Bulls Type Four: The Insistent Anti Intellectuals.
Although this group may also provide denizens of Type One when properly groomed by campaign strategists, or even otherwise, they are numerous -- and vociferous -- enough to qualify as an independent category. They prefer to compare all contemporary education systems with imaginary ones which have been dredged up from their memories of their own childhood. Of course, 95% of ideas introduced by them with phrases such as “In the old days ..” or “When I was young...” are entirely fabrications of equally unexamined appetites for a return to “The good old days...” In fact, these memories of theirs are not memories of their youth at all, but rather no more than poorly detailed constructs which have been placed in their minds by others (Types Two and Three).
Incredibly, these test complainers, considering their own uncertain educational status, would actually prefer that public school students not be educated too effectively. The prospect of losing an argument with one of their children is vastly more threatening to them than creating a nation where education is completely below the standards of its industrial competitors.
The Flesh Tearing Bulls have thwarted every attempt to adopt a rational testing program which might provide for merit pay for good teachers, lead to the selection of reasonably truthful textbooks or even provide some rational means to direct Federal educational subsidies beyond the current “graft and extraction schemes.” We might recall that George and Laura Bush “contributed” $10,000 to the Katrina emergency funds to restore education in that ravished city. What we might not recall is that this altruistic couple selected a gospel based history text written by the President’s brother as the exclusive “educational material” to be purchased with their money.
Somewhere, almost certainly, there was a test available to measure the success of that, ah, text book. Can we guess what sorts of answers might have been the “right answers” for public school students taking that test?
So how can we solve this testing problem? We have very confidently exhausted the more ideologically biased, the more expensive and profitable, the more credentialist innovated and the more anti-intellectual options. In various times and in various school districts we have experimented with every one of those approaches, and the result was a uniformly skeptical electorate which had reasonably concluded that none of their results measured any more than which interest had threatened or paid off the purchasing department most effectively.
What kind of test can actually measure educational progress? Certainly nothing similar to these previous cynical attempts. It will have to be a new approach, and here it is.
1. Unmanageable question selections.
The proposed test should present roughly 15,000 questions with topics scattered in an entirely random sequence. There should absolutely not be an English, history, literature, mathematics, civics or science section. Everything should be entirely mixed. A wildly uncontrollable collection of questions will end forever the lament of those who claim that testing means “teaching to the test.” No teacher can “teach” to this test.
2. No Grade Level Testing
There should be no implied educational level for any of the questions. In the body of the 15,000 questions should be plenty of choices for both first graders and seniors in high school. The public school students taking the tests will select the questions they want to take the risk of answering. The number of correct answers minus the number of incorrect ones will establish the educational status of the student. There will be no “upper” or “lower” limits on the knowledge level of the student respondent. If a senior scores at the tenth grade level or a third grader scores at the eighth grade level, so be it. The tests will have begun to generate some credible results which can actually be useful in tailoring an education for actual students, not theoretical ones.
3. Incorporate Real Critical Thinking
Probably the most lethal attack on measurement of critical thinking occurred when test results became little boxes to be filled in to answer multiple choice questions. The message of that “leap to efficiency” was that essay questions were simply too expensive to grade, and hence, not revelatory. Further, one unavoidably valid feed stock of critical thinking is the width of the spectrum of information available to the thinker. The Type Three folks have effectively controlled this area of testing for long enough. The results, with or without their confusing excuses, are abundantly -- and discouragingly -- clear.
4. Include Easy and Hard Questions
The test questions can have weighted difficulty. The first graders will be attracted to the one and two point variety, the juniors and seniors to the five and ten point variety. There won’t be enough time for a senior to select all the grade school questions and emerge with a good score. In fact, there won’t be time for any student to answer more than 10% or 15% of the questions in the difficulty -- and point -- range which will yield a good score on the test. That is the point. By selecting the questions they will attempt to answer from the vast number of questions offered, each student will effectively configure his own test. Good test results require a wide open testing process without preconceived limits.
5. Totally Random Question Selection
Let all the experts interested in participating submit their own questions. A good, rough estimate of the number of questions needed in the resource collection would be, say, 250,000. By the way, that would be 250,000 per year. Each annual test should be taken at random from a new, fresh collection of 250,000 possible questions. If the random questions selected are accidentally weighted toward a certain age group, the results can be quite rationally adjusted without impacting the objectivity of the effort. On the other hand, with 15,000 questions presented to students in the final form of the test, such a coincidental shift would be rather unlikely.
The testing regime could easily be spread over a three day period with 5,000 fresh, random questions being presented each day. If that approach was too exhausting, the tests could be issued in 1,000 question sets on fifteen school days distributed throughout the year or even at varying rates for students from different age groups. There would be no cheating, no prepping and very little arguing about the results. When such tests were administered annually on a nationwide student population, district by district comparisons might well prove chilling, indeed.
Legitimate testing results would deliver usable and effective data for directing and targeting the efforts of the entire public -- and private -- educational system. When test results are finally lifted above the contentious, yelling Bulls who have managed to neutralize every effort so far, we might see some material advances we urgently need in our national educational product. Until then, we will continue to fly blind.
We need them, and we need them soon.
For an interesting blog post on this subject, NCTE Elementary:
http://ncteelem.blogspot.com/2007/04/onerous-testing-in-public-schools.html
and for a sample of the confusing Type Three action, ( there are plenty)
http://www.fcrr.org/assessmentReadingFirstCriticalElements.htm
For example, the following article from the BBC concerning science testing in the UK:
http://news.bbc.co.uk/2/hi/uk_news/education/7966688.stm
A Proposal for A New Direction in Public School Student Testing
The “No Child Left Behind” idea has probably spun out its last ideological talking point. The ill fated program’s most destructive quality was that it provided way too much fertile ground for ideological “talk show” nit picking from both sides. It will die quietly among the corpses of other momentary political educational “snake oil” cure-alls dating back at least as far as the Eisenhower years.
All these programs died from the same pair of plagues. The first was that none of them could breathe itself into life without funding. Whatever funding was proposed was gradually eliminated by questionable Congressional opinions about results. The second was that none of these legislative efforts could present proof of effectivity adequate for its own political protection. There were never any credible, unassailable test results to substantiate progress which could survive the Congressional looting operations.
Now, President Obama has, predictably, propped education up as one the legs of his three legged stool. He has even dared to trot out the seemingly good idea of merit pay for good teachers, a traditional bipartisan "meat-puppet" fun factory with a history of utterly destructive results. No one in Congress from either side of the aisle can agree on a concrete approach for testing public school students or school teachers.
The truly new feature of the debate this time around comes not from the studied educators but from the headlines of the newspapers still surviving that industry’s head long slide into the irrelevance of its neo-con news manipulation. The United States, in a performance record eerily similar to our health care disaster, spends the most money for the worst out comes (36th in the industrial world in most areas) from public education. Today, we require neither a wizened old educational expert or a tax craving politician to tell us that the public schools are a mess. The Chinese, Scandinavians, Europeans and Asians are running laps around our graduates.
That is what our testing vacuum has led us into as a nation which was originally famous for its innovative public education. Facts on the ground always trump expert educational opinions if the media presents them to the American people. These current educational “facts on the ground” are all around us in a troubling, unavoidable way, yes, just like a herd of homesick wildebeests in the parlor.
So far we have managed to fill in the gaps left by the shortage of our own educated workers by siphoning off the best from Eastern Europe, India, China and other places where education is still being taken seriously. However, with the waning purchasing power of the new “Paulson” dollars, that solution may well be ending. In recent years we have focused most of our attention on exporting colonial, greedy aberrations of our export style of free market capitalism and crooked elections and the rest of the world has returned the favor by supplying the best and brightest of its chemists, doctors, mathematicians, physicists and programmers to run our country.
The dilemma with educational testing has a few issues in common with the wildebeest problem. Just as was the case with medieval martyrs who were to be torn apart by bulls attached to each arm and leg, the educational testing quandary is characterized by its own four, self-serving flesh tearing bulls. We will consider a definition of precisely who those parties might be in modern terms, then we will look at a possible -- and delightfully simple -- solution to the whole affair of sensible educational testing.
Flesh Tearing Bulls Type One: The dogmatic, ideological arsonists.
This bunch is always willing to rant about big government intervention in the educational system. The threat they harness so effectively is to suddenly arise during elections with their prepackaged inflammatory rhetoric. They are able to convert any outcome, good, bad, or indifferent, related to changes in the educational system to a creeping, liberal contamination of traditional American values, those would be values such as illiteracy. More importantly, they seem to be consistently able to do this at election time.
Consequently, these ideological arsonists, already suspiciously under educated, are able to impose their “fire up the crowd” generalities at just the most strategic moment, term and after term, repopulating the legislative bodies of the House and Senate with more talking point zombies.
Flesh Tearing Bulls Type Two: The "free market" educational profiteers.
By sustaining their influence with copious campaign contributions, this crowd is able to sell text books and tests of incomprehensibly low quality to school boards, state legislators and governors. The secret of accomplishing this travesty year after year lies in the careful marketing of the content of these books of theirs and the emphasis implied by the questions in the tests they provide.
This means that such obsolete concepts as a 6,000 year old earth, tax and spend liberals, historically justified wars of aggression and the like can be injected into public school text books on a “pay as you play” basis. The procurement offices of state and local governments purchasing these text books and tests are only too happy to get an opportunity to “reinforce societal norms” with the both the content of these shabby, confusing standardized books and the rather shadowy “right answers” which correspond to the shabby, confusing standardized tests.
Flesh Tearing Bulls Type Three: The Ensconced Educational Credentialists
This bunch seeks primarily glory, but very often find tenured job security a close second. If they are able to insinuate their consistently ineffective educational ideas into the testing system, their credentials as absolutely indispensable, educational geniuses are sustained. This may seem silly, but these Type Three Bulls actually labor constantly in the hope of complicating things so profoundly that the test questions they might approve have twisted, tormented baggage so circumnavigatory that graduate logic students would simply surrender when they read them.
The effect of this priority on typical public school students is to train their young minds to the elaborate "agenda dodging" tactics necessary to survive the test questions at the expense of what most tax payers would consider legitimate educational materials. Having materially failed at designing effective educational systems, these “parasites of complexity” have adopted a last, desperate ambition to keep all educational policy issues -- and test questions -- so complex that only they can interpret the results, an interpretation which, not surprisingly, usually suggests educational successes based on their plans.
Flesh Tearing Bulls Type Four: The Insistent Anti Intellectuals.
Although this group may also provide denizens of Type One when properly groomed by campaign strategists, or even otherwise, they are numerous -- and vociferous -- enough to qualify as an independent category. They prefer to compare all contemporary education systems with imaginary ones which have been dredged up from their memories of their own childhood. Of course, 95% of ideas introduced by them with phrases such as “In the old days ..” or “When I was young...” are entirely fabrications of equally unexamined appetites for a return to “The good old days...” In fact, these memories of theirs are not memories of their youth at all, but rather no more than poorly detailed constructs which have been placed in their minds by others (Types Two and Three).
Incredibly, these test complainers, considering their own uncertain educational status, would actually prefer that public school students not be educated too effectively. The prospect of losing an argument with one of their children is vastly more threatening to them than creating a nation where education is completely below the standards of its industrial competitors.
The Flesh Tearing Bulls have thwarted every attempt to adopt a rational testing program which might provide for merit pay for good teachers, lead to the selection of reasonably truthful textbooks or even provide some rational means to direct Federal educational subsidies beyond the current “graft and extraction schemes.” We might recall that George and Laura Bush “contributed” $10,000 to the Katrina emergency funds to restore education in that ravished city. What we might not recall is that this altruistic couple selected a gospel based history text written by the President’s brother as the exclusive “educational material” to be purchased with their money.
Somewhere, almost certainly, there was a test available to measure the success of that, ah, text book. Can we guess what sorts of answers might have been the “right answers” for public school students taking that test?
So how can we solve this testing problem? We have very confidently exhausted the more ideologically biased, the more expensive and profitable, the more credentialist innovated and the more anti-intellectual options. In various times and in various school districts we have experimented with every one of those approaches, and the result was a uniformly skeptical electorate which had reasonably concluded that none of their results measured any more than which interest had threatened or paid off the purchasing department most effectively.
What kind of test can actually measure educational progress? Certainly nothing similar to these previous cynical attempts. It will have to be a new approach, and here it is.
1. Unmanageable question selections.
The proposed test should present roughly 15,000 questions with topics scattered in an entirely random sequence. There should absolutely not be an English, history, literature, mathematics, civics or science section. Everything should be entirely mixed. A wildly uncontrollable collection of questions will end forever the lament of those who claim that testing means “teaching to the test.” No teacher can “teach” to this test.
2. No Grade Level Testing
There should be no implied educational level for any of the questions. In the body of the 15,000 questions should be plenty of choices for both first graders and seniors in high school. The public school students taking the tests will select the questions they want to take the risk of answering. The number of correct answers minus the number of incorrect ones will establish the educational status of the student. There will be no “upper” or “lower” limits on the knowledge level of the student respondent. If a senior scores at the tenth grade level or a third grader scores at the eighth grade level, so be it. The tests will have begun to generate some credible results which can actually be useful in tailoring an education for actual students, not theoretical ones.
3. Incorporate Real Critical Thinking
Probably the most lethal attack on measurement of critical thinking occurred when test results became little boxes to be filled in to answer multiple choice questions. The message of that “leap to efficiency” was that essay questions were simply too expensive to grade, and hence, not revelatory. Further, one unavoidably valid feed stock of critical thinking is the width of the spectrum of information available to the thinker. The Type Three folks have effectively controlled this area of testing for long enough. The results, with or without their confusing excuses, are abundantly -- and discouragingly -- clear.
4. Include Easy and Hard Questions
The test questions can have weighted difficulty. The first graders will be attracted to the one and two point variety, the juniors and seniors to the five and ten point variety. There won’t be enough time for a senior to select all the grade school questions and emerge with a good score. In fact, there won’t be time for any student to answer more than 10% or 15% of the questions in the difficulty -- and point -- range which will yield a good score on the test. That is the point. By selecting the questions they will attempt to answer from the vast number of questions offered, each student will effectively configure his own test. Good test results require a wide open testing process without preconceived limits.
5. Totally Random Question Selection
Let all the experts interested in participating submit their own questions. A good, rough estimate of the number of questions needed in the resource collection would be, say, 250,000. By the way, that would be 250,000 per year. Each annual test should be taken at random from a new, fresh collection of 250,000 possible questions. If the random questions selected are accidentally weighted toward a certain age group, the results can be quite rationally adjusted without impacting the objectivity of the effort. On the other hand, with 15,000 questions presented to students in the final form of the test, such a coincidental shift would be rather unlikely.
The testing regime could easily be spread over a three day period with 5,000 fresh, random questions being presented each day. If that approach was too exhausting, the tests could be issued in 1,000 question sets on fifteen school days distributed throughout the year or even at varying rates for students from different age groups. There would be no cheating, no prepping and very little arguing about the results. When such tests were administered annually on a nationwide student population, district by district comparisons might well prove chilling, indeed.
Legitimate testing results would deliver usable and effective data for directing and targeting the efforts of the entire public -- and private -- educational system. When test results are finally lifted above the contentious, yelling Bulls who have managed to neutralize every effort so far, we might see some material advances we urgently need in our national educational product. Until then, we will continue to fly blind.
We need them, and we need them soon.
For an interesting blog post on this subject, NCTE Elementary:
http://ncteelem.blogspot.com/2007/04/onerous-testing-in-public-schools.html
and for a sample of the confusing Type Three action, ( there are plenty)
http://www.fcrr.org/assessmentReadingFirstCriticalElements.htm
For example, the following article from the BBC concerning science testing in the UK:
http://news.bbc.co.uk/2/hi/uk_news/education/7966688.stm
I find the article a bit odd, but also thought provoking.
ReplyDeleteWhen students select questions to answer, they will gravitate to areas of strength rather than weakness. We have all known the brilliant writer who just couldn't do math, and the math genius whose spelling was laughable. There is nothing wrong with these polar opposites. It takes all kinds in this world. What is wrong is the expectation that every student will have some predetermined level of proficiency in all academic areas. This mindset develops tests that never test the upper reaches of a person's ability, but rather emphasizes areas of weakness.
Personally, I am against all of the testing. Money for instruction is instead spent for testing. Students with a primary language other than English and a primary culture other than WASP often perform poorly on state-wide tests. Students who don't do well on tests often leave school because they see the graduation test as a star beyond their reach.
Too many politicos have made friends and family rich off standardized testing programs they vote into use.
We need to look at Finland and Ireland if we want good schools. The Finns have the highest literacy rate in the EU. Teachers are autonomous monarchs in the classroom, with only a loose curriculum to follow and no testing until testing for university qualifications. Ireland, with the second highest literacy rate in the EU, functions much as Finland does. I have known Irish who left school at 14 who could write circles around most American college grads.
We need to stop tesing, eyeballing teachers, and writing curriculum no one can follow. Let teachers teach and let children learn.
Teaching is, after all, an art, not a science.