The latest phony educational “miracle” comes from Philadelphia. In this case, in addition to direct evidence of egregious cheating, the scores on the tests that count were completely contradicted by other test scores at the same school: 73% of seventh-graders were “advanced” or “proficient” in reading according to the high-stakes test, but only a third were actually reading at grade level according to another test.
At some point, do you think governors are going to stop showing up to shower praise on cheating principals before having staffers do elementary reality testing? Or will the people running the testing programs institute minimal consistency checks? Even a hotline where teachers and parents can anonymously report cheating incidents - and have them investigated by someone without a personal stake in having the testing program look good - would make a difference.
Given the absence of such simple integrity measures, it’s hard to escape the conclusion that the politicians who buy in to high-states testing don’t really want to know the truth about how the sausage gets made. The nonsense at this school has been going on for years, and is only now coming to light. Why should I believe any of the other claimed good results? And that applies to claims about superior performance by charter schools; is there any reason to think that charter operators aren’t cheating, too?
And when are the test creators going to stop making cheating so easy? Nowadays, why do we still need to give every student exactly the same test? Imagine a test which has three questions for each of 14 areas: one easy, one medium and one hard. The test makers could easily create 18 questions for each area (6 easy, 6 medium, and 6 hard) rather than just the three, and a computer could select them randomly when printing tests - it would be more expensive, since the questions would have to go on the test sheets rather than separate booklets, and it would require computers to score them, but it would make this kind of cheating very very difficult to do.
What is the incentive to report cheating? By anyone? As long as the scores go up the spigot is open. Why would the governor want more rigorous verification when that costs money and so far it is rare to find cheating. Now percolate all the way down to the staff level.
Why would you expect anything different?
Of course, the charter schools are likely to be cheating too. On the other hand, maybe not. Ohio has been pushing charter schools very hard, and in areas like mine (Dayton) there’s a bunch of them. Most of them don’t do any better than the failing public schools.
The one exception seems to be the Catholic schools, or at least the one we sent our kids to. We’re not Catholic ourselves, but it was the only option available at the time to the terrible public schools. Even with the high tuition payments we had to make, the Catholic schools spent less per student than the public schools. Nevertheless, when Catholic high schools started doing graduation testing (8th grade proficiency testing, administered the first semester of their freshman year and retaken until they passed) over 95% passed all the tests the first time. A good friend who taught at the elementary school my kids went to says it isn’t because they are better teachers, or have better resources. He said it’s because the parents valued education themselves, backed up the teachers on homework, and supported good discipline. That was the only recipe he could think of that explained the student’s performance.
The point not mentioned in the article, which I read this morning, that most saddened and upset me was this: it was necessarily obvious to the kids that their teachers and principal were orchestrating and encouraging systematic cheating, and were involving the kids in it, as something their elders were fully endorsing. The moral character lesson that this instills in young adolescents is terrifying.
Even in the absense of traditional cheating, it is easy to get meaningless results showing students are proficient. At my school, certain students are given a modified (easier) test. So if we know the students probably can’t test proficient, we give them an easier test that they can pass and then we get to say they’re proficient.
@Paulo
The cheating was reported to the Philadelphia Inquirer by several teachers at the Theodore Roosevelt Middle School. The teachers are in a difficult position. They can get fired for participating in the cheating and they can also get fired by reporting the cheating. If they report the cheating and get fired they are in a much better position to retain their teaching license and to get another job.
Link to Inquirer article. http://www.philly.com/philly/news/20110501_City_school_s_fast-rising_test_scores.html
Mark, it’s pretty clear that the Gates Foundation is in this corruption up to their necks. IIRC, it’s now clear that Michelle Rhee was, um - ‘creative’ - about her previous record. When an ordinary person applies for a job, and the prospective employer finds that the applicant’s resume doesn’t match reality, that’s it, even if the applicant was hired.
In addition, if the Gates Foundation was paying big bucks for reform in DC, it’d have been an elementary matter to make sure that they were getting information (such as the erasure data found by the testing company, among other things). They should have the incentive to do this, so that they could head off disaster. They either didn’t do this, or ignored the data.
Now, either the Gates Foundation doesn’t have the brains or resources to conduct basic background checks and monitor actual progress, or they don’t want to.
I know which I’d bet on.
If you think about high-stakes testing as a method for social control, the cheating (and the blind-eye/shock-and-surprise responses) make much more sense. It’s not about proficiency, but rather about setting up a system where people can’t win honestly and then segregating them according to which of several unsatisfactory choices they make. I’m not saying that this was done with malice aforethought (even though various forms of cheating were widely predicted when NCLB’s high-stakes testing was first announced), but cheating was certainly a foreseeable outcome, and pretty much all of the incentives were designed to make it a high-payoff choice.
I’ve always believed that all this heavy focus on testing was putting the cart before the horse. Attempting to evaluate and either validate or refute policy is of course pretty important, but shouldn’t we focus on putting solid educational policies into place first that we can THEN evaluate? My current understanding of the state of education in the United States is just that we aren’t at a point yet where we have done sufficient work on the “implementing educational policies” end, and also don’t really have a good idea how and to what end we should evaluate even those that are in place. At the moment, most of this testing seems to be more a mix between reading tea leaves and constructing lobbying tools than something related to a scientific approach.
As an interesting additional data point, Waldorf schools — they generally eschew grading and testing as much as possible, varying by country — do not seem to produce less successful students than regular schools.
(Katja): “…shouldn’t we focus on putting solid educational policies into place first that we can THEN evaluate?”
How do you know what is sound without testing? Divine inspiration?
The stake at issue is the $500 billion/year K-12 tax subsidy. If receipt of this revenue stream depends on test performance then test performance is “high stakes”. If receipt of this revenue stream depends attendance then attendance statistics will be “high stakes”. The incentive that the system offers school and district administrators to defraud taxpayers rises with the budget at issue and falls with the expectation of jail or fines for fraud.
Credit by exam, available to any student, of any age, at any time of year, for all courses required for graduation, from agencies independent of schools, would drain this swamp.
Malcom: “How do you know what is sound without testing? Divine inspiration?”
I think you’re missing my point in that you’re confusing two different meanings of the word “testing”. In science, you generally need to have a theory that you then try to test with experiments that attempt to disprove it (http://en.wikipedia.org/wiki/Falsifiability). That is called, “testing a theory”. This is generally an iterative process, under which the model gets refined or rejected.
Standardized testing in schools, however, is something entirely different. It is a simplified performance metric that mashes a number of factors (including parental income, teacher quality, student IQ, classroom atmosphere, textbook quality) together and maps them to one or more numeric grades. That has a number of methodological problems to begin with, such as the difficulty of evaluating the impact of the contributing factors from such a mash-up or trying to even tell whether a test result is close to the expected value or a statistical outlier. Standardized tests are also generally lacking in depth. I remember when I took the GRE, I was not actually reading the written sections of the verbal part with the goal of understanding them; I was instead basically using efficient pattern matching techniques that allowed me to answer the questions spending as little time on each as possible.
That is not to say that standardized testing is useless, especially once you’ve got the underlying factors under control and stabilized. But right now, it seems as though it’s largely being used as the input for a simple hillclimbing algorithm (http://en.wikipedia.org/wiki/Hill_climbing) with a considerable risk that you’re getting stuck in a local maximum. That’s assuming, of course, that the metric is even applied honestly and nobody is trying to game the system.
Speaking of gaming the system, in conjunction with ideas that “[t]he stake at issue is the $500 billion/year K-12 tax subsidy”, i.e. tying performance to financial incentives, this is exactly what you get. It encourages teaching to the test rather than ensuring that students have a solid grasp of the subject matter. It encourages schools to try and get rid of their underperforming students rather than helping them. It, as Mark pointed out in his post, encourages straight-out cheating. It creates curricula that are amenable to standardized testing. It discourages classroom activities that do not have a direct impact on test results, even those that are extremely beneficial in later life.
(Katja): “…shouldn’t we focus on putting solid educational policies into place first that we can THEN evaluate?”
(Malcolm): “How do you know what is sound without testing? Divine inspiration?â€
(Katja): “…you’re confusing two different meanings of the word “testingâ€. In science, you generally need to have a theory that you then try to test with experiments that attempt to disprove it…That is called, “testing a theoryâ€. This is generally an iterative process, under which the model gets refined or rejected. Standardized testing in schools, however, is something entirely different. It is a simplified performance metric that mashes a number of factors (including parental income, teacher quality, student IQ, classroom atmosphere, textbook quality) together and maps them to one or more numeric grades.”
Your definitions are not mutually exclusive. They overlap considerably.
A measure of the elements of a set is an order relation on that set.
A test is a procedure or device for establishing a measure. Standing back to back is a measure of height. Sitting on a see-saw is a measure of weight.
A standard is a unit of measurement. A kilogram weight is a standard.
A standardized test is a test which expresses its result in terms of a standard. Standardized tests facilitate inter-group comparison. In the US, school officials shun inter-group comparisons like vampires shun sunlight. The “metric” does not mash independent student performance factors together; reality does that.
(Malcolm): “The stake at issue is the $500 billion/year K-12 tax subsidy.”
(Katja): “…gaming the system, in conjunction with…tying performance to financial incentives, this is exactly what you get. It encourages teaching to the test rather than ensuring that students have a solid grasp of the subject matter.”
The State (government, generally) cannot pay for education without a definition of “education”. The current definition is “whatever happens to kids when they’re supervised by dues-paying members of the NEA/AFT/AFSCME cartel (unless it’s bad; then we blame parents)”. Now that’s gaming the system. How ’bout this: let school officials devise the measure which they wish that legislators and parents to apply to the district schools’ performance. So long as seat-time in class has no role and homeschooling parents and independent schools are eligible for subsidy at, say, 2/3 of the district’s regular-ed per pupil subsidy if those homeschooled or parochial schooled children achieve at or above the median of the State-schooled children, I will accept whatever test you propose. You cut. I choose. Fair?
Regarding testing, no, these two concepts are not comparable. Testing in science is done with the active goal of trying to disprove a theory (while hoping to fail at that). It’s not a metric. A standardized test is a metric. But that’s starting to get off-topic.
Malcolm: “The State (government, generally) cannot pay for education without a definition of ‘education’.”
Tell that to Finland? Finland consistently outperforms other countries in terms of PISA scores; Finland has a high school dropout rate of under 1%. Yet, from http://www.oph.fi/english/education:
“The student assessment and evaluation of education and learning outcomes are encouraging and supportive by nature. The aim is to produce information that supports both schools and students to develop. National testing, school ranking lists and inspection systems do not exist.”
To define education, you need a curriculum. Curricula do not require standardized tests. Strictly speaking, they do not even require tests.
When I was a teenage girl, my family spent several years in Germany, and I got to attend a German Gymnasium during our stay there. I didn’t take a single standardized test or even a single multiple choice test there — in fact, multiple choice tests were considered to be the mark of a lazy teacher, unwilling to spend the effort on proper evaluation of a test and feedback for the student and unsuitable to properly test if the curriculum had been understood. When we moved back, I had to skip a grade (and could probably have skipped two). Now, I wouldn’t recommend adoption of the German school system — in particular, not the ridiculous three-tier aspect with built-in social stratification straight out of the 19th century — but the education you get (or at least got, back then) even at an average Gymnasium was top-notch. That was with every single teacher having tenure, too, mind you.
Waldorf schools are yet another example. In some countries, they do not even test their students once (standardized or not) until the final grade. Yet, if the students are tested, they tend to outperform their peers.
Overall, the obsession with standardized testing is largely an American and Eastern Asian thing. Plenty of other countries do just fine without it.
There is no royal way towards a good educational system. Good education requires effort, and is ideally tailored towards the needs of the individual student and the special situation of each school. That does not mean that there should not be quality control, but standardized testing is not good quality control; it’s poor quality control and lazy policy to boot; it’s part of the problem, not part of the solution.
Yes, I agree that you do need good teachers. In Finland, for example, being a teacher is a high status profession (even though it’s not a high income job). It requires a master’s degree, and the admission rate into teacher programs is a staggering 10%. Schools get to pick the best out of the best. Unfortunately, I suspect that in order to get even a fraction of that level of competitiveness in the USA, you’d have to incentivize teachers with higher income, so as not to lose them to better-paying jobs.
Malcolm: “How ’bout this: let school officials devise the measure which they wish that legislators and parents to apply to the district schools’ performance. So long as seat-time in class has no role and homeschooling parents and independent schools are eligible for subsidy at, say, 2/3 of the district’s regular-ed per pupil subsidy if those homeschooled or parochial schooled children achieve at or above the median of the State-schooled children, I will accept whatever test you propose. You cut. I choose. Fair?”
Remember how I mentioned the bad parts about the German educational system? As I said, it’s got built-in social stratification. There are traditionally three school tiers in Germany, Hauptschulen, Realschulen, and Gymnasien, with different educational standards. If you think Hauptschule = blue collar workers, Realschule = white collar workers, Gymnasium = academics/elites, that’s about right. And just as the German Gymnasiums are among the best schools worldwide, the German Hauptschulen are among the worst (a teacher at a Hauptschule, not coincidentally, also gets paid quite a bit less than a teacher at a Gymnasium). This is part why on average, Germany, just as the United States, only has middling PISA scores. Which is also why German states are currently working on reforming their educational systems to get rid of this (or at least ameloriate it). In particular, it looks as though Hauptschulen are going to be eliminated entirely in most states.
And now you are essentially suggesting to indirectly introduce a similarly tiered system in the United States. You want to throw all those kids who can’t get into the better-performing schools (which will also attract the better teachers) under the bus. No, that’s not fair. It’s an introduction of social stratification through economic incentives (worsening the stratification we already have from low parental income and living in poor school districts). And last I checked, Bismarck’s Germany did not share a great many values with the United States of America. YMMV, of course.
In the end, I think the biggest problem is that we don’t have enough quality teachers for our children to go around and a lack of competitiveness, because as far as jobs for people with college educations go, being a teacher isn’t a particularly attractive choice. We either need to hire more quality teachers, or make our existing teachers (and schools) better. Neither is going to be cheap. If you want better schools, in the end you’ll have to pay for it. “There ain’t no such thing as a free lunch.”
(Katja): “Testing in science is done with the active goal of trying to disprove a theory (while hoping to fail at that). It’s not a metric. A standardized test is a metric.”
Who was talking about “testing in science”? We were evaluating students, curricula, and schools. If someone asserts that some method A produces better lightbulbs than method B, at lower cost, we sample lightbulbs and test them, and compare per-unit cost. I don’t see a difference in principle with testing students. A test generates a measure. That is all. Often measurement (testing) preceeds theorizing. Theorists need data. The Mars lander that dug into the soil and analyzed soil composition, that is, “tested” the soil for organic material. Construction companies and governments conduct destructive tests on concrete blocks. The only theory they “prove or disprove” is “did the supplier deliver what we ordered?” which is what legislators and parents want from standardized achievement tests: “Did this curriculum (or school, or teacher) deliver?”
“Metric” just means “measure” (n).
(Katja): “Finland consistently outperforms other countries in terms of PISA scores…’National testing, school ranking lists and inspection systems do not exist.’”
Make up your mind.
(Katja): “To define education, you need a curriculum. Curricula do not require standardized tests. Strictly speaking, they do not even require tests.”
Okay. That’s your definition. Parents could just as easily put the books under the kids’ beds at night and you’d have “education” by that definition. Why do we then pay for school?
(Malcolm): “How ’bout this: let school officials devise the measure which they wish that legislators and parents to apply to the district schools’ performance. So long as seat-time in class has no role and homeschooling parents and independent schools are eligible for subsidy at, say, 2/3 of the district’s regular-ed per pupil subsidy if those homeschooled or parochial schooled children achieve at or above the median of the State-schooled children, I will accept whatever test you propose. You cut. I choose. Fair?â€
(Katja): “…the German educational system…it’s got built-in social stratification…And now you are essentially suggesting to indirectly introduce a similarly tiered system in the United States. You want to throw all those kids who can’t get into the better-performing schools (which will also attract the better teachers) under the bus. No, that’s not fair.”
Why suppose that this would be the result of parent control? “You cut, I choose” would limit the demands that the NEA/AFT/AFSCME cartel’s pet legislators could make on taxpayers, because they would be subsidizing escape options if their demands became excessive. Political control of school exacerbates inequaliy.
(Katja): “as far as jobs for people with college educations go, being a teacher isn’t a particularly attractive choice.”
Not according to this:…
(Katja): “We either need to hire more quality teachers, or make our existing teachers (and schools) better. Neither is going to be cheap. If you want better schools, in the end you’ll have to pay for it.”
It does not take 12 years at $12,000 per pupil-year to teach a normal child to read and compute. Most vocational training occurs more effectively on the job than in a classroom. State (government, generally) provision of History and Civics instruction is a threat to democracy, just as State operation of newspapers would be, and are in totalitarian countries like Cuba and North Korea.
See here.
http://mercatus.org/publication/k-12-spending-student-oecd
(Katja): “as far as jobs for people with college educations go, being a teacher isn’t a particularly attractive choice.â€
(Malcom): “Not according to this:…”
I didn’t realize that you needed a master’s degree to become a white collar worker. I’m not sure how familiar you are with teacher education: To become a teacher, you need at least a Bachelor’s degree and a teacher certification in most states. That’s usually done by either (1) getting a Bachelor’s degree in education and getting certified in your minor or (2) getting a regular subject-based Bachelor’s and following up with an MAT (Master’s in Teaching). You can also get a subject-based Bachelor’s degree and enter a specific certification program, but since that’s almost as much effort and cost as a MAT without getting you the higher degree, it’s less common. Similarly, you can start out with a Bachelor’s in education and then get certified for additional subjects. Now, your annual salary with a bachelor’s degree will not be very high, less than that of a social worker (using BLS data). With option (2), your income should be decent or good, but when you are investing the time and expense to get a Master’s degree, there are plenty of higher income options that involve the same amount of degree work. For example, you can almost double your salary by getting an MS in computer science instead of becoming a teacher. In general, teachers do earn quite a bit less than other professions with an equivalent college degree (http://www.epi.org/publications/entry/book_teaching_penalty/).
This is not a an abstract concern, either. Half of all new teachers leave their profession within the first five years of starting teaching, primarily because of poor working conditions and poor pay (http://www.washingtonpost.com/wp-dyn/content/article/2006/05/08/AR2006050801344.html).
Just to be clear, my point here is not that teachers may or may not be compensated adequately; it’s about making teaching an attractive job that attracts well-qualified individuals (especially as, unlike in other countries, it is not currently a high status profession in America). Nor is it hardly the only concern that schools face in America; another serious problem is the high level of child poverty (both relative and absolute) in the United States. But good education generally requires having good teachers to start with, so you either need to attract talented teachers or make those teachers better who aren’t so great.
(Katja): “Testing in science is done with the active goal of trying to disprove a theory (while hoping to fail at that). It’s not a metric. A standardized test is a metric.â€
(Malcom): “Who was talking about “testing in scienceâ€? We were evaluating students, curricula, and schools.”
Read back. I was talking about testing as evaluating approaches to teaching scientifically, which is a different kind of testing that you do when evaluating students.
(Katja): “To define education, you need a curriculum. Curricula do not require standardized tests. Strictly speaking, they do not even require tests.â€
(Malcolm): “Okay. That’s your definition. Parents could just as easily put the books under the kids’ beds at night and you’d have “education†by that definition. Why do we then pay for school?”
It’s not my definition. Pretty much every country in the world that I can think of defines education via a curriculum. Most do not have standardized tests. Tests, in general, are evidence of you having an education, and are not necessarily part of the education itself (though they can be, such as with Mastery Learning; see below). It was pretty common in the 19th and early 20th century for women interested in science that they had to learn the subject matter without testing (Emmy Noether and Lise Meitner are examples) and then obtain proof of their skills in a test that was separate. In these days, lab courses may not involve a test, either, but be graded based on a personally chosen project. Of course, tests are very common, which is why I prefixed my statement there with “strictly speaking”. Standardized tests, on the other hand, can easily be done without and you will still have an education. Unless, of course, you wish to claim that Eton does not provide an education to its students …
A more general example: Since you have said before that you like the concept of charter schools, let’s consider Mastery charter schools, which employ Mastery Learning (http://en.wikipedia.org/wiki/Mastery_learning). Mastery Learning involves mastery tests, which are pass/fail to indicate whether you’ve mastered a certain level of learning or not. If you failed, that will provide you feedback on where you have to improve, and you will retake the test (generally in a varied form), until you’ve mastered the subject. It’s a learning model that ensures good understanding of the subject matter (since you cannot go on to the next level until you’ve mastered the previous one), but is awkward to reconcile with a grading scheme, let alone standardized testing (which can and has been done, but it’s not a natural fit). The goal of such a test is to provide feedback and ensure full comprehension of the subject matter, it is not meant to provide a ranking.
(Katja): “…as far as jobs for people with college educations go, being a teacher isn’t a particularly attractive choice.â€
(Malcom): “Not according to this:…â€
(Katja): “…I didn’t realize that you needed a master’s degree to become a white collar worker.”
You don’t need a M.ED. to become a teacher, either. Parochial schools hire applicants with a B.A. in the subject. Advanced degrees have no relation to teacher effectiveness; they just inflate teacher salaries, union dues revenue, abd the burden on taxpayers.
(Katja): “…I’m not sure how familiar you are with teacher education:…”
I have a B.A. in Math and P.D. (fifth year certificate) in Secondary Math Education. My colleagues unanimously regarded College of Education coursework as worthless. One class (Ed. Psych. Statistics) offered something worthwhile (theory and strategy behind grading). I testified against the creation of the Hawaii Teacher Standards Board, against the subsequent repeal of the sunset provision in the enabling legislation, and against the expansion of the Board’s power to include review the licenses of teachers already in service. I’ve read quite a bit about the relation between certification and student performance (as measued by standardized tests) and done some research using NCES data on State-level NAEP scores and district-level teacher requirements.
A former Dean of the Honors Program at UH said that Education majors had the highest GPA in their major and the lowest GPA outside their major of all undergraduates.
(Katja): “…good education generally requires having good teachers to start with…”
That probably helps in many cases. The current State-monopoly structure of the industry inhibits cost-saving innovation. Homeschooled children of parents with no schooling beyond high school, enrolled in Alaska’s government-operated correspondence school outperform the students of the college-trained teachers in conventional schools.
(Katja): “…testing as evaluating approaches to teaching scientifically…is a different kind of testing that you do when evaluating students.”
I don’t agree at all. The test of a lightbulb factory must include testing its product (individual lightbulbs).
(Katja): “…Mastery Learning…”
If governments subsidized parent control and used the device you describe its measure of success, I would not care about our other differences. Whatever measure the State schools are willing to apply to themselves would work if the subsidy follows students to any school or other arrangement that parents choose which achieves “success” (so defined) equal to or better than the government schools. The expansion of employment options would improve life for teachers too.