Current appearances to the contrary notwithstanding, the health care debate will not, in fact, go on forever. One of the items on the post-health-care agenda is reauthorization of the Elementary and Secondary Education Act (ESEA), as modified by the No Child Left Behind bill passed in 2002.
One of the striking features about NCLB is the primitive evaluation mechanism it employs. It’s pure defect-finding: measuring the percentages of kids of different types who fail to achieve some standard, as measured by standardized tests. Henry Ford would recognize it. W. Edwards Deming would be appalled by it.
Statistical quality assurance depends on sampling, not census inspection; on paying attention to the entire range of outcomes, not just whether a given outcome meets or fails to meet some standard; and on process. And it is continuous and interactive rather than purely retrospective. In Deming’s world, the purpose of quality assurance is to feed back information about processes and their outcomes to operators so the processes can be changed in real time.
One of the reason Honda and Toyota ate General Motors’s lunch is that the Japanese car companies adopted statistical quality assurance while Detroit was still inspecting every part coming off the assembly line to see whether it was within tolerance. Why are we using those same outdated principles to manage the much more complicated problem of teaching children to read, write, and reckon?
Applying statistical QA to education would involve:
- Selecting a sample of students for high-quality, expensive testing rather than settling for the level of observation we can afford to do on every student.
- Using information about the whole range of performance rather than fixating on an arbitrary cutoff.
- Taking measurements all through the school year, not just at the end, and getting the results back to the teachers promptly.
There’s really no excuse for running our educational system on the management principles of 1920.
It's actually worse - the method they use measures the change in the percentage of kids who meet such a standard, and if that change isn't sufficiently big, the school is failing. I kid you not - a school going from 80% at the reading level to 84% might be "failing", while a school that went from 2% to 12% is "successful".
Careful now, next you'll be realizing that demoralizing and convincing a large share of the "input" that it is defective IS the purpose of most contemporary educational institutions, as docile people conditioned to tolerate absurd dictates of school systems are easiest to control.
Deming's most important precept was "Drive out fear!" but schools only have fear to work with — if kids weren't conditioned to accept grading and ranking from day 1 (and the idea that they deserve whatever happens to them if they don't please teacher) schools would actually have to rely on their ability to teach and their ability to equip kids with critical thinking skills, and that's not going to happen any time soon.
Actually, Deming would reject the notion of evaluating students at all, just as he did with workers — he was the source of the oft-cited statistic that 95% of all problems are rooted in management, although 99.9999% of all problems in the workplace are treated as originating in the workers (and, no coincidence, require management attention to fix).
Speaking as somebody who's actually used SPC in the automotive industry, I feel obliged to point out that any rigorous application of it to education is extremely unlikely. When we're analyzing our capability on a stamping, we're allowed, no, mandated to notice the differences between nominally identical batches of steel that belong to different heats, or came out of different foundries. Maybe we can use more than one, but we have to adjust process parameters to stay centered for incoming materials of different properties, though identical specifications. Heck, even whether the steel came out of the center or edge of the billet is relevant.
The moment you try that with education, you're going to start noticing things that political correctness says you have to pretend don't exist. The left won't permit that because they ARE politically correct, the right won't because their aversion to being called racists is greater than their desire to help minorities.
So, not going to happen. You might get some half-assed version of SPC that deliberately ignores a lot of critical variables, but the result is not going to be all that impressive compared to real SPC.
GM would never have switched if competition hadn't forced them to. Perhaps if everyone were taxed to pay for their "free" GM car (but with an option to buy an expensive BMW or Mercedes if they could afford it) they never would have had the incentive to. That's why the vast majority of studies show that serious competition from charter schools and vouchers (where existing government schools face the threat of closure) improve the existing schools as well.
There are far too many variables to control for people to ever be happy with the results of statistical sampling to compare how well different schools and methods are performing.
The evaluation mechanism is a substitute for choice, and not a good one. (Consumer choice is why GM finally learned (to the extent it can be said to have).) I don't think that the proposed improvements to it are going to be effective either, and I don't think anything would change Mark's aversion to increased choice.
Since many of the production units (i.e., teachers) have fewer than 25 units a year, wouldn't any sample of that already small universe be too small?
NPM is right. You can't use SQA as a way of firing bad teachers. But that runs into another of Deming's principles: "Don't blame the worker for the failures of management."
Measuring change is the right thing to do, if you want to evaluate teachers and institutions. But you don't measure change by asking, "How much better is this year's batch than last year's batch?" That's the wrong question because this year's batch is different from last year's batch, so the change you want to measure (knowledge/skills/abilities added by this year's education) is confounded with the difference between this year's students and last year's students.
Instead, you have to ask questions like, "What level is Johnny reading at now, and what level was he reading at last year?" That measures what Miss Jones, Jornada del Muerto Elementary School, the Zozobra School District and the New Mexico Department of Public Education have done. If Miss Jones takes a fifth grader who was reading at the second grade level last year and he's reading at the fourth grade level this year, he (and she) have accomplished something significant, even though he's still not reading at grade level and by the lights of NCLB the Jornada del Muerto Elementary School's test results are being dragged down by this student's significant achievement.
We can do this sort of assessment, but it requires careful tracking of students through the system. That sort of tracking is expensive, and can only be done for a sample of students. This is simple sampling statistics — in terms of tracking the process, the sample sizes required are not inordinately large. Another advantage of this approach is that the results couldn't be used to fire or assess teachers, because the sample sizes are too small. This would take away the perverse incentive teachers have under the current system to teach the test rather than the principles.
Mark, what you're arguing here is intellectually unexceptionable but politically naive in the extreme. The vast majority of the opposition to the current testing regime comes not from rigorous measurement advocates like yourself, but rather from opponents of the whole idea of academically effective schools. Teachers' unions are the popular symbol of this opposition among conservatives, but they're backed up by very significant support from a large segment of the voting public who oppose academic measurement of any kind because they oppose high academic standards in general, on political or philosophical grounds.
I know-it sounds strange that anybody would actually oppose improving academic achievement. But as Diane Ravitch points out, many people-including plenty of "education experts"-view the primary purpose of schools to be something other than the acquisition of knowledge and intellectual skills. For some the goal is character development, for others the inculcation of values, and for still others the fulfillment of an idealized vision of childhood experience. For all of them, though, academic standards of any kind are distractions that interfere with what they see as schools' main function.
Given this hostile political climate, the only kind of standards that can survive politically are those that focus on addressing the system's most egregious failures. Even today, numerous states are busy dumbing down the already minimal standards imposed by NCLB, at the behest of its opponents-the vast majority of whom would howl even more loudly against the sophisticated QA methods you prefer than they already do against NCLB.
I take it for granted that your opposition to NCLB is based on sincere concern about its weaknesses as a tool for improving overall academic achievement. But in practice, your arguments will be used most vehemently, and most effectively, by people whose goals are radically different from yours.
Mark,
There is an educational equivalent of the HOPE program. Where they applied both behavioral analysis and an engineering quality feedback system to the educational process. The method is called Direct Instruction, and it was tested in Project Follow through a large longitudinal study of several educational methods. Basically they solved the problem of the educational gap, taking Title 1 schools with their existing personal, and without hire/fire ability and brought from the 20th to the 50th percentile. Unfortunately like the HOPE program, it has not spread as widely as its empirical success would seem to demand.
"Teaching Needy Kids in Our Backward System " Siegfried Engelmann
The data chapter is avaiable free:
http://www.zigsite.com/PDFs/chapter5-6intro.pdf
The QA system is best described here
"War Against Schools: Academic Child Abuse" Siegfried Engelmann
Where to start? When you're talking about public education in our country, you're talking apples, oranges and probably bananas. Our well-funded suburban schools are as good as any in the world, but yes, achievement levels in our inner-city schools stink. Do something to reduce the inner-city's grinding poverty and you'll see some improvement in the schools, otherwise you're blaming schools for circumstantances they can't control or effect. I freely admit I don't know anything about rural school districts, but I imagine they are another whole animal (thus, to mix metaphors, my remark about bananas).
The problems Mark lists with NCLB are features, not bugs (boy, am I full of cliches today). The real purpose of NCLB is to weaken, as much as possible, public schooling. Ever notice how wildly profitable all those new private colleges are? Now imagine opening up k-12 to that sort of thing, there's a lot of money to be made (try googling the White Hat schools in Cleveland for an example). Plus, you get to weaken the teachers' unions. What's not to like?
NCLB stipulates that every year more and more students will be proficient until 2014 when every child will be proficient in reading and math. In my state, schools are allowed to excuse less than 2% of students from this requirement; considering that special ed students typically account for about 10% of a school's enrollment, well, you do the math. It's impossible for a school not to eventually fail. Even a very good school, with well-fed, healthy and happy children.
Finally, in resonse to Dan Simon, the purpose of public schools is to produce citizens; academic achievement is certainly part of that, because to be a good citizen requires that one be able to critically evaluate all sorts of information. And I'm not sure you should be quoting Diane Ravitch, she absolutely HATES NCLB.
"The moment you try that with education, you’re going to start noticing things that political correctness says you have to pretend don’t exist. The left won’t permit that because they ARE politically correct, the right won’t because their aversion to being called racists is greater than their desire to help minorities."
I think this is actually a crucial insight into the state of left/right debate on social welfare issues, although there is much more complexity to the reality of where each is.
I apologize for the length of what follows, but I had been meaning to write a blog post on this specific topic and I kind of got carried away…
In a nutshell, current conservatism believes that all men in America choose their success. Unfortunately, because it supported the racist status quo during the civil rights era, their current position on poverty - blaming the poor (which skews terribly for minorities) seems a simple continuation of that bigotry.
Current liberalism, on contrast, believes men are products of their society - and America continues to fail its poor. As it opposed the racist status quo which actively oppressed people through discrimination, it sought not only to change public opinion, but to develop social theory that explained social injustice. Yet in an ironic twist, in the following decades as public opinion began to accept minority ethnicity (often at the behest of social research pointing out historical discrimination), social research gradually built up the theory of social capital. This basically says that, barring active discrimination, success is entirely dependent on factors such as parent education, wealth, family cohesion, neighborhood, peers, etc. - all of which act as determining resources. Powerfully predictive, this bore out in demographic data again and again.
Unfortunately, this sounded on awful lot like "blaming the poor". Many liberals found this concept unfair, and contrary to their narrative. But I think many more have come to embrace it as a well-reasoned explanation for poverty, and what is more an enticing insight into what needs to be done to minimize it.
Hence, we come to education. Basically, schools are where we see the results of inequities in social capital. NCLB had a powerful effect in galvanizing public attention to the realities of social inequity. By simply pointing at "failing" schools, a powerful argument was made for the impact of social capital. To anyone following social research, this was old news. But the public was now captivated. All we had to do was "fix" the schools, and inequality would be gone! Everyone would go to college!
Alas, this is a fundamental misunderstanding of the complexity of the theory of social capital. Schools do not operate in isolation. Certainly some may have problems, but nothing compared to the challenges they face with respect to the populations they serve. "Garbage in, garbage" out would be a terribly callous, yet largely correct summation.
But current liberalism, although loathe (as often times removed from witnessing the brutality of poverty as it is) to admit it, largely embraces this narrative. They want better schools, but they understand that schools alone cannot solve everything. Conservatives, on the other hand, have widely divergent views on the matter - although largely agreeing that if anything, less money is need for schools. The wonks would say it's the teacher unions and basic failure of productivity. The rank and file would agree, but add lousy parents who don't deserve help, and disdain the whole enterprise of wanting to "fix" things in the first place. To them, "failing schools" serves as a handy bogeyman for blaming social problems on the government in general, and lazy poor people in general.
The public now sees how powerful schools are in shaping individual success. NCLB promised that no child would be left behind. A most noble goal. But its error was in focusing on only a small portion of the components of social capital. As long as we expect schools to make up for the complexity of social inequity and dysfunction, we doom ourselves to failure - not to mention demonizing the very teachers who sacrifice so much for the common good. But with the success of programs that look beyond the school walls, such as the Harlem Children's Zone, we will hopefully make the next transition: putting resources into research-based, targeted community interventions that work. Only when all children truly have the support and social capital they need to succeed will we be able to claim America as being truly free.
We cannot jump immediately to the type of metrics you'd like because we had virtually no real data about the performance of K-12 education before the standards and accountability movement started.
All you need to know about teacher's unions and educational attainment is that-with the notable exception of Shanker at AFT-the teacher's union lobby has systematically impeded any attempt to measure any outcomes in any systematic and methodologically rigorous manner. At every point they try to water down standards, prevent the development of data systems that can track performance of individual students over time, and institute nonsense like "portfolio" based measurement.
Yes, George. That's why when I think of the states with the highest levels of achievement, I think of the southern states that don't have teachers' unions.
Rereading my first comment, I left out an important connecting point: after a school is rated "failing" because it didn't reach its NCLB benchmark of complete proficiency, one of the possible repercussions is that the school gets privatized. If you don't think this isn't a real strategy (on a par with the strategy of bankrupting the federal government so the New Deal can be euthanized — even partial effectiveness is a seen as a win) here's an excerpt from an AEI paper on 'a roadmap on chartering's way forward':
http://www.schoolsmatter.info/2009/11/from-vault_…
I think Toyota took GM's lunch money is because they turned away from batch production.
I'm confused - why did Toyota and Honda use statistical QC?
Oh wait, that's right - Toyota and Honda adopted statistical QC they wanted to make better cars, because they had to compete for customers. They had to go through the trouble of efficiently testing all stages of production, eliminating shoddy work practices, cutting ties with underperforming subcontractors, while adopting processes and buying from suppliers with proven track records, to build better cars.
Statistical quality control gave them the diagnostic tools to improve quality, and the metric with which to measure that improvement. The actual increase in quality required hard decisions about manufacturing processes, employees, and suppliers. Statistical QC was a means to an end, the end being the identification and elimination of sources of failure.
Let's see…is the public school system keen on identifying points of failure? Is the public school system in competition with other entities for the tuition of students? Are teachers willing to subject themselves to comprehensive, randomized scrutiny? Are school systems willing to overhaul their curriculum if those are found to be more effective? No. Teacher's unions are extraordinarily resistant even to the idea of merit-based bonuses, never mind merit-based pay.
The solution is simple - vouchers. Once parents can take their tuition money elsewhere, public school systems will have a sudden and urgent interest in academic improvement. They will voluntarily adopt effective diagnostic techniques. Perhaps even statistical QC.
@ Brett Bellmore:
Well said. I'm in the semiconductor industry, we also have some pretty serious SQC. There's a famous story about an Matsushita plant, where they suddenly had massive fluctuations in yield, day-to-day. They were baffled - until they found out that their wafer yields perfectly matched the attendance of one worker. But that worker had been with the plant for years - it couldn't possibly be her fault. They asked her if she was had started doing something different - she had started wearing a new perfume, which was reacting with their chemically-amplified resists. That was in the 80s, Matsushita doesn't make high-end semiconductors anymore. The SQC we have now tracks individual wafers, and measures line-machine performance by hour, to variations below 0.1%~0.01%.
Imagine that level of scrutiny in a public school system. Imagine the fine-grained statistical analysis you could do on the teachers, the students, the textbooks, the class-size, the teaching methods, and even the effects of weather, the height and size of desks, the color of the classroom walls.
It would mean instant death for political correctness. Therefore, it will never be allowed to happen.
Altoids, what do you mean by political correctness? …assuming an alternate reality did exist where you'd be able to find such beautifully homogeneous conditions in public schools…
PC is such a loaded term. You could be referring to anything from union-intransigence to phrenology.
altoids, to respond to just a couple of your points (don't have much time today to go into all of them): First, at least in Ohio, schools do not set the curriculum, nor do the teachers. The state does. I imagine this is true for almost all, if not all, states. Otherwise you couldn't have state-wide standardized testing.
Two short examples: our old family friend who has taught public preschool for many years complains that she is now required to "introduce" her students to the word "equal," something she says she would never choose to do, and for good reason. Preschoolers are no more ready for this concept than 3 months old are to stand and walk (the thumping you hear is Piaget turning over in his grave). Our next door neighbor, the beloved veteran 4th grade teacher, remarks that the curriculum is a mile wide and an inch deep, but soldiers on. He can't stop and teach anything in any depth, even something that excites his class (to grab that "teachable moment") because something else that is left out as a result could very well be "on the test."
Second, you should know that all sorts of aspects of education HAVE been studied in depth. To advance as a teacher, you need an advanced degree, and to get an advanced degree, you have to do research. You mention class size, for example. There's lots of research on that, and on school size, too. School districts that have the resources (i.e., well-funded suburban districts) are able to utlize the research findings, school districts that do not have the resources are not.
What you don't seemt to understand is that children are not widgets. They learn in different ways, at different rates. Different things challenge them and motivate them. There is no one single right way to teach, no single method that trumps all others.
@Eli:
Trying to identify if I am friend-or-foe, eh? I am referring neither to union-intransigence nor phrenology. (Phrenology? Where'd that come from?)
The definition of political correctness is like the definition of pornography - we instantly know it when we see it. But I'll try to define it anyways. Political correctness is the forced substitution of what is true for what we (the educated, cultural elites) would like to believe is true. The distinguishing characteristic of PC-truths, as opposed to objective truths, is that challenges to PC-truths are vigorously and vehemently contested, and the challenger is attacked as well.
For example, the law of gravity is not politically correct, it is simply correct. Its truth is undisputed and obvious. No one gets angry if you deny the existence of gravity, they just ignore you as crazy. Let's move into more dangerous waters. Is recycling paper good for the environment? No. The costs of collection alone are staggering - the amount of fuel and labor wasted far exceed the value of the paper collected, never mind the costs of washing and re-pulping. But we pretend that we're being noble by throwing our waste paper into little blue bins. Yet to deny the value of recycling paper makes you a somewhat evil person.
Let's move further into PC territory. What would you call a devout army psychiatrist, who was distraught over the prospect of being sent to fight fellow Muslims, who wished for martyrdom, who spent that morning in prayer robes passing out Korans, and than began shooting other soldiers while shouting "Allahu Akbar"? A "troubled soldier"? No. He is an Islamic jihadist.
That is political correctness. The forced substitution of what is true, for what we would like to believe is true, that this was just a troubled man, he was acting alone with no greater political motivations, it could have been anyone, move along, nothing to see here… PC-truth is not just a symptom of the Left, although the most head-slappingly obvious examples seem to come from there. The Right has a few sacred cows of their own - for example, evangelical Christians have higher rates of divorce and illegitimacy than secular urban elites.
Now let's move to the topic at hand. Public schools and statistical quality control. SQC will remorselessly find exactly who is to blame for what. We have a small taste of this in the original Dubner and Levitt book Freakonomics. Reading to your children, preventing them from watching TV, taking them to museums, these things are all irrelevant. Far more relevant is whether the parents were educated, had a high-income, were involved in the PTA, and spoke English. In other words, who the parents are and how they behave is far more important that what the parents actually do with their children.
Sacred cows will be skewered, and what we want to believe is true (all kids are equally gifted, they just need the right nurturing environment to flourish, and if they fail, it's not their fault, it's because the system has failed them, and the solution is to try harder) will be challenged by the SQC methods. Just as Toyota and Honda had to overhaul assembly lines and rebuild supply chains, we will have to reconstruct an entirely new model for how education actually works.
I apologize for the length of this comment.
A related aside:
There seems to be some confusion about what quality control is and how it works. Analytical quality-control methods are not made invalid by heterogeneity. Analytical quality-control is designed to compensate for heterogeneity - that's the whole point! Let's say I'm running the QC program at a large widget manufacturing firm. Everyday, there are differences - the machines are wearing-down, the work-force is turning-over, the raw materials coming in have varying quality. And let's say factory X has a particularly bad day…let's say over 1 in 1000 widgets made there are defective. What is the problem? With SQC, I can run regressions to find out what the effects are - what was the yield when the humidity was the same as today? Did we have similar problems with the same batch of raw material in another plant? Did all the defective widgets come mostly from one machine?
With SQC, we can find that out - we don't just throw-up our hands and say "it all depends, we don't know, everything is different." We are explicitly trying to identify what works, not because pointing fingers is fun, but because we want to make a good product.
Kids are not widgets, but if we are genuinely trying to educate them the best we can, shouldn't we try to figure out the differences? If kids are motivated differently, let's create diagnostic processes to find out what motivates them. Then let's assess the effectiveness of teaching methods for groups of kids with different motivations. With millions of kids in the school system, should we be able to discover something useful by SQC?
As I said before, I'm not holding my breath for anything to change.
"[O]ur old family friend who has taught public preschool for many years complains that she is now required to “introduce” her students to the word “equal,” something she says she would never choose to do, and for good reason. Preschoolers are no more ready for this concept than 3 months old are to stand and walk (the thumping you hear is Piaget turning over in his grave)."
Yes, heaven forbid we introduce concepts to children that they may not yet be able to understand. The poor dears might, uh, not understand it. (Then again, some of them might.) Forget school violence-who will defend our schoolchildren from advanced concepts?
As I said, amazingly many people-and certainly the vast majority of NCLB opponents-view maximizing children's academic progress as, at best, a low priority, and perhaps even as a terrifying threat. Opposing NCLB because its standards are too weak and crude simply plays right into their hands.
I came here via Joanne Jacobs and have decided to post the same comment again.
I think there should be a distinction made here between different purposes of tests. High-quality testing on a sample might be good within a school, and taking measurements all through the school year, not just at the end, and getting results back to teachers promptly strikes me as likely to be essential for top quality teaching. But that’s something that can be done, and sometimes is done, within a school.
An advantage of testing everyone at a school is that it reduces the chances of an unscrupulous school system manipulating the test sample.
Another one is that for many purposes we want to know if a particular kid can do something (read a newspaper, meet the pre-requisites for engineering school). In that case knowing the range that a sample of the student’s classmates perform at is not sufficient, as it’s the individual we care about. I know that many of the students I was at high school with would have dropped out of engineering school, but I completed it (just, but still). A sample could not have told the engineering faculty staff that it was worth giving me a shot.
Furthermore, Kleinman doesn’t mention the moral problems with looking at the whole range of performance rather than setting what he describes as an “arbitrary cut-off”. If you look at the whole range of performance, then how many poorly-performing students are you willing to disregard to get more performance at the top of the range? There’s no non-arbitrary answer to that one.
Also OhioMom, as an aside: there's something you miss out in your comparison of suburban schools with inner-city schools. American suburban schools don't appear to be doing any better at teaching poor students than inner-city schools do. See http://d-edreckoning.blogspot.com/2008/03/theory-… for some analysis of American students from low socio-economic groups attending schools in high SES areas. This implies that surburban high schools' high results is on average only from the quality of the student intake, not because the schools provide a fundamentally better education.
American suburban high schools may be as good as any in the world, you didn't provide any supporting evidence for this claim, but if so then American inner-city schools are also as good as any in the world. (And it's quite possible that the rest of the world has many schools that also fall far short of what can be achieved). You also don't mention the massive amounts of money to be made from providing services to public schools. How many school district supervisors make masses amount of money despite no knowledge? See Ohio Mom - there's something you miss out in your comparison of suburban schools with inner-city schools. American suburban schools don't appear to be doing any better at teaching poor students than inner-city schools do. See http://kitchentablemath.blogspot.com/search?q=sal…
Altoids, the phrenology reference was basically an attempt to tie the notion that poor minority students underperform whites due to genetics to phrenelogy. Which is always at the margins, and defenders of the Bell Curve hypothesis and other racialisms consistently cry PC.
And I'm not sure I follow the logic of your post. Levitt & Dubner's conclusion in Freakonomics was as incongruent: It doesn't matter what parents do with their kids, it matters how they "behave". What does that mean? Well, what their superficial analysis of child development neglected to explore was that, while reading to your kids, taking them to museums, etc. are all things that high SES parents do, they are not sufficient. In the same way, being college educated, involved in the PTA, and speaking English don't automatically produce successful kids. What really matters, is the quality of language spoken to your children, how much cognitively stimulating social activities they are engaged in, and the extent to which the environment is free of toxins (specifically lead, which is found in much higher levels in poor communities and is linked to brain damage). Gladwell does a much better job with this in Outliers, where he discusses Hart & Risley's groundbreaking study on SES and language development from ages 2-4, and Lareau's theory of high SES "Concerted Cultivation" vs. low SES "Natural Growth" models.
I'm still unclear as to what you meant by political correctness. On the one hand you say that it doesn't matter what we do with kids, but who their parents are - but then you suggest vouchers will force schools to do a better job, well, "doing things with kids".
The whole line of reasoning seems very messy. The real political correctness it seems to be is that most people seem not to understand that public education is an entirely socialist enterprise - in that it is state enforcement of social equality. The middle classes seemed fine as long as this project never quite grappled with seriously bringing kids out of poverty. Yet now that NCLB is doing exactly this, middle class parents are fleeing the new emphasis on remediation and lowest-common-denominator curriculum. Yet given the limited budget, schools have been forced to sacrifice high SES, grade-level oriented education because it, as the law declares, leaves children - namely lower-SES - behind.
I've taught Kindergarten in low-SES, immigrant neighborhoods. I've seen where these kids come from. I've also taught high school and see where the kids end up - pregnant at 16, dropping out, in jail, or smack into low-skill, low-wage, dead-end careers. Their babies come right back when its time to enroll in Kindergarten and the cycle repeats. This is not news to any of us actually doing the teaching. This is not news to any of us who know what needs to be done. "Political correctness" - whatever you mean - has nothing to do with it.
@ Eli:
This is not a structured reply, more of a point-by-point response to your last comment. Everything I wish to say in a narrative form has been said previously.
I see. You're right, phrenology is a interesting parallel, because on one basic level, phrenology is obviously true. A cricket is less intelligent than frog, which is less intelligent than a cat, which is is less intelligent than a chimpanzee…you get the idea. Brain size is clearly correlated to intelligence. But within the human species, there isn't much evidence for large brain size being better. Women have smaller brains, with no apparent detriment. Similarly, genetics play an obvious role, as the heritability of intelligence is undisputed. But any individual's intelligence is poorly estimated by his or her genetic inheritance. Just because statistical trends can be drawn from populations doesn't form a basis for discrimination against individuals.
I invoke Levitt & Dubner because they are a small example of what statistical analysis can do. You accept that reading to your kids and taking them to museums are irrelevant, but then draw a parallel with being in the PTA and being college-educated, that neither set of activities are sufficient or would automatically create successful kids. Yet they show that there is a great difference between those sets of activities. With SQC, you can expect the discovery of more distinctions.
I dislike Gladwell. I simply can't take him seriously, he is not an honest broker of the facts in his books, and he certainly does not have the mathematical training to understand the studies he quotes. If you haven't heard of his "igon value" blunder, look it up. Eigenvalues, as the rest of the world calls them, are absolutely fundamental to statistical analysis. They are taught in any freshman linear algebra class. This is like a priest misspelling Jesus.
You seem to disagree with my definition of political correctness. Do you find the term valid? How would you define it?
I don't suggest that all interactions with kids are useless. Clearly, teachers are more effective than feral wolves at teaching children. I only suggest that vouchers would give substantial incentive to adopt effective diagnostic techniques, and any effective diagnostic technique will inevitably expose some sacred cows.
I am a semiconductor process engineer, not a K-12 educator, so forgive my amateur observations. This is just one anecdote, but SES cannot be a total explanation. I was born to immigrant parents who are just now speaking passable English. When I walked into kindergarten, I did not speak English. Yet here I am, demonstrating a high level of English proficiency, SES from ages 2-4 notwithstanding. And I am not a special case. First- and second-generation immigrants constitute the bulk of graduate degrees in engineering and science disciplines.
This is where statistical analysis methods can really help. SES is not the whole story - what are the other factors? What are their relative levels of importance? The goal is to dissect how education works, pinpoint relevant factors, and identify points-of-failure. I don't believe our current education system is willing to be subjected to that level of scrutiny. It's a question of political correctness because it is an unwillingness to confront hard truths.
Altoids,
I'll give you that on Gladwell - although I stand by his application of the authors I mentioned.
My hunch is that were talking past one another, yet likely in general agreement - if not on solutions (vouchers is a whole conversation unto itself - I'm actually quite sympathetic to a parent not being forced to send their child to their ghetto school… but, well, it's complicated).
I still don't see where the political incorrectness lies - what is this hard truth? For decades social scientists have been gathering data on exactly what you're asking: what are the factors that go into success? The far left (still stuck in the 60's) hates to acknowledge the dysfunction in poor families. Yet the far right would have us throw them out as so much trash. To my mind the only real sacred cow is the notion that any of us is in any way a self-made man - that is, one not entirely determined by his genetic and environmental endowments.
I gave you a couple of factors that you still left out of your anecdotal analysis! Granted, SES is blunt, but it points toward obvious trends; it is highly predictive of risk factors associated with poor performance. Hart and Risley pioneered the use of language coding to measure vocabulary development among SES groups and found distinct patterns of developmental distribution. Whether a child speaks English has much less to do with academic performance than their language/cognitive development. English Language Learners (ELLs) get mandatory periodic language testing upon enrollment. Yet many of these students perform much better than their native language peers!
I have a 5 (almost) year old at home right now who is reading. Granted, she's pretty bright - but both her parents have graduate degrees and we've been giving her intensely stimulating communication since before she was 2. We ask her to describe, compare, question, etc. We provide her with opportunities to grow her brain. Statistically she's right in line with our demographic. I also have a 2 year old who is about at the level most of my kindergarteners were. Also in line with demographic data. I guarantee you that the 17 year old mother of two I had in my high school class was not exposing her son to a remotely comparable level of stimulus. And no one in their right mind should expect her to be able to.
And yet when they come into our classrooms we are judged against the same rate of progress as middle class schools - or worse, tasked with bringing them up 1-8 grade levels… while most of our parents have no discipline at home, many are on drugs, single, dads in jail, etc. Meanwhile the endless cycle of remediation is slowly driving the poor kids crazy. White flight has already siphoned so many to the suburbs… and with vouchers removing motivated parents only the most wretched will be left to fill the ever-increasing class sizes and ever-decreasing community support.
I better stop before this turns into an even worse rant. I'll just say that I am frustrated to no end by the degree to which the complexity of education is misunderstood and teachers are unfairly blamed when so many of those who pretend to have easy answers could never themselves handle the pressure of being an American teacher at an at-risk school. Sure, many of us are terrible, and many of us are heroic. But the devil is in the details and education is officially the savior of every modern democratic state, in terms of guaranteeing all citizens an equal chance at fulfilling their dreams.
…sorry, should have read, "17 year old mother of a 2 year old". (4th paragraph)
@ Eli:
Just as an aside, thanks for the civil tone you've maintained throughout. Very rare, and very appreciated.
To avoid talking past each other, I'll try to be as direct as possible.
The hard truth is that genetics, culture, "SES", early exposure to vocabulary development, and all these other things play a role, but we don't know what the relative importance of each are, and we are scared to find out. For good reason. Is our society prepared to accept the possibility that perhaps genetics or culture play a dominant role in lifetime success? It would strike at the core of the American Dream.
As long as social studies tip-toe at the edge of what I call political correctness, we can maintain the following illusion - all kids are equally gifted, they just need the right nurturing environment to flourish, and if they fail, it’s not their fault, it’s because the system and society has failed them, and the solution is to try harder. As long as we keep things comfortably fuzzy, we can attribute success or failure to things like SES or environment. Social problems that can be solved through the application of more yet more spending.
Social studies are much less comfortable pointing their fingers at what I would call un-PC-truths. That the children of divorce, or single-parent families are at a disadvantage. That the heavy emphasis of certain cultures for academic achievement actually works. That children of intellectually gifted parents tend to be smarter.
A truly rigorous statistical analysis can show which factors are important. The mathematical machinery has existed for over 100 years. We just need the data to do it. But why would we do this? To satisfy the need of bigots to feel better than other people? No! We would do this to understand why some people do better than others, and to build a rational education system.
Let me summarize in concrete terms. There is an obvious education gap between the rich and poor, between certain ethnicities, between urban and suburban, between two-parent homes and single-parent homes. Why? We currently point our fingers at a litany of reasons, but we have no idea what their relative importance are. Statistical analysis methods give us the means to discover the precise reasons in a quantitative way. I believe that such an analysis would be incredibly damaging to the PC-establishment, and therefore it will never be adopted.
To give an example: it is claimed that the SAT is culturally biased. If that is the case, we should be able to isolate the culturally-biased questions by looking at the correct-response rates for each question. We could run an analysis against ethnicity, income-level, and geographical location. In a truly culturally-unbiased test, the correct-answer rate for any particular demographic should be perfectly correlated (but not necessarily equal) to the correct-answer rate for the general population.
I don't believe such a study will ever be publicly conducted, despite it's obvious value in diagnosing the validity of the SAT. Is the SAT biased? I suspect it is much more socially useful to claim that the SAT is biased then to ever determine how biased it is.
There is a good, real-life example of this. It was claimed that home-plate umpires are biased against pitchers of different ethnicity. A statistical study was done, and it found that the bias of the umpires was less than 1%, only 1 in 700 pitches was biased! And the discrimination cut both ways - white pitchers were penalized against black refs, and vice versa. This basically put bias in baseball off the table. It's a non-issue, thanks to statistical analysis.
"Just as an aside, thanks for the civil tone you’ve maintained throughout. Very rare, and very appreciated."
Likewise.
I think a lot of what you seem to be saying is reflected in a general schism between the hard-left and moderate-left. The former says its all institutional discrimination, while the latter says it has more to do with cultural dysfunction - although one perpetuated by social structure (low wages, access to health care, incarceration, drug laws, etc.). I'm squarely in the moderate camp - I think kids are unsuccessful because of who their parents are/how they are raised. Genetics plays a role, but only at the margins - you'll have as many low/high IQs within a given SES group.
And fortunately these are all things that we can address. There are many examples of very successful interventions. The Harlem Children's Zone is one. (They actually just did a great story on them on 60 Minutes a couple weeks back). KIPP schools are another. Studies have found that certain models of early childhood parent services can have dramatic results. But these all require a large amount of funding. The HCZ relies on large private donations. The KIPP schools demand sacrifices from teachers that are simply not scaleable - although their model could work with a moderate increase in funding. Fortunately there is a ton of waste in education, and we could do much of what is needed by redirecting resources. But the key is actually looking at what works and being bold about funding it.
Again, I appreciate the discussion. Merry Christmas. 🙂
@ Eli:
I wish I could just say "yeah, I agree, hooray!", but unfortunately, I can't.
I won't repeat my last comment, but clearly we have different views on what should be done in education. I'll repeat the original statement, by Brett Bellmore.
"The moment you try that [SQC/SPC] with education, you’re going to start noticing things that political correctness says you have to pretend don’t exist. The left won’t permit that because they ARE politically correct, the right won’t because their aversion to being called racists is greater than their desire to help minorities."
So, not going to happen. You might get some half-assed version of SPC that deliberately ignores a lot of critical variables, but the result is not going to be all that impressive compared to real SPC."
KIPP and Head Start and all these programs are great, but we've been throwing money at education for decades, and education is getting worse. No nation spends more and gets as little as the the United States. We need to really understand what is going on, and the first step is metrology - designing effective measurement methods. We don't even have that - we argue about whether the tests themselves are unfair!
Instead of spending the next billion dollars on social programs that look a lot like the billions we have already spent, we should spend it to devise truly accurate testing systems. If the SAT is biased, that's a serious problem. Let's study, quantify, and correct that bias. Or scrap it and create a new test. Without accurate measurement, we cannot gauge the success-or-failure of any educational initiative. This isn't even SQC, this is just basic trial-and-error.
I just don't believe it will happen. Lack of good measurement also means lack of accountability - there is simply no way to know who is to blame for America's sub-par systems. I have no reason to believe public school systems will welcome increased scrutiny. This is one of the many reasons I support vouchers - giving parents who can't afford private school the means to opt-out.
Merry Christmas!
It is a difficult profession today to teach in the underfunded classroom, crowded with kids, infested with tests that are silly and distracting. How does one test tell you how effective I am? How does it tell you how effective my students are? In teacher school, they emphasized assessment must be made often, in varied manners, taking in the multi-dimensional creatures we were testing. Yet the tests used to assess our students (and teachers indirectly) are not varied, frequent, multi-dimensional things that assess the many-faceted learning modalities of little people. They are multiple choice tests, primarily, with a short and a long essay, graded not on style or creativity or such, but graded against an easy-to-score rubric that is sometimes confusing to truly simple minds and to truly creative minds. Those who think out-of-the-box need to get special instruction to dumb down and stick to the rubric, don't use confusing metaphors or illusive examples, because the people scoring the tests will take points off for leaving the runway of rubic-laden grading. Students learn the value of following directions, and the insensitivity of a learning system that doesn't care what they really think, just how well they keep their ideas in a fenced yard. What are we teaching? How do those mastering NCLB fare at universities? Can they write? No. Can they think out of the box? Not from much of the test prep we have to give now. Can they write interesting sentences? NO time for that.
What do you want? Accountability? Or Encounters with all their abilities? There isn't time for both this way. There just isn't.
WHo comes to school? In an honors class there are mixtures of students with varying degrees of motivation and abilities. Remember, most of their designations as honors students happened the same way as NCLB, a test or series of tests, evaluating their skills. If their scores are phenomenally high, they become honors students even if they don't participate in classes, do homework, or hand in papers. An F student can be an honors student — I've had them — even if they do nothing. Schools like kids who test high, whether they work, learn, grow, behave, or don't. Their scores are like diamonds, and that is a shame. Some of my worst students were those with perfect or near-perfect test scores, cocky from years of moving along without achievement.
The problem with competition isn't that schools don't have enough. THe problem with competition is that kids don't have any idea what the world is which waits for them outside. Asia will blow over our students in a huff and puff. I have students who need to learn everything from the ground up, because they have been in some isolated bunker until middle school. Punish me for not doing more, when I already work ten or more hours weekly unpaid, for the love of my students, and I'll leave for another career. I was a mid-career professional who became a teacher. I'm not a low self-esteem entrenched robot who will put up with more abuse. I'm a creative, motivating force in the classroom, and you'll lose me in a heartbeat with vouchers and more fragging and nagging at the core of our system. We need investment, creative assessment, not divestment.
@MeasureItAll:
Ironic you would bring up Asia - I was raised in one of those vaunted education systems. We had one-hour midterms and finals in six subjects, starting from first grade. Two midterms and one final per semester. From this class rank was calculated. Not that class rank mattered - school grades were totally irrelevant. The only determining factor for four-year college admissions was the national college entrance exam. A three-day test, administered only once a year. You could have never lifted a pencil at school, and as long as you did well on that test, go to the #1 university in the country. To cheat on that test or to aid in cheating is a criminal offense. The entire test is printed, with answers, in national newspapers the next day.
Do I want this system for the US? No, I went through it. For some kids, this is basically a mild form of torture. But for many East Asian countries, this is an acceptable bargain - forfeit some of the happiness of children for the sake of future labor quality. It works, it's cheap, but it's a system with no mercy. It's not uncommon for grown adults to have nightmares about national exams. The US would never stand for it, but it serves as a useful foil for comparison.
I bring this up because I don't buy the idea that just because NCLB dares to test, that the mere existence of the test squeezes out all other material. Compared to Asian school systems, NCLB is not a high-stakes test. Even in the extreme conditions I was raised in, we still had time for many non-test subjects. I'll illustrate.
Besides the basics (Official Language, English, Western History, Eastern History, Basic Sciences, Math) which were tested, by senior-high graduation, everyone was required to have basic competency in charcoal-sketch, watercolor, and acrylic, play the recorder, identify well-known pieces and basic historical periods in Western classical music, have basic technique in basketball, volleyball, tennis, badminton, ping-pong, soccer, and softball, swim 50m, run 800m, and have decent penmanship/calligraphy with both pencil and brush. Males had additional instruction in survival skills, march and drill, in preparation for military service. We had electives/club-activities that ranged from astronomy to wood-working.
I do agree that one problem is "that kids don’t have any idea what the world is which waits for them outside." But we have very different ideas as to what awaits them. Asian students have been tested their whole lives. We perform under pressure, as we have known nothing else. Americans need to wake up to this. NCLB is a ham-fisted attempt to rectify the situation. It may hurt more than it helps. I'm not in a position to know. But I absolutely believe that effective diagnostic testing is needed. The world is a tough place, and we should train the kids to deal with it.
@Eli:
I submit MeasureItAll's comments as proof of my statements, school systems are extraordinarily resistant to any increase in scrutiny. Just look at the proffered defense. The defense is that everyone is multi-dimensional and had multi-faceted learning modalities, the tests are silly, distracting, not varied, not frequent, and not multi-dimensional, and because they are graded on rubrics, they can't teach about metaphors, and they have no time to write interesting sentences. They can't think outside-the-box, and all the creative, motivating forces in the classroom will leave with all this nagging. The problem is underfunding and NCLB, they need more investment and less nagging.
In other words, more money, and less people checking to see if the money did any good.