Category: Management

Do professors care whether college students are actually learning?

The lead article in the 25th anniversary issue of the peer-reviewed Journal of Public Affairs Education , by Heather Campbell, is a deep-dive review of research that throws significant shade on student evaluations of teaching (SETs). SETs do not measure student learning, and may actually have the wrong sign (not to mention that their inherent gender, age, and racial/ethnic bias means their use for personnel decisions is probably illegal). Universities like my own, in which SET scores are the main, or often the only, teaching evidence used in promotion and tenure decisions are systematically damaging student learning.

From the abstract:

In many if not most colleges and universities in the United States,
raw scores from Student Evaluations of Teaching (SETs) are the
primary tool of teaching assessment, and teaching evaluations
often have real consequences for promotion and tenure. In 2005,
JPAE published an article on teaching evaluations, and this article
added to what was at that time a somewhat thin literature indicating
that SETs are systematically biased against female faculty,
and probably against older and minority faculty. Since that time,
this literature has swelled and grown and now the evidence that
SETs are invalid and systematically biased is too strong to ignore.

Why do we keep doing this? I have four hypotheses:

(1) A body of research of which I am unaware and can’t find refutes the findings Campbell summarizes. This is certainly the most generous conjecture, but JPAE is peer-reviewed and reviewers would have rejected this paper if they knew about such material. Not to mention reviewers for all the publications Campbell cites.

(2) Confirmation bias. We used to think SETs were valid indicators, and we subconsciously reject evidence that would change our mind. This mode of inference has recently been validated by our US president, who “just knows things” that are contradicted by every kind of expertise and evidence.

(3) Fecklessness and laziness. We assert our commitment to good teaching if asked, but actually we just don’t care enough to do anything that would actually advance it. The joke is, “Teaching is the tax you pay to do your research; tax evasion is a crime, but tax avoidance is the duty of a citizen” and the corollary is, “why are you talking about whether students are learning, when I have a research article to finish writing for my academic peers to admire?” SETs take a distasteful task (collegial responsibility for better teaching) off my desk and load it onto an unpaid, docile labor pool (students); what’s not to like?

(4) Fear. I have never received evidence I can respect as a scholar, or any other way, that I am any good at teaching or could become so (I do have evidence of this kind that I can do research OK and that I’ve gotten better at it over the years). Teaching is affectively fraught, and like everyone I know, I’m sure my emotional intelligence is not what it should be. The ego hit of talking while a roomful of people write down everything I say (lecturing, just as an example of a dubious pedagogical habit) is a lot to risk by trying to learn a new skill. Anyway, improving my teaching will take a lot of time and my job depends on publishing.

We would be a lot better off if we could shift our attention more generally from summative evaluation (at promotion time) to formative methods (coaching and experimentation between these high-anxiety moments). I suppose one could believe that college teachers only respond to money and status incentives, so if we reward good ones and ding or fire bad ones, we will eventually have only good teaching, but one would be wrong (ask any successful manager whether you can fire (or bribe) your way to success). One would be especially wrong if your filter for “good teachers” doesn’t measure student learning.

What we need is not a cheap, lazy way to pretend we are improving our teaching, but a real quality assurance program that a Google or Toyota manager, for example, would recognize as such. Got kids choosing a college? on your junior year visit, ask what their QA program is, be sure it doesn’t depend on SETs, and don’t be distracted by the fancy athletic facilities. Are you a student, paying through the nose with your time and money for the best possible education? Do the same, and if you don’t get good answers, recruit your classmates to go in the quad with pitchforks and torches.

Staffing for success

Every management book says choosing the right lieutenants and partners is critical to success for any enterprise: always surround yourself with the best people. Even Donald Trump at least paid lip service to the idea when he promised to do that coming into office (that was before the best, the pretty good, the OK-I-guess, and strata right down to unqualified figured out it was time to run away from him, so he had to hire out of dumpsters). So his judgment of Michael Cohen as “weak” and “not very smart” seemed odd given their two-decade relationship…was that a new insight?

Readers will surely be as puzzled as I was, but Alexandra Petri brilliantly resolves this mystery today.

Historians of our current malaise will have to credit Trump, against all his evils, for inspiring Petri to new personal bests.

Enforcing laws against interstate tobacco smuggling

Cigarette taxes protect health by reducing smoking.

But tax disparities across states create a multi-billion-dollar annual market in smuggled tobacco products. Current enforcement efforts are inadequate and ill-organized.

As a result, the illicit trade in tobacco products (ITTP) is growing, and the larger the market grows, the harder the problem of controlling it. (That’s the usual positive feedback problem in violation rates due to enforcement swamping.) So inaction now has long-lasting costs.

Tax equalization would solve the problem, but isn’t likely to happen. Feasible changes in enforcement strategy could protect health and revenue while reducing crime.

Further thoughts on this from Mike DeFeo and me now up on SSRN.

Berkeley athletics task force ducks engagement, steams onto rocks

Berkeley’s Task Force on Intercollegiate Athletics (TFIA)  has submitted its report  on what to do about an enterprise that soaks up tens of millions of dollars of subsidy each year while the university as a whole is being asked to eat almost $200m of budget cuts (see the TFIA assignment below). It’s appropriate to thank the members for the time they spent on this project. I wish it were possible to recognize their efforts as consequential or useful, but no such luck.  My expectations were modest, but this result dashes even my cautious hopes, and as I know several of the members to be smart people with the best intentions, it saddens me to say so. Eleven members, working for almost a year, have come up with five single-spaced pages of content (no, this is not the executive summary, and it is not a memo boiled down for Donald Trump), and one big table of financials uncritically assembled from IA’s annual NCAA P/L reports.

A bitter irony is the appendix listing (without links to access any of them) five previous TFIAs, from 1991,’92,’99,2000 and 2010. Isn’t a standard definition of insanity “doing the same thing again and again expecting a different result”?

Continue reading “Berkeley athletics task force ducks engagement, steams onto rocks”

Why does art stupefy otherwise smart people?

What is is about art, that when smart, tough-minded people get near it, their brains turn to mush? I’ve worked in a museum and universities, and studied the former professionally: while management of the latter is often very feckless and lax, museums take the cake. Most recently, but not exceptionally, a board of trustees starring the business élite of New York City has managed to let the Metropolitan Museum of Art go seriously into the financial toilet, despite having assets worth at least $100 billion.

Today we have a lawyer, apparently capable of actual research and inference from evidence and writing literate English, proposing that artists should have a full value deduction for the untaxed value of gifts of their own work, something we fixed fifty years ago.  He managed to get that truly loony and regressive idea (like all deductions, this one is only valuable for successful artists who are already rich) past the editorial page editors of the New York Times. I can see them now, looking at this piece of copy and going all gooey-eyed and misty…”Art! Awww…we love art! Let’s print it!”

OK, Mr. Rips and NYT tough-minded skeptical journalists, how’s this idea?

Janet Napolitano

President, University of California

Dear President Napolitano:

Because of my great love and affection for the University of California, I propose to give half my working hours to Cal as pro bono work, and only take a salary for the other half. Now, I will need you to double my salary rate for the half time I’m on the clock, but this won’t cost you anything. What it will do is enable me to deduct my unpaid time against my new salary under the new rules, which will leave me with no taxable income at all: we can stiff the taxpayers for my whole tax bill! Naturally, I’m happy to give you a cut of this windfall, shall we say 20%: you make money, I make money, the students still get their courses…who could object to this?

I might add, doctors in our hospitals can really clean up this way; in fact anyone who works for a nonprofit or a government agency is looking at a historic opportunity to rip off the taxpaying public, and surely we’re as lovable and deserving as artists whose work sells for hundreds of thousands of dollars, and knowledge and health are as important as art.

Do we have a deal?

Very truly yours,

Michael O’Hare

[my coauthors and I get well into the weeds of this foolishness in Patrons Despite Themselves: Taxpayers and Arts Policy, if you want to follow up. Sheesh.]

Annals of Commerce: customers are not inventory

Our captains of industry (the kind of people Donald Trump loves to throw into government jobs for which they are completely unprepared) are really smart in so many ways, or they wouldn’t be so rich, right?  The latest awkward exceptions are the Wells Fargo bosses who supervised a national enterprise of cheating retail customers, and Oscar Muñoz of United Airlines, who somehow contrived to be the last person on the planet to figure out that dragging [sic] a paying passenger off his plane so he could ferry four staffers to a location convenient to United was Not OK.  Muñoz’ witless behavior is something of a surprise, because he has not spent his life in a privileged bubble; according to Wikipedia he was the first in his family to go to college, from a family of nine kids, and last year took off a couple of months from work to have a heart transplant. The man has paid some dues.

I would like to add to the ample discussion of this episode (see especially Helaine Olen’s piece enlarging the scope of debate beyond the case at hand) an insight I owe to my late colleague Robert Leone: Muñoz’ problem, and his lieutenants up there in the executive suite, have their feet nailed to the floor for two reasons.  One is that they are actually not that smart, and do not understand their own cost structure. The other, and my main point here, is because they have no clue what it’s like to be one of their customers! They never fly Y class, or at least haven’t since that product became a hated, degrading, wretched experience in their hands. Continue reading “Annals of Commerce: customers are not inventory”

Enabling institutional decay by bad journalism

For the second time in a month, Robin Pogrebin points her keyboard at the ‘crisis’ afflicting the Metropolitan Museum, and misses the central fact her sources are desperate to hide from her (and from us). The broad outlines of the situation are that the Met is going broke, bleeding money from reckless initiatives and programming it can’t afford (and, presumably, fundraising that’s falling short of hopes).  The sums involved seem quite large, a $40m deficit and a $600m new modern art wing it can’t afford to build.  But you would never know from Pogrebin’s reporting, and certainly not from any official Met information, that the museum owns a collection worth at least $40b, of which it shows or can ever expect to show only a tiny fraction.  Sell, say, 2% of that off the bottom and out of musty storage (much more than 2% of the objects by count, of course), and there’s the new wing with money to spare, or an endowment that will cover the deficit forever, and the best 98% of the collection still in hand.

Better yet, the sold works would almost certainly go to buyers who will show them, like smaller museums outside NY, or even private collectors. Why can a serious reporter like Pogrebin skate right by relevant facts like this? Because museum administrators have conspired (literally, in writing themselves a code of ethics that forbids selling anything except to acquire more work) to hide that wealth from view (museums simply omit their collections from their balance sheets), and pretend that redistributing it to where it could actually provide some art engagement is some sort of moral offense against art.

It’s a wonder of the world that trustees, many of whom are tough-minded business people, go completely soft in the head when they sit in museum board meetings. If accosted by a homeless beggar saying “please help me! I’m hungry, and completely penniless. Well, except for the million-dollar art collection Dad left me, but you wouldn’t want me to sell any of that, would you?”  I do not think Mr. Gotrocks would pony up, but he and the missus are happy to watch their museum cut programs, go on collecting art it has no space to show and no money to conserve, and run deficits, on exactly the same absurd proposition.

 

Another Cal athletics moment

With our intercollegiate athletics department’s typical management finesse, we extended our football coach’s contract about a year ago…and fired him this week. He walks away with about $6m in severance, but that’s OK because we’re just finishing up the zillion-dollar severance payments for the previous coach, and the AD who is now at Penn State, so there’s lots of money just lying around that would otherwise be wasted on fixing classrooms, or scholarships for non-athlete students who just play sports for fun and don’t put any eyeballs on TV commercials. The intercollegiate athletics program at Cal costs about $30m a year (net), a sixth of a $180m campus deficit; a task force of alums, faculty, and staff is working on proposals to fix this.

The athletic director shares a set of insights that deserve attention, and translation:

We are continuously evaluating our program and looking for ways to make it better - whether that’s through additional academic support, recruiting, facilities, staffing, culture, leadership or anything else that can help our football program succeed. [1] Primarily, we want what’s best for our student-athletes [2] and have a head coach in place who is fully committed to our program  and our university [3].

….Our objective is long-term financial sustainability for our department. In order to do this, we understand that investing in football is critical [1]. We believe that this change will reinvigorate the program, stimulate lagging ticket sales and renewals, and energize our donor base. [4]

….We want to win championships. The success of our football program is vital to both our department and our university community [5], and its influence can be felt well beyond Berkeley.

1: Almost everything in this list costs money, and we intend to keep spending it no matter what that task force says, or what weird mission the outgoing or incoming chancellor thinks a university has.  It’s our tradition of a decade here at Cal to keep throwing money at a mediocre football program, and we take our traditions seriously. Sooner or later, maybe as little as $6m later, inshallah, the larger forces of big-time college sports will abate, the bleeding will stop, and we will reach some sort of equilibrium, right?

2: To be clear; we retain the services of the conditioning coach who killed one football player, sent another to the hospital, and cost us $5m in a liability settlement. We certainly aren’t going to rein in practice times so they can sit around in classrooms or labs, or do a bunch of wussy problem sets. “What’s best for our student-athletes” is not what people outside the cult might think the phrase means.

3: This is just sports PR blather, of course, the language of press conferences and after-game interviews; $3m college coaches are fully committed to their careers and if anything, expect their employers to be committed to them with limitless staff, facilities, money, and perks. Pete Carroll’s effortless leap to the Seahawks from the shambles he left at USC is instructive.

4: Chronicle reporting is occasionally sloppy, and in this case we are not informed whether Williams clicked his heels together three times and closed his eyes as he said this last.  Nor whether the “we” actually includes any living person on earth with a three-digit IQ.

5: This combines a statement of fact with a religious utterance based on faith. No, it’s not vital to the community, not even close, though its ruinous cost certainly inflicts a lot pain on the rest of us. My department just completed a faculty search and not one of the candidates asked about the prospects of the football team or even knew our record. We lost a prof to Stanford a few years ago, and in all our discussions of his move he never once brought up Stanford/Cal football. I have talked to dozens of undergraduates and grad students over the years and not found one who came to Cal because our football (or men’s basketball) teams were better than those at other schools to which they were applying. I have been here semester after semester, talking to colleagues in social science, humanities, and hard science across that university community, and the salience of football in our socialization, community spirit, and plain water-cooler schmoose is similar to the salience of pro wrestling. Big-time sports may be ‘vital’ for Clemson or Florida State, but not for us.

What can we expect in the near future?  The new coach will need to do some really desperate recruiting of defense players at the least, and develop a quarterback (unless he decides to bring in a graduate transfer ringer as Dykes did this year).  The program will be even less attractive to high school stars, however, so I have probability of about 0.3 that we will be reading about recruiting violations of the type fictionalized in the memorable movie Blue Chips, or recruiting and oversight failures of the type that recently humiliated Baylor. Meanwhile, ticket sales will keep going down, the athletics deficit will grow, and the new chancellor will find his feet stuck in the big muddy from his first day.

The Oakland Warehouse Fire

Three big lessons from this catastrophe.

First, think before you wish for ‘job-killing, economy-crushing regulations’ to be swept away. Fire and housing codes would have saved 33 36 young lives here if they had been enforced; an enormous fire in Cambridge the same day killed no-one, partly because there weren’t as many people crammed into one space, partly because the eleven old buildings involved met codes, or close, and had many ways out, partly because they weren’t full of paint thinner and the kind of flammables artists use at work.

Second, primary responsibility obviously rests with the owner and the building manager. But this was an implementation/management failure, not a policy failure: Oakland’s codes are entirely adequate to prevent this kind of thing, but they weren’t effectively used, whether because California has crippled its local governments financially by Proposition 13 and other short-sighted tax choices, or because the enforcement function in Oakland was incompetent or feckless.

The inspector who visited this deathtrap on Nov. 18 was “unable to gain access” and apparently the matter dropped there. It’s possible California needs some new legislation. For example, I have no trouble with the idea that the owner of anything larger than a single-family house has a duty to make himself  (or a subordinate or attorney with the keys) reachable for purposes of inspection access within 48 hours of any safety-related complaint the city chooses to act on. If he doesn’t open the building for the inspector, the inspector can admit himself, by force if necessary, during business hours.

Berkeley had a similar episode a decade ago, which unfolded quite differently because the city kept after the landlord. No fire, no deaths, no tragedy…

…but a bunch of artists out on the street. Third, the housing/workspace crisis for artists in happening cities is real (not to mention for teachers, students, civil servants, and every kind of poor person). The resistance to cleaning up the Drayage building came from the tenants whose safety was the point of the enforcement action, and they correctly understood that they had no workable options; things are worse for artists now.  Running around rousting artists from improvised housing and homeless from tent camps won’t fix this. Unless we make it easier to build, confront NIMBYism, and shovel out more housing supply-yes, including subsidized live-work spaces-we will have nightmares like the Ghost Ship and homeless camps under freeway ramps. People who can’t afford housing, whose price (in the Bay Area, and other places) has sailed into a completely unattainable stratosphere, will live somewhere, and that somewhere will be inhumane, intolerable, and dangerous in so many ways.

 

Giving management a bad name

My graduate school at UC Berkeley has raised some funds and we are embarking on a new building. This morning a group of staff and I met to kick off the programming process, the critical stage in which what we want to do in our new space, and how we want to do it, gets translated into something we can give an architect to start with. Inevitably, this needs to specify named functional spaces with sizes along with narrative material describing how we want to do our work.

There is an office high up in our organization chart called “Space Management and Capital Programs”.  As an architect who has been at the intersection of building users and designers more than once, I would welcome guidance from such a unit such as “How to decide whether you want cubicles plus a lot of small conference rooms, or private offices” or “New options for classroom design: thinking outside the lecture hall box.”  No such luck; instead we were provided this remarkable document, new since our last building project more than a decade ago (that has been widely admired as a big success). Apparently the campus administration intends it to be regulatory, not advisory.  Not surprisingly, no-one was willing to put his or her name on such an ill-informed, incompetent exercise of mindless bureaucratic pound-foolishness.

The authors obviously hail from a ruthlessly hierarchical private sector culture, where the size of one’s workspace must precisely indicate one’s place in a pecking order.  One would think the right question would be “how much value would an additional square foot of space for someone doing job X add to the organization”, but one would be wrong.  Professors are all alike (not to be confused with adjuncts and lecturers, who do more teaching and actually meet with more students in office hours), they all do the same thing, and what bricks and mortar are for is to indicate precisely how much better and more important they are (50%) than  than the staffer who manages their research funds or gets students enrolled in their courses.

This document describes a world in which all meetings are held in the office of the senior person attending, are populated in proportion to his rank, and in which peers never need to collaborate; don’t even ask about faculty meetings with student groups.  That is not the world we live in, Mr. Space Management bean counter.

The rigidity of this absurd effort by central administration to tell us how to do our jobs — jobs that differ widely across individuals, departments, and units - and its insistence that we use the precious resource of physical space to pointlessly signal status are not, however, the worst part of this fiasco. The worst part is its relentless, insistent, ignorance of the real benefit-cost facts that reasonable people would use to make decisions like, duh, “how big should whose office be?”  I railed about this a few years ago, and see no reason to revise the analysis. All of these standards are - put aside their mindless rigidity - much too stringent. Building space by these rules sabotages everything we do, from research to student learning. If there is such a thing as government waste, and abuse of personnel and citizens, this is what it looks like.

I haven’t seen the corresponding classroom design standards document, if there is one, but I await its appearance with real alarm.

We are currently under really severe financial pressure owing in part to some reckless, foolhardy, and uninformed investments in intercollegiate athletics facilities and in part to our failure to educate our legislature about how central higher education is to the welfare of the state, now and in its future.  One meme constantly rolling through our discussions is that our senior administrators seem to be paid an awful lot, and there seem to be more and more of them. I teach management and I do not tolerate mindless disrespect for public officials and people who make organizations work, but a document like this is a problem for me, because it makes a prima facie case that at least some of those very well-paid senior administrators suck at what they do.

It’s apparently news to at least some of our managers that the purpose of overhead agencies and administrators is not to save money! If that’s what we are about, we can just shut down and save it all. Guess what, folks: your job is to help shop-floor workers create the most possible value for the resources we consume, and when you get this mixed up, you do a lot of damage.  And another thing: this is a research university, and our duty to society is not to see what everyone else is doing and copy it (“based on …space guidelines from other higher education institutions and the private sector”), it is to learn from others (of course), and do our own thinking and push boundaries of habit and convention. If you don’t like that duty, please go work somewhere else, and if we’re stuck with you, well, I’ll quote Randy Newman:

“…if you won’t take care of us
Won’t you please, please let us be?”