Back to Articles
Feb 22, 20261 day ago

What Went Wrong –– How Institutions Eat Themselves

V
vittorio@IterIntellectus

AI Summary

This article investigates a pervasive and puzzling modern failure: why do our major institutions, from transit agencies to schools and hospitals, consume ever more resources while delivering worse results? It moves beyond simple accusations of waste or incompetence to uncover a deeper, structural decay. The piece argues that the system isn’t broken; it’s working perfectly to produce exactly what it now rewards—endless process, risk aversion, and impeccable paperwork—rather than tangible outcomes like finished infrastructure or educated students. The author traces this dysfunction to two intertwined forces: selection and incentives. Over decades, a combination of legal rulings, credential inflation, and social norms has created a hiring environment that filters for bureaucratic compliance over raw capability. Once inside, employees are rewarded for managing liability and following procedures, not for achieving the institution's original mission. This creates a self-reinforcing cycle where the very people who thrive in the system are those least likely to reform it, leading to staggering costs and mediocre performance across seemingly unrelated fields. To understand how well-intentioned reforms led to this stagnation, and to explore whether there’s a way out, the full article offers a compelling and sobering analysis. It connects dots from subway construction to classroom grades, revealing why our bridges aren’t building themselves and what we might do about it.

New York's Second Avenue Subway cost $2.5 billion per mile. That number is real. The MTA spent it, the city borrowed against it, and what they got was three new stations on the Upper East Side. Comparable subway builds in Paris or Madrid come in at a quarter of that, maybe a fifth. Same engineering. Same concrete and steel and tunneling machines. The difference is everything wrapped around the engineering.

I keep coming back to that $2.5 billion because it captures something that I find hard to talk about in a way that doesn't sound like complaining. It's easy to say waste, corruption, or incompetence. And there is some of each. But the number is too big and too consistent for that to be the explanation. Something structural produces a cost like that, and I think it's the same structural thing that explains why California's high-speed rail project, budgeted at $33 billion in 2008, now sits somewhere past $128 billion with barely any track in the ground.

This is part 5 of the What Went Wrong series

Part 1: We Forgot We Were Animals

Part 2: The Thirty Months That Broke Everything

Part 3: How Families Disappeared

Part 4: The Attention Economy as Superstimulus

And it's why the CDC couldn't get diagnostic testing together during the first weeks of COVID when every week mattered enormously. These outcomes are so reliably bad, across so many different institutions, that the badness itself starts looking less like failure and more like output. The system is producing exactly what the system produces.

The usual response to any of this is that it's about underfunding. Give us the resources and we'll deliver. It sounds reasonable until you go look at the spending data.

Per-pupil education spending has more than doubled in real terms since 1970, according to NCES data. Healthcare spending per capita has gone through the roof. Adjusted for inflation, we pour more money into nearly every public function than we did fifty years ago. The money went up. The results went sideways or down. More spending bought more staff and more process, and the thing it was supposed to buy — better outcomes — never showed up.

The other thing you hear is that everything is harder now. More regulated, more interconnected, more complicated than it used to be. And that is true, up to a point. But the previous generation built 48,000 miles of interstate highway, roughly on budget, and put people on the moon using slide rules and filing cabinets. We have GPS-guided tunnel boring machines and struggle to get a rail line through the Central Valley on schedule.

So what actually changed?

Organisms optimize for survival in whatever environment they find themselves in. That's how everything has always worked. Put a monkey in a hierarchy that rewards paperwork and you get a monkey who is insanely good at paperwork. The question was never whether people inside institutions are smart (often times they are) but what the selection environment made them smart at.

I think it's about who ends up running things and what they're rewarded for once they get there. If I had to boil it down to two words they'd be selection and incentives, and I mean that almost literally.

Selection used to be direct. A bank hired someone who seemed good at banking. A school hired someone who seemed good with kids. The assessment was subjective and sometimes it was wrong. But when it was wrong you found out about it fast because incompetence was visible, and visible failure informed the next round of hiring. The whole thing was sloppy and biased and it more or less worked.

That broke down, and it broke down for reasons that each made sense on their own even though the accumulated effect has been brutal.

Credential inflation is a big part of it, I think. When I went through school, having a degree still meant something about you because not that many people had one. Now roughly 40% of American adults hold a bachelor's. The degree communicates that you can sit through four years of formal instruction and come out the other side with paperwork to show for it. Whether that tells you anything about how they'll do at a job depends on the job, and for most of them the correlation is pretty loose. But the threshold keeps rising. Bachelor's is baseline, master's is the distinguishing credential, and the actual information conveyed by any of it creeps toward zero while the cost of acquiring it goes up year after year. The people who are really good at collecting credentials tend to be people who are comfortable with bureaucratic process, tolerate structured environments well, and defer gratification in pursuit of tokens. Those are all fine qualities to have. They just don't map as cleanly as you'd hope onto being good at most of the actual work.

The legal dimension matters more than most people think about, and I want to spend a minute on it because the chain of consequences here is genuinely interesting.

In 1971 the Supreme Court decided Griggs v. Duke Power Co., which was supposed to stop discriminatory hiring screens but ended up doing something much stranger over the next fifty years. The ruling said that if your employment test has disparate impact on protected groups, you have to prove it's directly job-related through validation studies, and those studies are expensive. So what happened is that direct ability testing became legally frightening enough that most employers just stopped doing it. Easier to require a degree and let the university system do the filtering. The credential serves as a proxy for ability while providing legal cover. Nobody sues you for requiring a master's degree. People do sue you for requiring a skills test that their demographic group fails at higher rates (this is one of those outcomes where everyone involved had defensible and reasonable intentions and the result is still catastrophic, which is most of institutional failure if you look closely enough).

None of this made direct assessment illegal, to be clear. It just made it risky and expensive, and it turns out that rational organizations will go to great lengths to avoid risk that they don't have to take. So here we are.

Social pressure pushes the same way. Telling someone they're mediocre produces conflict. Writing an honest negative reference burns relationships. So evaluation drifts toward positive regardless of what the person actually did. Grade inflation at American universities makes the dynamic almost comically visible. The percentage of A grades has more than tripled since the 1960s, based on data Stuart Rojstaczer has been compiling at GradeInflation.com for over twenty years. The students didn't get three times smarter. Professors figured out that handing someone a C means complaints, formal appeals, bad course evaluations, and uncomfortable meetings with department chairs. Handing out an A means none of those things. Each professor independently makes that calculation and the system collapses toward cheerful meaninglessness.

Once you're inside the institution, incentives finish what selection started.

I knew a special education teacher, some years back, who spent about forty minutes per kid filling out individualized education plan compliance documents. She logged every accommodation, tracked every modification, recorded every parent phone call. Her paperwork was immaculate. Several of the students she was documenting accommodations for could barely read. But the forms were done correctly, so as far as the system was concerned, things were working. The process was the product.

That story captures something about what happens in a system where you can't easily measure the thing you're supposed to be achieving, or where measuring it honestly would be embarrassing for everyone involved. You measure the procedure instead. Did the steps happen? Were they documented? Did the right people sign their names? Those questions give you clean answers. "Did the child learn anything this year" is messy and potentially incriminating and nobody wants to be the one asking it.

Healthcare runs on what is basically the same logic. The question that I think shapes more clinical decisions than any diagnostic guideline is "what does this look like if we get sued." Defensive medicine, ordering imaging and lab work nobody medically needs because the absence of that imaging could become evidence of negligence, eats up enormous resources. The doctor isn't really treating the patient's condition. The doctor is treating the hypothetical future lawsuit, and whatever happens to fall out of that process is what the patient gets.

There is a ratchet mechanism in how bureaucracies expand that I find almost elegant in how inevitable it is.

The basic dynamic is that every administrator you hire creates a constituency for more administration. The people managing compliance need managers of their own. Expanding your headcount means a bigger budget, more direct reports, more organizational weight, and all of that signals success for career advancement purposes. Cutting staff creates enemies. Expanding creates allies. The incentive is completely unidirectional and nobody has to be self-interested or cynical for it to work. It's just how promotion functions inside big organizations. NCES data on higher education shows administrative positions growing much faster than either faculty or enrolled students across multiple decades. Hospitals tell the same story. A growing share of institutional capacity goes toward administering the institution itself.

And then the feedback loop locks it all in. Organizations that hire for credentials and compliance end up full of people who are good at credentials and compliance. Those people, predictably, design hiring systems that prize what they're good at. The organization calcifies around its own selection criteria. The kind of person who would have done well under the old system doesn't apply anymore, or gets screened out in interviews that test for cultural alignment instead of raw capability, or looks at the credentialing requirements and decides it isn't worth the trouble. And the people already inside get progressively better at playing the institutional game, because the institutional game is the only thing there is to be good at.

The pattern recurs across sectors that have nothing to do with each other, and that is how you know it's structural rather than just coincidental or anecdotal.

The pattern recurs across sectors that have nothing to do with each other, and that is how you know it's structural. ETS data from 2014 through 2019 shows students heading into education programs scoring below most other professional tracks on GRE scores by intended field. Those students become teachers. Some of their students become education students. The cycle tightens.

Administrative headcount in healthcare has outpaced clinical staff growth for decades running. Physicians spend more of their day on documentation and less on patient contact. Costs spiral. Outcomes, by international comparison, stay mediocre. The system bills insurance companies. Whether the patient improves is a secondary concern. Alon Levy's Transit Costs Project put hard numbers on something everyone already suspected, infrastructure that went up in four years in the 1970s takes twenty now, costs ten times as much, employs vastly more lawyers and consultants and compliance officers. The delays and the expense are exactly what the process was designed to produce, which means the process is working perfectly. It just wasn't designed to build things quickly.

HR departments that barely existed fifty years ago now represent a real chunk of organizational headcount, and their primary function has migrated from talent acquisition to liability management. Documented credentials provide a paper trail. Gut reads on someone's ability do not. These institutions answer to different authorities, serve different purposes, operate under the same selection pressures, converge on the same dysfunction. The convergence was never planned. It emerged.

These institutions are nominally independent. They answer to different authorities and serve different purposes. But they operate under the same legal constraints, the same credentialing infrastructure, the same social norms around how you're allowed to evaluate people, the same regulatory frameworks rewarding process over outcomes. Similar pressures produce similar results. The convergence was never planned. It emerged.

The honest counter-argument is that the world did genuinely get harder. Regulations are more complex. Legal exposure is higher. The population is more diverse, which creates real challenges for standardized assessment. Technology shifts fast enough to make expertise perishable. Fair points, all of them.

But if difficulty alone explained it, you'd see uniform degradation within any given set of constraints. You don't. Some companies massively outperform their competitors operating under identical regulations. Some countries build equivalent infrastructure at a fifth of the price under comparable legal frameworks. The constraints are real. They do make things harder. They don't make things this hard. The gap between "harder" and what we actually observe is where the selection problem lives.

A second counter worth mentioning: maybe credential-based selection is actually the correct adaptation to how modern institutions work now, and what I'm interpreting as dysfunction is really optimization for a different environment than the one I think should exist. This has some force for individual behavior. Playing the game as configured is rational even if the game's rules drifted from its original purpose. But it doesn't survive the observation that performance keeps declining. If the current selection truly fit the current environment, you'd expect improving results. You get the opposite.

So what do you actually do about any of this.

See the game for what it is and play it without pretending that winning the credential race is the same thing as being good at anything. Those are separate accomplishments that overlap sometimes and less often than most people assume.

Build your real capabilities outside institutional validation channels. Demonstrated work communicates ability in ways a diploma can't. The diploma gets you through the door. What you can actually do determines whether being through the door matters.

If you're in a position to hire people, fight for work samples and trial periods and demonstrated performance. The legal risk is there but it is substantially overstated by most of the HR departments I've talked to about this. Documented job-relatedness provides considerable protection, and the gap in signal quality between watching someone do the work and reading their resume is large enough to justify some friction.

If you're building something, stay lean. Every administrative hire you make creates pressure for the next one, and the ratchet turns in one direction. Fight that early because you will not be able to fight it later.

Look for places where competence still determines outcomes. Smaller firms. Mission-driven groups. Fields where results are measurable and hiding behind process doesn't work. Those environments are less comfortable and less forgiving than the credentialed bureaucracies, which is why they function.

Some institutions are past the point where reform from the inside is a realistic project. That's not defeatism, it's arithmetic about where to spend your effort. Building something new with better selection principles can be more productive than trying to turn around fifty years of accumulated selection pressure pushing the other way.

The institutions aren't evil. I want to be clear about that, because I notice how easy it would be to read this whole thing as an attack on the people inside them. The people inside them are mostly doing what the system rewards, which is what I would do and what you would do. The system rewards the wrong things, and it does so for reasons that each made individual sense as they happened. Legal protections meant to prevent discrimination ended up discouraging assessment. Social norms meant to be generous ended up being dishonest. Accountability systems meant to ensure quality ended up measuring paperwork instead.

Each step was somehow defensible in the moment and nobody really chose the destination.

I genuinely don't know whether any of this can reverse. Probably not from within since the people who would have to change the selection criteria are the very people the criteria selected for and benefit the most from them. It's going to be hard, and I don't see a simple way out of it. Maybe the answer is parallel institutions, new ones built from scratch outside the captured system. The track record on that is mixed at best, if I'm being honest about it.

Does any of it get better? Probably no.

The bridges aren't building themselves, and the machinery that used to build them is too busy filling out forms to notice.