The evolution of software development over the past decade has been very frustrating. Little of it seems to makes sense, even to those of us who are right in the middle of it.
We usually only notice trends and popular frameworks and libraries after they’ve exploded in popularity. By that time they’re often so far removed from their original context that their initial technical merit is a hard-to-distinguish quiet signal in the overwhelming noise of hype, grift, and false promises.
If you do take the time to trace these trends back to their early beginnings, you often discover that, no, it wasn’t just somebody’s con or VC startup pitch that happened to get traction. Usually there was something there at the beginning, however slight that “something” might have been.
- MongoDB has its uses. It’s a specialised tool that works remarkably well in certain contexts (and remarkably badly in others) and many of its very earliest adopters adopted it for solid reasons.
- React was, at the time, a very clever and ergonomic solution to building a “view” layer for the web. It’s well past its sell-by date but it was a remarkable innovation when it was first released over a decade ago.
- Large Language Models are a remarkable discovery that should, in the long term, tell us something interesting about the nature of text. They have some potentially productive uses. Its destructive uses and the harm it represents, however, outweigh that usefulness by such a margin that, yes, I absolutely do think less of you for using them. (You can argue about productivity and “progress” all you like, but none of that will raise you back into my good opinion.)
- Rails and Django made an entire class of software – medium-sized web services – more straightforward to create, develop, and maintain. They were a genuine paradigm shift in web development that probably ended up being a bit overused and extended into types of software they weren’t suited for.
There is generally something “there” in the various technologies that become developer darlings but that merit often gets lost in translation as the hype surrounding them spreads.
That isn’t what’s perplexing. What’s harder to understand is the reason why these technologies take off when others don’t. That’s despite having equal merit and being equally loved by the devs that use them.
Some of it’s down to resources. MongoDB had VC-funding. Facebook was, even a decade ago, one of the largest companies on the planet.
But that doesn’t entirely explain it away. Plenty of VC-backed plays falter and pretty much every major tech company has implemented and promoted their own frontend view library at one point or another. Ebay’s Marko, for example, is at least as old as React.
Some of it is simply down to time. You’d expect Electron to be more popular than Tauri because Electron pioneered the category of “use web tech to make mediocre desktop software” and Tauri is a johnny-come-lately that commits the cardinal sin of requiring that you care about the security of your app. But, again, that doesn’t explain all of it. The Cordova/PhoneGap family of mobile app frameworks have very steadily lost mindshare to React Native despite predating it by six years. CoffeeScript disappeared seemingly overnight while TypeScript is a juggernaut that has pushed pretty much every other “compile to JavaScript” language out of the market.
This means time, or order of release, doesn’t explain it either.
Complex systems are inherently unpredictable. We’ll never have a complete answer to why something takes off in popularity in the software developer ecosystem any more than we’ll ever figure out exactly why some books become international blockbusters and others don’t.
But we can form theories about what sustains popularity – why do some frameworks and libraries go from strength to strength and while others plateau or even peter out.
My theory is fairly straightforward:
The long-term popularity of any given tool for software development is proportional to how much labour arbitrage it enables.
The more effective it is at enabling labour arbitrage, the more funding and adoption it gets from management.
What do I mean with labour arbitrage?
Labour arbitrage is, effectively, when employers play employees in different regions and fields against each other:
The primary purpose of using labour arbitrage is to enhance operational efficiency by reducing labour expenses without compromising the quality of goods and services. Firms can leverage differences in labour costs, skills, and labour availability between geographical regions or different labour markets to achieve cost savings.
A key requirement for effective labour arbitrage is that the employees can be treated as interchangeable, well, components. Jobs that require expertise lend themselves less to arbitrage than jobs that don’t.
MongoDB’s popularity among managers during its peak was largely down to the idea that you no longer needed a database expert. Just throw the data into the document DB puddle and let your existing less-specialised developers handle it. The promise of the document database during the peak of their hype was that you didn’t need to employ as many specialists.
Electron, PhoneGap, and React Native promised to let companies replace their expensive platform specialists with more commodified generalists.
Standardisation in web development lowers costs, increases predictability, and makes the various browsers more interchangeable. It’s a hedge that reduces the individual market power of each browser, but usually in equal degree while increasing the value of the overall web, leaving each browser vendor better off as a result. Their market share might not increase but they have a bigger cut of a larger pie. Incompatibility tends to drive developers and companies to other platforms, reducing the overall pie.
Standardisation of labour, conversely, does not benefit labour.
“Little boxes on the hillside”
React and the component model standardises the software developer and reduces their individual bargaining power excluding them from a proportional share in the gains. Its popularity among executives and management is entirely down to the fact that it helps them erase the various specialities – CSS, accessibility, standard JavaScript in the browser, to name a few – from the job market. Those specialities might still exist in practice – as ad hoc and informal requirements during teamwork – but, as far as employment is concerned, they’re such a small part of the overall developer job market that they might as well be extinct.
The component model itself is another layer of arbitrage, especially when coupled with CSS-in-JS approaches or libraries such as Tailwind. Instead of building teams around cross-functional specialities, you can build your software around interchangeable components.
Components can be mixed and matched:
- Some from other departments.
- Some outsourced.
- Some of them open source.
- Some dysfunctionally emitted by a Copilot.
The components sourced from an intern fixing ChatGPT’s output just enough for it to run and the exhaustively tested ones from a senior developer are equivalent in the eyes of management.
And one is much, much cheaper than the other.
This trade-off worked for developers for as long as demand for developers outstripped supply. The pie kept growing so the popularity of arbitrage-enabling systems among management wasn’t a cause for concern. We even came up with our own excuses for it, to defend the choice against observations of their obvious technical deficiencies.
“React simplifies recruitment.”
That’s labour arbitrage. Recruitment is easier because labour has been commodified. It results in workers who don’t have the training to troubleshoot DOM code or solve problems using CSS (often a much cheaper, more effective, and faster technical approach over doing it in JavaScript). But that’s fine because the market is growing and you can just hire more of them. Once they’re employed they’ll get the chance to practice and learn on the job. There’s always plenty of web developer jobs to go around.
Until there isn’t.
It’s all about deskilling developers, not productivity or code reuse
Many early readers of this essay found the concept of labour arbitrage to be quite unclear.
—“Aren’t you just arguing against productivity? If an excavator replaces a hundred men with shovels, isn’t that the ultimate kind of labour arbitrage?"
It’s not about output or productivity. The hundred men with a shovel are, generally, interchangeable labour in the eyes of the employer. If any one of them asks for higher pay, the employer can replace them with somebody untrained who doesn’t make pay demands. If they die of heat exhaustion, there’s thousands waiting in line to replace them. If the dig collapses and they all get buried alive – no worries! One worker is much like the other from the perspective of the employer.
Heavy machinery is different. It requires expertise. A trained operator of an excavator is less interchangeable than the untrained workers. If your excavator operators make demands or exhibit safety concerns, they’re harder to ignore because they’re harder to replace.
And, if they unionise, you pretty much have to listen. Unless, that is you live in a country that has laws biased against collective bargaining and where the police are ready and willing to use violence to suppress it.
This is why it’s very common for employers to act as if their employees don’t have the expertise they do have. They’ll have untrained people working heavy machinery despite the dangers involved. After all, if people die, you can just replace them with somebody else.
(This dynamic has pretty much been a constant throughout the history of labour.)
This is also why unions very commonly fight to have expertise and training recognised.
Productivity improvements on their own do not make labour more interchangeable. They’re more likely to make labour less interchangeable – discourage labour arbitrage – because most productivity tools require more expertise than the tools they’re replacing. This is what makes the tools that get adopted in software development different from your excavators or power drills. The unified component model requires less expertise than the diverse specialities – CSS, rendering, accessibility, direct use of DOM APIs, etc. – it’s replacing. The full stack component model requires even less expertise as server-side coding is additionally glommed into a one-size-fits-all mode of development. Electron and React Native additionally roll up the various different native app specialities into the Katamari ball of shit that is the modern component model.
Readers also asked whether my labour arbitrage argument wasn’t in fact an argument against making code more reusable or reliable.
But, again, code is output. Labour arbitrage is about the worker, not the output. A Test-Driven Development process that requires full unit test code coverage, for example, requires more expertise than not using that process. Test-Driven Development is a skill that takes practice. It’s expertise.
Throughout the history of software development, employers have consistently preferred to fund tools that deskill and attempt to abstract expertise away over tools that genuinely improve productivity and the quality of the output, but also happen to require expertise and skill.
This has worked so far because the software industry is usually flooded with money. The past few decades have been years of explosive growth. Most of the big tech companies are either monopolies or oligopolies and have the easy cash that come with a stranglehold on a market. Interest rates were at record lows.
When money is easy, things don’t have to be well-made.
Things have changed
When the market is no longer growing and funding is harder to come by, we’re all discovering that many of these technologies are actively worse than the alternative by any objective measure.
They are often:
- More expensive.
- Slower.
- Require more bandwidth.
- Lead to interface designs that are harder for the end user. (A native app that is consistent platform conventions will always be more predictable and easy to use to the end user than a native app that isn’t.)
React is, for the vast, vast majority of organisations making web-facing software, objectively worse than many of the alternatives.
React meta-frameworks that try to abstract away platform differences both on the client-side and the server, are never going to outperform dedicated server frameworks or tailored front-end code.
They make hiring easier but they also make it easier for management to fire your entire team and replace you with somebody or something cheaper.
So it doesn’t matter if it’s worse software. It’s good enough to ship because the bar for shipping code is on the floor.
Tech management will sacrifice technological progress – performance, design, and general product effectiveness – if it disempowers labour.
What does this mean to you?
That depends.
If you’re working in free or open source, it might help you understand why some tools and frameworks seem to get endless support and buy-in from the management of various organisations and why some do not – despite being both superior technically and being easier to use by both the developer and end-user.
If you’re trying to get funding for a project, it might help you predict which idea is more likely to be funded and which isn’t. For example, an LLM-based defect discovery tool that is tightly-integrated with both the runtime and its debugger (and so genuinely useful) is less likely to get funding than the umpteenth implementation of a component-spewing copilot.
If you’re unlucky enough to have to use any of this garbage we’re shipping and calling ‘software’, now you know why it all feels a bit shit.
If you work as a software developer, it means employers will continue to emphasise frameworks over functionality because that makes you easier to replace. They will sacrifice software security to make your job easier to outsource. They will let their own businesses suffer by shipping substandard software because they believe they can recoup those losses at your expense.
This is what unions were made for
The only way we can counter this trend is through collective bargaining. Historically, the only path towards forcing employers to require and invest in training and safety, to guarantee job security, prevent mass layoffs, and combat outsourcing has been through concerted effort by workers as a collective.
And, thankfully, there is a book for that:
You Deserve a Tech Union, by Ethan Marcotte.
In this, the latest book from Ethan Marcotte (he of responsive web design fame), unions aren’t anachronisms but rather a set of structures for workers to practice mutual aid, solidarity, and democracy with each other and across their workplaces—practices which are necessary not only for improving working conditions but also for attending to the harms the industry continues to commit.
Maybe as more experience is accumulated in the tech industry Marcotte will deliver such a volume for us, but in the meantime this is a very useful tool and stimulating read for any tech worker contemplating improving their workplace and enhancing their power over their work life and the products they are fabricating.
While acknowledging the depressingly increasing role that digital tech products play in government actions and the military industrial complex, he also provides examples where collective action has been successfully taken by tech workers.
Now, it’s time to formalise these actions, and get organised.
I couldn’t put this book down once I started reading (released on August 15th, 2023). I ended up diving into the multiple references and resources mentioned while reading.
Ethan uses the initial chapters to quickly show how the tech industry is, despite its self-described exceptionalism, yet another site in the long history of capital exploitation. Fortunately, there’s an equally long lineage of worker solidarity actively resisting that exploitation, and that’s why I was so excited about this book.
But what are unions? And why do they matter? Ethan answers these questions in the book through extensive research and by interviewing tech workers with real-world union-building experience. Ethan shares these workers’ insights and stories, weaving them together to outline the process for forming a union of your very own.
The book is informed by almost 50 interviews with economists, labor researchers, and also workers who have formed unions.
You Deserve a Tech Union is exactly the book we need today as an industry. You owe it to yourself to read it.