Disclaimer: until a couple of months ago, I was paid to work full-time on Open-Source Software (OSS) software in the Open Educational Resources (OER) space. There’s a distinct possibility that I’m biased in some way as a result. There’s also the possibility that working in OSS and OER for over four years might mean that my opinions are backed by experience. You get to decide.
The symptom
Babel is used by millions, so why are we running out of money?
This doesn’t surprise me. The purpose of the web software industry is to extract value out of Open-Source Software (OSS). Everything is built under the misconception that OSS is abundant, replaceable, and free.
People don’t appreciate just how much web dev is about extracting value from OSS, both on individual and corporate levels.
Make a native iOS app, and the only OSS you need to use is directly integrated into the OS, and a lot of it is maintained by Apple itself. Apple may use open-source font rendering libraries or SQLite, but you rarely have to deal with it yourself, as a developer, unless you want to.
Web development? Everything is built or run directly on OSS.
Almost everything we do in web development exists as a thin layer over open-source software.
Servers, build tools, databases, ORMs, auth, client-side JS, web browser: we are all building on a vast ocean of OSS labour without paying back a fraction of the value we generate. It isn’t just big, direct dependencies like Babel that are suffering. The stuff your stuff is using—the infrastructure code everything needs—is surviving on sheer inertia as well.
That’s value extraction. Strip-mining if you want to hammer home the unsustainability. Looting if you want to emphasise the moral dimension.
We don’t even know the extent of the problem. Has anybody looked into the sustainability of the npm ecosystem, for example? How much of it is backed by a sustainable business model or even any kind of ongoing revenue?
I suspect that the sustainable fraction is tiny. At some point, something’s going to give. Something’s already giving. We keep seeing projects like Babel: used by thousands; running on fumes.
I’m reminded of the only real tech bubble I’ve personally witnessed: the rise and fall of blogging. They’re very different phenomena, but blogging and open-source are solid examples of big corporations strategically subsidising a complement to their core business with no care being paid to collateral damage.
(I was in university when the dot com bubble collapsed, so I missed that. And I was doing my PhD when Web 2.0 peaked. Somebody needs to write about the true history of the rise and fall of the blog as the dominant web paradigm as a lot of it seems to be forgotten or just plain misunderstood today.)
There are a lot of misconceptions about what exactly happened. People think that Google killed weblogs in 2013 when it discontinued Google Reader. Which isn’t entirely true. Google Reader was a murder weapon, not a victim.
The blogging bubble and its modern parallels
Google did kill weblogs, but that’s fair since they also created the weblog bubble.
You might not believe this if you weren’t there, but one of Google’s main problems, once they got going, was that there just wasn’t that much to see on the web.
Having a great search engine is kind of useless if, for example, somebody types in “how to take better landscape photographs,” and nobody online has an answer.
Google Adwords changed all of that. That, as well as free weblog hosting, fuelled the blogging bubble. You wrote a blog using a weblog system that came with decent SEO baked in (semantic structure and cross-linking, that’s all you needed back then). Most of your traffic came from Google’s search results. All of your revenue came from Google’s Adwords. It became profitable to churn out indistinct pap that passed as informative to fill Google’s search engine results, so people did.
The weblog ecosystem was built entirely around extracting value from Adwords. For a few, it was a springboard to launch something else. A few writing careers got off the ground. But the vast, vast majority was just Adwords. Weblogs as social media? A sideshow. Weblogs as a unique medium? Incidental.
Even outfits with paid subscriptions, like MetaFilter, relied on Adwords and had to course-correct once Google popped that bubble.
Which they did because they had to: most of it was fraudulent. Fake clicks. Spam blogs. Link farms. Black hat SEO. The blogging economy was filled with bad practices all around. People today don’t appreciate just how rampant these practices were. Most of us didn’t notice because we were in our tiny corner, all reading the same few popular bloggers (an early version of the modern ‘influencer’). But outside of that corner, blogs were done for Google and paid for by Google. Outside of a small number of active commenters (many of whom were toxic as hell), the traffic these blogs had existed solely because it suited Google to give blogs a high ‘PageRank’. They had no meaningful community or engagement to call their own.
After a few years of buying into the hype, advertisers started to push back, forcing Google to clean up their index. That consisted of downplaying blogs and blog-like sites and purging spam-blogs and blog farms (many of which had been hosted on Google Blogger, natch).
The blog gold rush ended. The tools surrounding it started dying, hastened by Google sucking the oxygen out of the software ecosystem by acquiring, offering for free, and then putting on life support the core tools in the ecosystem.
FeedBurner and Google Reader were not victims of Google’s policies. They were the weapons Google used to ensure that the only player extracting value from blogging was Google.
Because people misunderstand what happened, they tend to take away the wrong lessons about what worked or didn’t work about blogging. They wrote off blogs as a genre of design: content, a sidebar with blog roll and categories, comments, and a casual style of writing. They ignored the tech that blogs were built on: RSS, easy-to-use CMSes, reverse-chronological listings mixed with categories and the like to highlight posts.
Blogs, by the technical definition, are having a resurgence. Substack is a blogging system with convenient payment and email integration(RSS, reverse-chronological listing). As Jeremy Keith enjoys pointing out, Medium is Ev’s blog and offers RSS feeds and reverse-chronological listings for each guest author. Most mailing list platforms are either technically blogs (RSS plus a pleasant CMS) or integrate with blogs using RSS.
This time around, the audience seems, for the most part, to genuinely be invested in the blogs themselves. They aren’t being funnelled there by Google or Facebook because it’s strategically convenient for them, for the moment. Substack isn’t the only player in town, so we seem to be avoiding the monoculture pitfall. Blogging for an earned audience is probably bigger than it has ever been before in the history of the web.
The blogging bubble reminds me in many ways of the current state of web development:
- We have an economy that exists solely because big tech companies are pouring money into strategic complements to their products.
- This money-hose distorts the economy in ways that kill off businesses and skew education and recruitment.
- Much like the blog farms of old, many companies get away with extractive, unsustainable, and generally awful practices because so many of their resources are in effect subsidised by Google, Facebook, or Microsoft.
- Most web software companies are built to quickly extract value from the underlying open-source layer. Ever wondered why management practices in web software are dominated by short term concerns that are self-destructive in the long term? Because managers know what they are supposed to do, even if they don’t exactly know why. Speed is of the essence because you aren’t doing anything unique. You aren’t creating tech of your own or building on a hard-earned understanding of the clientele. You are using your funding to race against others to see who manages to squeeze the most business out of the underlying OSS before the funding runs out.
The willful blindness
Most people in the industry don’t realise that the web dev economy is primarily extractive. As with the earlier assumption that Google must be trying to keep Adwords clean and sustainable, there’s the assumption here that somebody in tech must be making sure the work behind OSS is paid for. Or, almost as naively, they assume that it can all subsist as donationware.
People in web dev constantly assume that something somewhere must be paying for the OSS dependencies they use. This is why developers come in with the entitlement of a paying customer. The OSS maintainer is supposed to serve them with the enthusiasm of somebody being paid. Their worldview just doesn’t accommodate the idea that this is unpaid labour because then they’d have to start questioning the very sustainability of their own careers.
The money hose
Many of the misconceptions about OSS stem from the fact that the core of the ecosystem is funded.
Facebook is pouring money into the React sub-ecosystem, and Google keeps burning cash on the Chrome bonfire. Microsoft keeps funding developer infrastructure like Visual Studio Code, Typescript, npm, and GitHub. Those are just the most prominent examples. They all fund ton of other projects, either directly or indirectly, through a wage surplus.
A surprising amount of OSS is made by former big tech developers. They can afford to subsist on meagre revenue—for a time—because their pay and stock options have left them free of debt and with well-stocked savings accounts.
This is much more common than you’d think. Scratch away at the surface of pretty much any active OSS project that has no discernible revenue, and you either get a burnout waiting to happen, or you’ll find a formerly well-paid dev coasting on savings. Many of the rest have solid VC funding. Though, VC funding always runs out at some point. The business fundamentals just aren’t there for open source when you have Google, Amazon, and the rest gatekeeping all of the value in the market. This is why the ecosystem is already beginning to pull apart at the seams.
And Microsoft… Like I wrote above, MS is just carpet-bombing the web developer community with open source software and OSS infrastructure. Typescript, Visual Studio Code, GitHub, npm, and so much more exist primarily because Microsoft executives believe this will lead to more business for Azure and other Microsoft offerings. In turn, most of the offerings on Azure (as with Amazon Web Services) are leveraging OSS to maximise the value of the platform.
Most of the rest of the ecosystem—excluding the tiny fraction like SQLite who have few dependencies and sustainable revenue—exists as backsplash from those two projects. Node, Deno, Electron, most front end tooling, and so many web dev libraries only exist because of projects that Google and Facebook are funding.
Google, Facebook, and Microsoft are probably the biggest players using OSS this way, but they are not alone. Leveraging the positive externalities of OSS has been the go-to strategy for many big tech corporations for a while now.
It isn’t just software: web dev education, training, and recruitment exist primarily to extract value from Facebook’s React or Google’s OSS projects. Very few of them invest in figuring out what sort of training will serve their students the best. The easiest thing to sell to both recruiters and students is the big framework on the block, so that’s what they sell and very little else.
The extraction mentality is baked into the business. Which is sort of fine when you’re dealing with projects funded by mega-corporations but disastrous when applied to the unfunded or poorly funded rest.
The money hose, combined with free or subsidised services, is a control mechanism that lets big tech companies control the OSS ecosystem. Projects they want to promote will get the money spigot. Other projects, like MongoDB or Redis, get turned into commodities and resold as cheap services.
Most notably, projects that try to independently leverage open source to become fast-growing tech companies are disproportionally likely to be ‘embraced and extended’ this way by AWS, Google Cloud, or Microsoft’s Azure.
The rest of us aren’t much better. The SaaS companies that funnel a bit of their revenue back to the OSS projects they’re building on are few and far between. The companies that maintain an OSS framework or library as a positive externality to their business are few and far between. VC-funded OSS companies are moonshots that tend to crash well before they get an opportunity to blow up in orbit.
We’re honestly bloody bad at paying for the work of making software as an industry.
The deterioration and potential collapse
Then as now, every stepwise deterioration in the ecosystem is explained away. Back when the primacy of blogs was sliding, every time Google changed their algorithm to downplay blogs or updated their Adwords policies, you got thousands of responses ‘explaining’ how this wasn’t the end of the world and here’s what you need to change to keep up.
Meanwhile, legitimate blogs started dropping off the web, services began to shutter, and RSS feeds went dark.
We’re starting to see the initial decay hit the parts of the web dev ecosystem that are the furthest away from the cheap money fountains Google and Facebook are providing. Core projects run out of money. Git commits stop. A dependency you use breaks when one of its dependencies stops working, leading somebody to fork it with a quick fix or replacement dependency. Bandaid fixes to decaying OSS projects start to crop up in more and more places. We start to see blog posts saying that all we need to do is get enough people to donate money or pay for support. Everything will be fine. Just look at how OpenSSL got turned around.
All of which is bad enough but also misses the point.
Weblogs were a strategic lever for Google. Funding them made sense at a time where they needed the web to have more content. When they didn’t need them—when they stopped being a profitable lever—they were discarded. And the ecosystem sunk fast.
Open-source software is a strategic lever for big tech companies. They fund when it helps their core business and strip-mine projects when it doesn’t. Cloud hosting has, slowly, been ushering in an era of extraction, where tech companies specifically target serverside OSS projects that they can leverage with little investment. Broad sections of serverside software are woefully underfinanced. Many server frameworks in the node ecosystem, for example, coast along on burned-out maintainers and sheer bloody-mindedness.
Chrome and React are strategic levers for Google and Facebook. Electron, GitHub, Visual Studio Code, TypeScript, and npm are all potential strategic levers for Microsoft. V8, npm, React, Visual Studio Code, and Github: they are the foundation of modern web development.
How confident are you that all of these projects will remain strategic for the life of the web? Losing any one of them would knock the entire software economy to the ground. Are we so sure that nothings going to change for these companies?
Facebook as a business is, in my view, harmful for society in general.
Which used to mark me as a contrary weirdo but is now the political consensus in many countries.
Apple’s pro-privacy stance is manoeuvring Chrome into direct conflict with other parts of the Google behemoth. To remain competitive as a browser, Chrome must work against the interests of its owner.
You can argue whether there is any substance behind Apple’s privacy efforts or not. But it’s a highly effective strategy for striking at Google’s longstanding core strategy: using free products and OSS to bolster their web-based ad business.
Microsoft is still in the progress of figuring out which of its acquired assets directly benefit its core businesses and which don’t. Their track record of embracing and extending and ruthless competition doesn’t inspire trust.
We’re still going to see a tonne of projects burn out and die out. Diversification will mostly be a question of which flavour of V8 and what flavour of React-like front end framework you’re using. Everything else would slowly become a barren wasteland. But overall? It would survive. A bit boring but still alive. Microsoft’s OSS endeavours might suffer as npm and GitHub more and more become homes for dead and broken software, but the tools should still work, right?
But…
What happens if Facebook’s fortunes change? Or if its reputation becomes so bad that React ceases to be a good recruitment tool. Or, if the web dev training ecosystem shifts away from React because most of the time, it isn’t the right tool for the job unless you’re Facebook? What if Facebook’s management decides they can get most of the benefit they want from React without pouring money into React? Let the community coast on voluntary contributions for a while?
Do we want to risk becoming a web development ecosystem that primarily exists as a subsidised complement to Microsoft’s Azure and, maybe, GitHub? (That’s assuming GitHub is a viable business in its own right.)
Are we all okay with the collateral damage of burning out the OSS ecosystem?
Strategy isn’t permanent. It’s always a response to needs and to the surrounding landscape. If needs change or if the landscape changes, then the strategy has to change as well. It’s a cliche to say this, but change is constant.
If we want software development to last, then we need to work on our attitudes towards open-source and reconsider our reliance on software that, at the moment, happens to be strategically relevant to big tech.
The price we’re paying
Despite everything I wrote above, this apocalyptic vision of potential collapse isn’t what worries me. The biggest problem I have with the status quo isn’t that it might be unstable.
What we learned from blogging is that we can rebuild. If the tech giants shift their attention away and the bubble bursts, the skills and knowledge will still be there. And rebuilding is often an opportunity to fix things.
So, instability doesn’t worry me. It’s just an inevitable part of working in an immature medium, dominated by immature mega-corporations.
The biggest problem—and this isn’t limited to web development—is how it has baked exploitation into the core worldview of so many people. We use open-source software. We get paid to use open-source software. Our employers benefit, but the money never trickles down—money never trickles down. This is fine when the project in question is directly funded by a tech multinational. Less so when the project is something specialised, a little bit niche, or inventive, and therefore not financed by a gigantic corporation.
We train each other with a mindset to treat each other poorly and consider the people—whose free work we are turning into money—to be disposable.
We’re working in an unstable industry, surviving at the whim of callous mega-corporations, and we spend our days treating people like shit.
I don’t feel proud when I look around at the web developer community and economy. More the other thing.
There are a few bright spots. Interesting companies that build a lot of OSS while flying sustainably under the radar. Turns out you don’t need that many people to maintain open-source if you set strict boundaries and make sure all of the work is paid for.
Then you have companies like WordPress who have been around forever and form the backbone of the entire web.
Sustainable open-source looks possible if you manage to balance looking uninteresting to big tech but interesting enough to generate revenue. As WordPress shows, you can get pretty big while remaining mostly uninteresting to the tech behemoths.
It’s a tricky needle to thread but seems to be entirely possible.
The biggest threat, what kills off project after project, is scope creep (a universal problem for software development) and other developers who feel entitled to your time and work.
Twenty years ago, when I was doing my postgraduate studies at the University of the West of England, one of my mentors was Bob Hughes. He wrote the book Dust or Magic, an idiosyncratic study of what makes great interactive media great. It’s a lovely book that’s hard to find these days, and it led him to hold a couple of Dust or Magic conferences in Oxford over a decade ago. Those two conferences are, to this day, the most engaging and inspiring conferences I’ve ever attended.
He has done quite a few things since. Some activism. A bit of writing. Like an excellent book on the toxic nature of modern tech. (Highly recommended.)
One day, when I was still doing my Masters (ca. 2000), I ran into him in the courtyard at UWE’s Bower Ashton campus. I enthusiastically began to subject him to my latest enthusiasm as I’m wont to do. I spoke about this great thing called Free Software and how it would fundamentally change our relationship with technology: make tech free and fair. Free Software was going to make software a force for equality and progress!
—While I’ve never been bright-eyed or bushy-tailed by nature, I’ve always had a strong tendency towards idealism.
Bob patiently listened to me, gave me the time to make my case and argue my point before sighing.
Punctuation can’t really capture the way Bob speaks. The pauses and hesitations make his arguments feel like he’s leading you down a winding path, even though, at the end of it, you look back and see that you just went down a straight, narrow, and unmissable road.
“Capitalism will always find a way to exploit common resources. It’s just a matter of time.”
‘Loot’ is a word that he used a lot at the time. It was the primary theme of the closing talk Bob held at the last Dust or Magic conference he led.
Our world is built on looting what others have to give—what we make them give.
And we’re fine with it.
We’re all fine with looting.