On the Consolidation of the Web Created: Wed 29 Jul 2020 · Updated: Wed 07 Oct 2020

In recent years, the web has been consolidating. From the servers to the apps, a growing majority of the web is controlled by a small pool of companies. When AWS was founded in 2006 I was just starting out with my first VPS, running this blog on WordPress (the good ol’ days!). For the last 10 years I have part-run a small VPS (“cloud server”) host called Afterburst. Throughout these years I have watched this consolidation, and these are my observations.

The Pre-Cloud Days

Way back in 2009 blogs were booming. The market for personal servers was growing rapidly. People often used forums (remember those?) to find providers. Providers would compete to attract the most eyeballs to their sale posts.

The quality and cost of hosting varied wildly. During summer there would be a huge influx of budget “summer hosts” during school holidays. The majority of these would then fold only two months later. Thinking back, it was The While West. Of course there were big players; but the smaller hosts had the cheapest offers and captured the market for personal servers.

I believe that this was a great market for all. Buyers had a wealth of choice of companies tiny to massive. Providers were kept in check by thriving forum communities, leading to better services. The smaller providers would offer a personal touch, often partaking in forums alongside their customers. To me, this was an amazing environment in which I learnt a huge amount about servers but also customer service.

The Clouds Ascend

The VPS market was exploding when we started Afterburst in 2010. Shared hosting/PHP was stagnating and the prices had bottomed out. Shared hosting was consolidating fast, hosts were folding daily. Dedicated server and VPS markets remained strong and WebHostingTalk (our “home” forum) was buzzing with activity. The competition for the best VPS was in full swing.

And then came DigitalOcean.

When DO arrived in 2011 everything changed. They managed to make the much hyped “cloud” accessible to everyone where AWS had so far struggled. You could click a button and have a cloud server available within minutes. The all-SSD package, “cloud” marketing and a wave of free launch coupons caused them to explode onto the scene.

It was fascinating to watch the “cloud” hype train. “Cloud servers” were, almost overnight, seen as superior to “VPS”. This is despite most “cloud server” providers offering nothing different to VPS. Now I totally get that The Cloud goes way beyond servers. The offerings today include a staggering number of services. But an individual looking for a server to host their blog? They don’t need any of that, just the server space.

In the years since DO arrived they, AWS and later Azure/GCloud boomed. Cloud was/is the future - we must move everything “to the cloud” many a huge tech company would say. As much as it was marketing hype, the individual started to follow. The cloud was only a little more expensive and came with fancy UI and excellent developer tooling. It was cool to be using the cloud. The small/traditional VPS provider market began to slow, and later reduce. The golden days were over.

Whilst this was very frustrating at the time it also forced us to review and improve our marketing and customer experience. We started marketing cloud servers and optimised the checkout and customer sign-up flow. The competition led to an improved service for existing and new customers.

So - where are we now?

10 years later, we’re still here! The consolidation of providers has slowed and many continue to survive. Forums like WHT struggle on but are shadows of their former selves. There’s still a market for individual servers - people have a natural desire to tinker in ways that specific services cannot provide.

I think there will remain a cohort of individual bloggers and websites. But I also believe the web is dividing. On one side a small number of platforms the vast majority of “normal” people consume from and share to. And elsewhere a separate “old style” web of fragmented loosely connected websites/forums/blogs formed by those who tinker. Perhaps something will merge the two together in the future.

It’s like supermarkets consuming ‘Mom and Pop shops’, a trend that goes back decades now. Yet small greengrocers and butchers still live on. There are signs of people returning to these shops in growing numbers. Perhaps this is the start of a reverse trend, could the web follow suit? I’d like to think so.