The Pendulum

My Uncle was a data analyst for United Airlines from the late 1960s to the late 2000s. In 2011, following Apple's rebrand of MobileMe to iCloud, we started talking about the cloud. "It's like a pendulum" he said, "I remember having to access the mainframe with terminals. Then personal computers got powerful enough that everyone had their own storage and compute. Now we're going distributed again. Back and forth."

I have thought about this conversation for fifteen years. For the pendulum theory to hold, we would need to reach a zenith of cloud capabilities and some driving force would need to arise to push society back towards a more localized solution. As with many things, the solution that more localized technology needs to solve is, in fact, the problems with the cloud, which was itself a solution meant to solve an entirely different problem. Let's review.

Computers started intensely locally: one machine with one operator. In the 1950s and early 60's, you'd hand your punch card program to an operator, it'd land in stack of other punch card programs, and you'd wait until the operator got to your punch card in the stack and usually receive your output on a print out. It took forever. Little mistakes meant redoing your whole punch card program.

As processors got more sophisticated though they began to outpace the speed that the operator could load the punch cards. Rather than isolating to one operator, users could be distributed across screen and keyboard terminals that accessed that compute. At imperceptible intervals, the mainframe would cycle between the different terminals: you might get milliseconds of compute, then the processor would shift and give someone else milliseconds of compute and so on, but to you, the result was near instantaneous. The speed to run programs was no longer bottle-necked by a technician manning a single computer to run your punch card.

Mainframes with terminals eventually run into the problem that every communal situation does: Jeff*. When Jeff would run his programs, he'd hog more than his share of mainframe time, and everyone else would be stuck waiting for Jeff's massive programs to finish. As the technology got sufficiently smaller and more advanced, the personal computer arose. Now you could have your own computing resources without having to share with Jeff.

As computers went personal and local, so too did data. Connecting to the company server could be finicky, your personal computer had its own file system, so sometimes, you just saved stuff to your own machine. Likewise, computers moved home for the first time ever. The one computer Linda had to manage her salon didn't connect to a server for storage. It was just Linda's one Macintosh.

In the age of personal computing, computers got smaller and smaller. By the late 2000's everyone wanted a computer that would fit in their pocket. The problem with a computer that can fit in your pocket is somewhat obvious: Jeff* can drop a pocket sized computer into the toilet, at which point you lose all the data that's on your pocket computer. Less violently, there was a technical challenge associated with shrinking the power of a personal computer into a pocket computer, so we needed creative solutions to access compute and storage on a device that had neither.

The solution was to put the files for both your personal computer and your pocket computer somewhere else - a magical place that never loses power or connectivity and is always there for you when you need it - Virginia. We built data centers everywhere and instead of having all of our stuff locally, we rented data center space so that all our stuff was everywhere.

There are two different problems that have arisen from this model in recent years. First, Jeff* built the data centers in Virginia and became fantastically wealthy by renting the space to you. As billionaires tend to do, he got into weird sex stuff and now nobody trusts him, in spite of the fact that Jeff* still has everybody's stuff in his data centers. Second, a new technology came along that appears more capable if it knows everything about you. Since Jeff* already knows everything about you, he's trying to sell you that software on the same rental model that made him fantastically wealthy. The populace can't hold the two thoughts currently: that Jeff* is an untrustworthy guy who likes weird sex stuff and that we need to trust untrustworthy Jeff* with all of our most private information to access this software.

The writing is on the wall: the first company that can give you this fancy new software in such a way where you don't have to trust Jeff* or anyone else will win. You do that by making it local. The launches of Meta's Muse and OpenAI's Dots in the last couple weeks ironically reveal why this is the right approach: Muse runs in a Linux Virtual Machine with 8 GB of RAM. All of us have machines with at least 8 GB of RAM that can run this stuff in an environment where we don't have to give Mark Zuckerburg all of our credit card information.

I'd call the Pendulum Theory a socioeconomic theory because its how humans are interacting with their technology. The transition from me caring a lot about what was on my external hard drive in 2008 to not migrating files when I bought a new computer in 2018 is has more to do with what was cheap and common than what was technically feasible. The technology to move files over the internet or over physical storage existed at both points in that timeline. The question was more what was easiest.

The actual technical theory here is Moore's Law, which states that the number of transistors in an integrated circuit doubles every two years with minimal cost increase. The more transistors, the more compute we have. The more compute we have, the more novel ideas we come up with on how to use it. Sometimes brand new sophisticated ideas need a computer the size of a building to run, but inevitably, the circuits get more powerful, the software gets optimized, and ideas that once needed building-sized computers make their way to our pocket computers. Likewise, sometimes we decide we need an even smaller computer (i.e. glasses) and we find imperfect solutions to make that smaller computer work until Moore's Law makes the actual product idea possible. This push and pull between compute-heavy ideas and product form factor ideas has pushed the pendulum.

Folks are intuitively opposed to new data center build out for AI. Even if someone doesn't know Moore's Law, we have seen things get better and faster our whole lives enough to know intuitively that building-sized data centers will be rendered obsolete by time. Stir in that nobody trusts Jeff* and you've got a perfect recipe for opposition.

You can already run the statistical inference we call "artificial intelligence" locally on your machine. All of the Chinese models are built this way. The frontier labs all release smaller models that can run on consumer grade hardware free of charge. These models will only get more capable over time.

The future I see is not tithing to OpenAI to use their cloud models every month. Instead of sending everything about me to a Sam Altman's cloud, I've got an application harness for a local model on my own hardware. If I was the one releasing such an application, I'd give it the highest third-party security audit there is to prove to the public that it is theirs and that nothing comes from the application back to the software provider's servers. Within this application, my model can have all my private data, my preferences, and can act on my behalf. As the marketplaces of the world move towards surveillance pricing, I can train my model to outwit and circumvent their schemes. My model can defend against targeted new cyber security threats. I can know that my model is working on my behalf because there is no structural way for it to phone home and have another incentive.

This is not to say there won't be issues. The scourge we used to call Search Engine Optimization will shift to trying to feed your model instructions that corrupt its preferences toward their schemes. Stir in that this theory is functionally giving everyone a gun and hoping we don't all hack each other into oblivion. The law will need to step in to address the new and novel problems this raises.

Little, however, will convince me that a concentrated group of powerful men holding all the power is better than each of us having a little bit of power. Here's to the new age of local compute.

*This is a creative work. Any similarity to actual persons named Jeff, living or dead, or to actual events involving a person or persons named Jeff is purely coincidental.