My Disenchantment with Software

My Disenchantment with Software

I've been programming for 15 years now. But lately, during development, it has become unacceptable to think about efficiency, simplicity, and excellence: to the point where it makes me sad about my career and the IT industry in general.

For comparison, modern cars operate at roughly 98% of the physical limits of their current engine design. Modern architecture uses precisely calculated amounts of material to fulfill its function and remain safe under given conditions. All airplanes have converged on an optimal size/shape/payload and mostly look the same.

Only in software is it considered normal for a program to run at 1% or even 0.01% of its possible performance. Nobody seems to object. People even take pride in how inefficient their software is, saying things like "why bother, computers are fast enough":

@tveastman: I run a Python program every day, it takes 1.5 seconds to execute. I spent six hours rewriting it in Rust, now it runs in 0.06 seconds.

You've probably heard the mantra: "A programmer's time is more expensive than a computer's time." This means we waste computer time on an unprecedented scale. Would you buy a car that burns 100 liters per 100 kilometers? How about 1,000 liters? With computers, this happens all the time.

Everything Is Unbearably Slow

Look around: our portable computers are thousands of times more powerful than those that took a man to the Moon. Yet every other website can't maintain a smooth 60 FPS page scroll on the latest top-of-the-line MacBook Pro. I can comfortably play games, watch 4K video, but I can't scroll web pages! Is this normal?

Google's Inbox email app running in Google's own Chrome browser takes 13 seconds to open a medium-sized email:

It even animates empty white forms instead of showing their content, because that's the only way to animate anything on a web page with decent performance. Not 60 FPS, mind you, but rather "as fast as possible on this page." I can't wait to see what the web community comes up with when 120 Hz displays go mainstream. They can barely handle 60 Hz.

Windows 10 updates take 30 minutes. What can possibly be happening for that long? That's enough time to completely format my SSD, download a fresh build, and install it about 5 times in a row.

Pavel Fatin: Typing in an editor is a relatively simple process, so even a 286 could provide a fairly smooth typing experience.

Modern text editors have greater latency than 42-year-old Emacs. Text editors! What could be simpler? With each keypress, you only need to update a tiny rectangular area on screen, and modern text editors can't do it in 16 ms. That's a lot of time. A LOT. A 3D game fills the screen with hundreds of thousands (!!!) of polygons in those same 16 ms, while also processing input, recalculating the world, and dynamically loading/unloading resources. How is this possible?

The trend is that software isn't getting faster or more functional. We get faster hardware on which software with the same features runs slower than before. Everything runs far below maximum speed. Ever wonder why your phone takes 30 to 60 seconds to boot? Why can't it boot in, say, one second? There are no physical limitations here. Personally, I'd love that. I want developers to push the limits, using every bit for performance.

Everything Is ENORMOUS

And then there's the bloat. Web applications could open ten times faster if you simply blocked ads. Google is begging everyone to stop the sluggishness with its AMP initiative — a technical solution that doesn't require any technology, just a little common sense. If you strip out the bloat, the internet would run at insane speeds. Is that really so hard to understand?

Android without any apps takes up nearly 6 GB. Just pause for a second and think about how obscenely huge that number is. What's in there, HD movies? I think it's mostly code: the kernel, drivers. Plus some resources, of course, but they can't be that big. How many drivers do you need for a phone?

Windows 95 was 30 MB. Today we have web pages heavier than that OS! Windows 10 is already 4 GB — 133 times larger. But is it 133 times better? I mean, functionally they're practically the same. Sure, we got Cortana, but I doubt she weighs 3,970 MB. But fine, that's Windows 10 — does Android really need to be even one and a half times bigger?

The Google Keyboard app casually consumes 150 MB. This program draws 30 keys on a screen — is it really five times more complex than all of Windows 95? The Google app, which is basically just a wrapper for Google Web Search, takes 350 MB! Google Play Services, which I don't use (I don't buy books, music, or video there) — 300 MB just sitting there, and you can't delete them.

After installing all the essential apps (social media, messengers, maps, ride-hailing, banking, etc.), I'm left with just 1 gigabyte for photos. And that's without any games or music! Remember when the OS, apps, and all your data fit on a floppy disk?

Your note-taking app is probably written in Electron and thus ships with an Xbox 360 controller driver, can render 3D graphics, play audio, and take photos with a webcam.

Plain text chat was always known for speed and low memory consumption. So Slack is an example of a very resource-intensive application. I mean, chat and a text editor are the most basic things — they should consume the fewest resources. Welcome to 2018.

You might say they at least work. But size growth doesn't mean improvement. It means someone lost control. We no longer know what's happening. Size growth means increased complexity, reduced performance, and reduced reliability. This isn't normal and shouldn't be considered normal. Bloated size should be an immediate red flag — and you should stay far away from it.

Everything Rots

A 16 GB Android phone was perfectly fine three years ago. Today, under Android 8.1, it can barely function because every app has at least doubled in size for no apparent reason. There are no additional features. They didn't get faster and the look didn't change. They just... bloated?

The iPhone 4s launched with iOS 5, but can barely run iOS 9. And it's not because iOS 9 is so much better — the system has mostly stayed the same. But the new hardware is faster, so they made the software slower. Don't worry — you got exciting new capabilities like... running the same apps at the same speed! I think.

iOS 11 dropped support for 32-bit apps. This means if the developer isn't willing to go back and update the app, chances are you'll never see that great program again.

A DOS program can be made to work unchanged on virtually any computer made since the '80s. A JavaScript application might break because of tomorrow's Chrome update.

Today's web pages won't work in any browser in 10 years (maybe sooner).

"You have to run as fast as you can just to stay in the same place." But what's the point? I can keep buying new phones and laptops like everyone else, but doing so just to be able to run the same apps that have only gotten slower?

I think we can and should fix this. Right now, everyone is developing software for today, occasionally for tomorrow. But it would be nice to make things that work a bit longer.

Nobody Understands Anything

Right now, nobody understands anything. And nobody wants to. We just release half-baked nonsense, hope for the best, and call it "startup common sense."

Web pages ask you to refresh when errors occur. Who cares? JavaScript applications continuously generate random errors on compatible browsers.

The entire web database architecture is built on the premise (hope, really) that nobody will modify data while you're looking at an open webpage.

Collaborative applications lose data. Have you ever seen a dialog that says "Which version would you like to keep?" I mean, the app has full information about document changes — it could merge them itself. But no, they'd rather dump the problem on the user.

Linux intentionally kills random processes. And yet it's the most popular server OS.

Devices regularly malfunction. My Dell monitor requires a hardware reset because its built-in software crashes. AirDrop fails to detect devices. Bluetooth is so complex in its specifications that periodic reboots are considered the "optimal" solution.

And that's all just the tip of the iceberg. It's just a pile of barely working code, piled on top of previously written barely working code.

Development Chaos

Build systems are inherently unreliable. They require full cleanup despite having the invalidation information needed to work correctly.

It seems that rm -rf node_modules is an integral part of the workflow in Node.js/JavaScript projects.

NPM exists in an "it sometimes works" state. Nothing could be a stronger condemnation of the software industry.

Compilation times stretch to minutes or hours. Despite having fast hardware, we somehow managed to make builds slower than they were decades ago.

Docker, virtual machines, and containerization solve problems caused by chaotic development environments rather than fixing root causes.

The single binary remains a HUGE advantage of Go. No mess == success.

Dependencies sprawl uncontrollably. Complex packages are added thoughtlessly, creating dependency trees that are enormous and riddled with conflicts. Programs cannot run for years without reboots. Some require restarting every few days.

It Doesn't Have to Be This Way

There are bright spots. Individuals who show a better path.

Martin Thompson's work on LMAX Disruptor proves that you can process millions of transactions per second in Java, as long as you understand what you're actually doing. Raph Levien's Xi editor demonstrates that a modern text editor can achieve minimal latency. Jonathan Blow's compiler processes 500,000 lines per second in cold compilation — that's the entire project in one second.

These are ordinary people, not wizards. They just care about their craft.

Manifesto

I want to see progress. I want change. I want a better state of the art in software engineering. I don't want to keep reinventing the same thing over and over, each time worse than before. I want something to believe in — a worthy goal, a future better than what we have today, and a community of engineers who share this vision.

The current situation is total crap. As engineers, we can and must do better. We can have better tools, we can build better applications — faster, more predictable, more reliable, using orders of magnitude fewer resources. We need to deeply understand what we're doing and why. We should take pride in our work. Not "I completed my mass of tickets and mass of stories," but actual pride: "I wrote software that people are delighted to use." Self-respect and professionalism in our work.

I hope I'm not alone. I hope there are people who want the same thing. I'd like to at least believe it's possible.