Collectively speaking humans have always been terrible at programming. Let's look at some real examples
We panic about AI writing bad code, but after twenty years in this industry, I’ve come to an uncomfortable conclusion: the majority of software engineers aren't just below average, they are objectively terrible programmers. Human-written code has crashed markets, destroyed rockets, and literally cost lives.
Welcome to Lead Prompt // executing leadership from the root. I’m your host, John Collins.
Every week, my teams push several releases to production of various applications for our large enterprise. And every other week, some feature that worked with the previous version is now broken with the new version, a common anti-pattern called a regression, where the state of a previously working feature has regressed to a broken state.
The accepted solution for this is to increase unit test coverage on your application codes, which are little automated tests that run against your application each time to build it, that are designed to capture these regression failures before they reach your production deployment. In principal this is a great idea, but in practice it often fails for two main reasons.
- The amount of your application code covered by the unit tests needs to be high, otherwise regression issues fall through the gaps of low test coverage.
- Unit tests are still largely being written by humans in enterprises, and as these tests are code they also fall victim to my central thesis that humans are terrible at programming.
To strengthen my case, rather than providing my own anecdotal evidence let's look at some infamous examples of human programming errors from the past, that each had devastating impacts.
1. The Therac-25 Radiation Machine (1985–1987)
In the 1980s, the Therac-25 was a state-of-the-art medical accelerator used for radiation therapy. Unlike previous models, its safety relied almost entirely on software rather than hardware mechanisms. Due to a race condition and integer overflow errors in the code, a fast-typing operator could accidentally set the machine to deliver a massive, un-shielded electron beam instead of a low-dose X-ray. Because the software hadn't been properly tested for these edge cases, patients received up to 250 times the normal dose of radiation. This resulted in at least six severe accidents and three patient deaths, marking it as one of the most tragic software failures in history.
2. The Patriot Missile Failure at Dhahran (1991)
During the Gulf War in 1991, an American Patriot missile battery failed to intercept an incoming Iraqi Scud missile, which tragically hit an Army barracks, killing 28 soldiers and injuring nearly 100 others. The root cause was a precision problem in the software's time calculation. The system's internal clock measured time in tenths of a second, but converted it to a 24-bit floating-point number for tracking calculations. Because 1/10 cannot be perfectly represented in binary, a tiny rounding error accumulated over time. After operating continuously for 100 hours, that small error compounded into a 0.34-second delay. At the speed a Scud missile travels, that fraction of a second threw the radar's tracking off by over half a kilometer, causing the system to completely miss the target.
3. Ariane 5 Flight 501 (1996)
On June 4, 1996, the maiden flight of the European Space Agency's Ariane 5 rocket ended in a massive explosion just 40 seconds after liftoff. The culprit? A single line of code. The guidance software attempted to convert a 64-bit floating-point number that represented the rocket's horizontal velocity into a 16-bit signed integer. The velocity was far greater than what a 16-bit integer could hold, resulting in an integer overflow. The guidance computer crashed, passing diagnostic data to the flight control system, which interpreted it as a sudden change in direction. The rocket violently over-corrected, aerodynamic forces ripped it apart, and the self-destruct mechanism was triggered. Over $370 million went up in smoke because programmers reused code from the slower Ariane 4 without updating the mathematical constraints.
4. Mars Climate Orbiter (1999)
In 1999, NASA lost the $125 million Mars Climate Orbiter simply because of a failure to agree on units of measurement. The spacecraft was supposed to serve as an atmospheric research tool and communications relay. However, the engineering team at Lockheed Martin produced software that calculated thrust using imperial units (pound-seconds), while the team at NASA's Jet Propulsion Laboratory assumed the data was in metric units (newton-seconds). This mismatch caused the navigation software to miscalculate the probe's altitude. When the orbiter attempted to enter Mars' orbit, it flew dangerously close to the planet and completely disintegrated in the Martian atmosphere.
5. Knight Capital Group's $440 Million Glitch (2012)
In a stunning example of deployment error, Knight Capital Group, a prominent Wall Street trading firm, lost $440 million in just 45 minutes on August 1, 2012. Engineers were manually deploying a new software update across eight servers but missed one. The missed server still had an old, dormant testing algorithm on it. When the markets opened, the new system reused a software flag that inadvertently triggered this old code on the unpatched server. Lacking any safety limits, the rogue code relentlessly bought stocks high and sold them low on every incoming order. It executed over 4 million trades in less than an hour, burning through roughly $10 million a minute and nearly bankrupting the entire company.
These examples prove that the problem isn't new. Whether we are launching rockets, trading stocks, or treating patients, placing blind faith in human-written code without rigorous, comprehensive testing processes has historically been a recipe for disaster.
In my social media feeds and within my own teams, I frequently see software engineers expressing doubt about the coding abilities of AIs and their inclination to hallucinate, and I get it those concerns are real: but can we honestly say that human programmers are much better?
We have all worked with a "10x programmers" at some point in our careers if we have been around long enough, but we remember them and tell stories about how skilled they were because they were the exception, not the norm, and they stand out in our memories for that reason.
Talk to you next time.
References
- The Therac-25 Radiation Machine: Leveson, N. G., & Turner, C. S. (1993). "An Investigation of the Therac-25 Accidents." Computer, 26(7), 18-41.
- The Patriot Missile Failure: U.S. General Accounting Office (GAO). (1992). Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia (Report No. IMTEC-92-26).
- Ariane 5 Flight 501: Lions, J. L., et al. (1996). Ariane 501 Inquiry Board Report. European Space Agency.
- Mars Climate Orbiter: Stephenson, A. G., et al. (1999). Mars Climate Orbiter Mishap Investigation Board Phase I Report. NASA.
- Knight Capital Group: U.S. Securities and Exchange Commission (SEC). (2013). SEC Charges Knight Capital With Violations of Market Access Rule (Release No. 70694).
Download audio
File details: 9.9 MB MP3, 7 mins 33 secs duration.
Title music is "Apparent Solution" by Brendon Moeller, licensed via www.epidemicsound.com
