The “Shoemaker's Children” Problem
Software has a problem.
OK, it has many problems. I've already highlighted one of them. But this is another important one.
The problem is that software—all software, with no exceptions—sucks. The reason for this is multifaceted and we could spend years and years arguing about who has the larger list of reasons, but in the end it boils down to the proverbial shoemaker's children: Our development tools are the worst of the worst in software.
The problem in a single question …
How are we meant to make good software when the software we rely on to make our software is such utter shit? Why do we tolerate such shit tooling?
Picture a bicycle repair shop. Go ahead. Get that picture in your mind. Now picture the person in the shop working with, say, wrenches that are made of such soft and malleable steel that they routinely get stripped or bent out of shape. What do you think the response would be? (Hint: it rhymes with “they'd get fed up and toss them out before buying new ones that work”. Because identical rhymes are rhymes.)
Picture a carpenter. Picture said carpenter working with, say, chisels that can't hold an edge for longer than four or five taps. How long until that carpenter throws them out (adding a lot of salty language to the bargain) and buys a set that doesn't suck?
Picture any tool-using profession. Picture how any such tool-using profession deals with tools that are simply bad.
Now look around you in the programming world and ask yourself “what the flying fuck happened to us!?”
The state of software tooling
This is a question that should be asked because software tooling is utterly fucking terrible. For example, our problem domains have ballooned in size and complexity exponentially, yet we still chiefly use products that are literally a single dimension line of text. I mean sure, we try to make it look at least two dimensional like this:
switch (foo)
{
case bar:
do_something_with(bar);
break;
case baz:
do_something_with(baz);
break;
default:
do_something_else();
break;
};
But the tragic secret here is that this is all just formatting convention. It's literally identical to this when the compiler sees things:
switch (foo) { case bar: do_something_with(bar); break; case baz: do_something_with(baz); break; default: do_something_else(); break; };
There's something fundamentally broken in how we try to map these pathetic single-dimensional strings into a world that has concurrency, parallelism, distributed computation, and a whole host of other horrifically complicated, multi-dimensional logic.
N.B. Before you proudly trot out your language that isn't just a one-dimensional string, answer two questions:
- ARE YOU SURE? For example Python is not, despite its attempt to force that “structured” style above.
- How common is your favourite tool if it really is? (Hint: not very.)
But this goes beyond tooling choice. It's frustrating. It's difficult. It's error-prone. But it is possible to make software that isn't fundamentally broken, even with the tooling we've got. There's more to it than this.
Programmer attitude
This is the real culprit. It's the shoemaker's children problem only it's not the children who don't have shoes. It's the shoemakers themselves. The shoemakers are so intent on making (slipshod) shoes for other people (because that's how they get their money) that they don't realize they're barefoot themselves.
While crouching on a floor consisting of sharp flint shards.
They don't understand that if they only had, say, a good workbench, some nice hammers and nails, some high quality leather, and even (gasp!) a chair, they could make nicer shoes that don't make people want to throw them at the shoemakers' heads.
A case study
I use microcontrollers from STMicroelectronics (the STM32 line) in my day-to-day work. One thing that ST likes to push is their STM32CubeMX code generator. It generates startup code, peripheral configuration code, and even, lately, code for specific external peripherals and middlewares. They recently added Azure RTOS support to CubeMX for select processors, and in December-ish time frames they expanded it to cover almost their entire line.
Including their STM32WL line of LoRa-enabled MCUs.
Let's take a look at the Azure pack for STM32WL in STM32CubeMX, shall we?
First, here's my version of STM32CubeMX:
Version is 6.4.0. Now let's check ST's site for what the latest version is:
Yep. I have the latest version. Now let's go install the Azure RTOS pack for STM32WL:
WTF!? It really wants a version that doesn't exist! This must have just been put out and v6.5.0 is due to be released soon, right? When was this pack put up?
… What the actual…? This has been sitting out there, unusable with any released version of STM32CubeMX for almost two months! WHY!?
Why this happened
Well the reason why is clear: in modern business thought, work on the STM32Cube infrastructure is not considered a benefit. It costs money to make the firmware (STM32Cube HAL and LL drivers), the code generator (STM32CubeMX), and the various packs and they don't sell these. There's no direct income from them, so they're loss items on the balance sheet. You don't direct engineering, marketing, and quality control resources to things that don't directly earn you money, so the quality languishes and development is slow and shoddy. (When I look at the STM32Cube HAL drivers, for example, I'm absolutely positive they assigned interns to write them given just how naively and stupidly written they are. And don't get me started on the STM32Cube documentation!)
The deeper reason
Of course there's a reason why they don't sell the tools: nobody would buy them. Even if they didn't suck bowling balls through garden hoses (and ST's tooling is nearly best of breed in this industry: Altera's tooling sucks matter out of galactic core black holes!), nobody buys development tools. If it's not free, it's not used, essentially. Why is this?
Another CASE study (hah!)
Way back when, in the days I'd succumbed to the OOP Kool-Aid, I spent my own money on a copy of Rational Rose, a UML modeling/code generation tool. I'd fallen for the marketing. It could read in my C++ (ugh!) code and generate UML models that would help me understand it better, refactor it, and improve it all! It could generate code from models I made to practically write my application for me! It would improve my productivity in ways that I'd never believed possible!
And it was all a lie.
Rational Rose was a buggy piece of shit that crashed multiple times per session of use (usually taking its fragile model database with it each time). It could only import a subset of C++, choking (and, naturally, crashing) on constructs it couldn't work out … you know, such alien C++ things as nested templates … and it generated code that wouldn't compile with any compiler known to humanity at the time. (Seriously: I tried to get it to compile with MSVC, gcc, and several commercial compilers and never had any one of them compile the generated code out of the box.) It not only didn't improve my productivity, it actively damaged it.
And for this I paid five digits. In 1990s money!
And I played this out for several tools in a row, spending easily in the range of six figures of personal money (in 1990s money!), with only one such tool—Clarion—having actually profited me any, before finally deciding that, well, software tooling sucks no matter how much you pay for it, so I might as well just use free (as in beer) software.
The shoemaker paradox death spiral
This is where the death spiral begins. I'm not the only developer in the world who's spotted the pattern “software tools don't work as advertised, so spending money on them is pointless”. This means software tooling (like the aforementioned CubeMX) is not viewed as a profit line item. This means that resources aren't properly allocated to make them work properly. This means that the quality slips, reinforcing the position that spending money on software tooling is a fool's game.
In the end this results in: – compilers that are so buggy that I have a hobby of collecting bizarre compiler bugs; – libraries that are so buggy I often just read the source (if available) for inspiration as I hand-roll my own (differently buggy) clones; – non-coding tools (like SCMs) that are so hopelessly convoluted (cough Git cough) that their use is a full-time job of frustration; and, finally, – a software ecosystem of crappy software made because programmers won't self-nurture.
What's the solution?
There isn't one. Not any longer. Not in the current climate. Because the real solution is professionalism, and this is anathema to the Big Tech™ TechBroDudes churning out one shitty web page after another to monetize all teh thingz!.
No, what's going to have to happen first is a major disaster—a huge catastrophe that involves loss of life on a grand scale, like five figures or more in a single incident—so that software and its practitioners are called out on the societal carpet for being the incompetents we are. Legislation will have to be passed that mandates standards of behaviour and quality in software (like every other branch of engineering already has!). The software industry will have to collapse into a fiery heap, and then get rebuilt from the ground up based on sound principles with a focus on quality engineering.
And then we have to get people away from thinking that quality is free. We'll have to pay for tools and customers will have to pay for software again.
Yes, even if it's F/OSS.