Bad engineering, the FreeBSD way

I'll put the TL;DR summary of my experience with FreeBSD on a Raspberry Pi 4B here: https://www.youtube.com/watch?v=K4Fvsgv0bYw The best thing I can say about the experience is “at least it wasn't OpenBSD”. (More on that in the postscript to this essay.)

…

OK, so, that's a bit unfair. Not everything about the experience stank. And this is the problem. I really wanted to like this. Of the non-Linux, non-Windows alternatives, FreeBSD comes so close to being usable that it's incredibly frustrating that it fails so profoundly in the end.

I'll be dividing this review up into three parts: Plus, Minus, and Interesting. Plus has the good things about the experience of trying out FreeBSD on a Raspberry Pi 4B. Minus has the bad things. Interesting is a grab bag of observations that are not good, not bad, but, well, interesting. Things to note that may be bad or good for individual users, or which are just noteworthy in some way or another but not really a judgement. Then, at the end, I'll have a postscript, as mentioned, about OpenBSD and why there will not be a full review of OpenBSD on the Raspberry Pi 4B.

Plus

Despite the negative overall tone this review is going to have, there actually are positive things in my FreeBSD experience.

Installation

The first (and largest) of these is that FreeBSD is astonishingly easy to install. The last time I tried FreeBSD (on a PC) was version … I want to say 9, but it might have been 7. Then it was a problem. A big one. The only reason I continued into it as far as I did was I was curious to see where the rabbit hole ended. And I did finally get a working FreeBSD. FSVO “working”.

This time the process was utterly painless. I dded the image to an SD card, stuck that into the Raspberry Pi, plugged in the network cable and powered up the machine. Eventually nmap found a new host on my network and I was able to ssh into it using the account freebsd (password freebsd). From there everything just worked, modulo the different approach to things.

Documentation

Other nice things were that it was pretty easy to find information. Between the installed man pages and the FreeBSD web site, I was rarely left at a point where I had to decode what the next step would be. (This is not new. I had a similar experience the first time I tried FreeBSD on a PC.) The FreeBSD community seems to believe in documentation, which is a rarity in the field. (It's still documentation written by software people, so terrible, but it's actually present which is a huge bonus.)

pkg is kind of slick

In terms of functionality and ability, pkg is pretty slick. I rarely had to refer to its documentation on how to do things. The only thing that confused me a little until I checked the docs was what pkg list really means. (And the docs cleared that up instantly.) Were it not for a major, glaring, horror show of a problem I'd put pkg up there on the list of package managers I don't hate.

Minus

All that being said, the minuses in my opinion grossly outweighed the pluses. One was a showstopper for me, though arguably not for all possible users.

pkg problems

One of the very first things that is suggested is to type the command pkg list. This begins a rabbit hole of lovingly invoking every single one of the 8 fallacies of distributed computing. For some reason the package manager isn't installed by default on the boot image. Instead a placeholder script downloads it and installs it. (For all I know it may even build it.)

This was disastrous. Typing pkg list (or any pkg subcommand) resulted in a message saying that pkg isn't installed, that it would have to be downloaded before it could be used. It even said what URL it was downloading it from. Then nothing. For ages. No feedback. No progress indication. No pacifier of any kind. Then, about 15 minutes later, typically, it would just say it failed. Type pkg <whatever> again and it starts up the process all over again. And again. And again. And again. Whatever progress it had made was lost and it seemed the task just began from the beginning again.

Three hours later, just as I was about to give up and scrap the use of FreeBSD entirely, it finally came through. Then normal operations started to work and, thankfully, with both progress feedback and without the constant failure. The package lists got updated and the package database could be queried quickly and easily.

But…

That heralded the real problem: poor infrastructure combined with even worse handling of the poor infrastructure.

The infrastructure

Poor infrastructure is a given when you're a fringe project. This is unavoidable. Being fringe means having fewer resources (in this case mirror sites with high bandwidth). The result is an infrastructure that is painfully slow outside of narrow blessed areas.

I'm not in a blessed area. My download speeds for packages ranged from about 8-32KB/s. Picture downloading a full GCC implementation (the embedded systems spin) at that rate. We're talking download times of over an hour. (About 1.5 hours, actually, predicted by the progress pacifier.)

Only when you smear out a download over that time, you increase the chances of there being an interrupting glitch dramatically. Indeed about 15 minutes on average was how long it took before it just gave up.

The handling

Which is where we segue neatly into the handling of the infrastructure. If your infrastructure sucks (through no fault of your own!) you have to be smarter about its use.

The FreeBSD developers are not being smart about its use.

When (and not if) the download fails, instead of saving progress thus far and continuing, they throw away the download. You start over again from scratch. So if you get 10% of the way through, instead of continuing with 90% left over, they continue with 100% left over. Repeatedly and at length.

In practice this means that the package will never be installed.

There is no excuse for this handling either. We've known how to do restartable downloads for decades. If I partially install a package using APT or YUM, for example, killing it (or having it die) mid-install, it picks up from where it last was and continues. Because it uses signatures at the end to make sure that the whole file got picked up without an invalid package getting in the loop. (An even smarter system could use hash trees to identify the specific blocks that got damaged and re-download justt those blocks.) Restarting from scratch on something that failed is just going to make it fail again with no progress.

From the 8 fallacies of distributed computing this is the first three put together into one flaw. It's a flaw that all package managers used to have, but most today don't. Except, apparently, pkg. And it's the flaw that made me in the end have to drop FreeBSD as an option on my Raspberry Pi 4B not because I didn't like it, but because it literally couldn't actually do what I needed from it.

Which is a real pity.

Performance

The very badly-implemented package manager aside, there was one more weird problem that made FreeBSD look dubious: looooooooooooooooooong delays in things.

If I were to take an educated guess, the hardware RNG is not being used, so FreeBSD's kernel is using software means to fill the entropy needs. This wasn't a showstopper problem (and may have had a solution findable had the pkg problem not stopped me in my tracks), but it was definitely not a good look. And really, given that this is a build for a specific machine with fixed, known-in-advance hardware, there isn't a good excuse for it if there was, indeed, a solution to it.

Interesting

Some observations that aren't praise or critiques (or may be one or the other depending purely on taste).

sudo

The FreeBSD approach to system use and administration is very old school. Where literally every Linux distribution I've tried on the Raspberry Pi 4B used the sudo approach (to the point that logging in as actual root was actually a problem in some cases!), FreeBSD defaults to old-school root/user divides where you have to su to the root account and do maintenance tasks there. It took me a while to unlearn the reflex habit of typing sudo before admin commands. Note that you can install sudo if you really want to do it the other way, but this is not the native way of doing things and I'm not sure that it's fully supported. I'm not sticking around (for reasons already cited) to figure out if using sudo is viable or not, but it does at least seem to exist which suggests it might be.

“Ports” vs. “Packages”

Another case where FreeBSD differs from most other OSes is the schizophrenic divide it has between its “Ports” and its “Packages” infrastructure. “Packages” are precompiled binaries of software (in the vein of Debian's APT or Red Hat's YUM suites) while “Ports” are a massive tree of Makefiles that you use to download and build software according to presupplied recipes (and configuration scripts) more in the style of Gentoo and that family of Linux distros. Packages tend to be easier to work with while Ports covers more software. Which you use is a matter of taste.

It is generally strongly advised in the FreeBSD docs to pick one or the other and to not try and mix them. I chose Packages because the notion of compiling everything (with the inevitable screw-ups caused by bad configuration files—the reason why my previous trial of FreeBSD ended, in fact) just gives me hives. You may choose differently and it's interesting that FreeBSD supports both. (It's a pity, however, that the two don't appear to coexist peacefully.)

Conclusion

FreeBSD is not ready for mass consumption yet, but it is frustratingly close. The FreeBSD community has put a lot of work into improving it and it shows. Unfortunately it doesn't show enough. There is still the traditional Unix cowboy coding attitude showing through in both the utilities they provide and the infrastructure that supports the system, and that results in a system that is personally completely unusable.

But it's close. Indeed it's so close that if you're in a place that is well-supported by their infrastructure it may actually be ready for you. It may be a viable alternative. But until the developers of the tooling and infrastructure learn and understand the 8 fallacies of distributed computing and incorporate this understanding into their tooling, FreeBSD remains an also-ran for the majority of their prospective user base.

Which is too bad, because the world needs more competition in OS space, not less, and FreeBSD is oh-so-close to being viable as that.

Postscript: At least it wasn't OpenBSD

As bad as the FreeBSD experience was, the OpenBSD experience never happened because it's a fucking disaster from before you even install the operating system. I was six clicks deep into that before I finally hit the ARM64 support page for OpenBSD. That page contained link after link after link after link after link after link to things that had absolutely nothing to do with OpenBSD on ARM64 platforms. They were mostly link after link after link after link after link to product home pages that didn't mention OpenBSD in any way, shape, or form. Finally, in the bottom three lines of a very long web page there were links to:

Getting and installing OpenBSD/arm64:

The latest supported OpenBSD/arm64 release is OpenBSD 7.0. Here are the OpenBSD/arm64 installation instructions.

Snapshots are made available from time to time, in this location as well as on a few mirrors. Here are the OpenBSD/arm64 snapshot installation instructions as well.

What's a snapshot? Fuck knows. Which is the preferred approach? Don't look here for clues. I mean why the fuck would you want to make installation of your product easy to gather more users? Obviously you want to make it as difficult and confusing as possible so that you can only have the 1337 users, leaving behind the people who actually just want to do things with their hardware. (You know, the fools who think computers are tools, not an end unto themselves!)

Oh, note that the instructions are delivered in plain text. Despite, you know, having links inside of them. That's a bonus “fuck you” for the end-user, as is the fact that you require the serial console to install and run the images … if you ever figure out how the fuck the images are supposed to be loaded(foreshadowing!: hint, it's install70.img).

I'm going to say this again for the people in the bleachers, unlike literally every operating system I tested for the Raspberry Pi 4B, including FreeBSD!, OpenBSD requires the serial terminal to get anything working. (Almost as bad, to be fair, is Fedora and Void, two Linux distributions that need a screen and keyboard to configure before you can use them. No headless installs.)

At this point I stopped even considering OpenBSD (as I did Fedora and Void). BSD fans are always going on about how, supposedly, it was legal shenanigans that killed BSD in favour of Linux. The reality is that BSD killed itself by sucking. FreeBSD is frustratingly close to being usable and really just needs an actual infrastructure in place, but OpenBSD is almost the Platonic Form of an utterly useless piece of shit for 99.44% of people who might be interested in installing something other than The Latest Version of Linux™.