Michael Bien's Weblog

The Raspberry Pi 3b+ runs on a Cortex A53 quad core processor which is using the Armv8 architecture. This allows it to run 64bit (AArch64) kernels, but be also backwards compatible with Armv7 and run 32bit (Armhf) kernels.

One of the most convenient ways to get started with a Raspberry Pi is to use the Debian based Raspbian distribution. Raspbian maintains its own customized linux kernel, drivers and firmware and provides 32bit OS images.

For my usecase however, I required the following:

• get a server friendly 64bit OS running on the Pi3b+ (for example Debian)
• use a mainline kernel
• avoid proprietary binary blobs if possible
• boot from a SSD instead of a SD card or USB stick (connected via USB adapter)

It turned out most of it was fairly easy to achieve thanks to the RaspberryPi3 debian port efforts. The raspi3-image-spec project on github contains instructions and the configuration for building a 64bit Debian Buster image which is using the mainline linux-image-arm64 kernels, ready to be booted from.

The image creation process is configured via raspi3.yaml which I modified to leave out wifi packages (i don't need wifi) and use netfilter tables (nft) as replacement for the now deprecated iptables but this could have been changed after initial boot too if desired.

The image will contain two partitions, the first is formatted as vfat labeled RASPIFIRM and contains the kernel, firmware and raspi specific boot configuration. The second ext4 partition is labeled RASPIROOT and contains the OS. Since partition labels are used the raspi should be able to find the partitions regardless of how they are connected or which drive type they are on (SSD, SD card reader, USB stick..).

Once the custom 64bit Debian image is built, simply write it to the SSD using dd, plug it in and power the raspi on. Once it booted don't forget to change the root password, create users, review ssh settings (disable root login), configure netfilter tables etc.

Some additional tweaks

Reduced the GPU memory RAM partition to 32MB to have more RAM left while still being able to use a terminal via a 1080p HDMI screen. Set cma=32M in /boot/firmware/cmdline.txt. I did also add a swapfile to free up additional memory by writing inactive pages to the SSD. The default image does not have swap configured.

force_turbo=1

flag to /boot/firmware/config.txt. This will lock the ARM cores to their max clock frequency (1.4GHz). Since the linux kernel doesn't ship with a Pi 3 compatible cpufreq implementation yet, the Pi3b+ would just boot with idle clock (600Mhz) and never change it based on load. This is just a workaround until the CPU frequency scaling drivers made it into the kernel (update: commits made it into kernel 5.3). I didn't experience any CPU temperature issues so far, but i am also not running CPU intensive tasks.

The files will get overwritten on kernel updates, I added most of the /etc folder into a local git repository which makes diffs trivial and simplifies maintenance. If the raspi stops booting you can simply plug the SSD USB adapter into a workstation and edit the config files from there, diff, revert etc - very convenient.

USB Issues

I ran into an issue where the SSD connected via USB got disconnected after 5-30 days. This was very annoying to debug because it would not finish writing the kernel log file since virtually everything was on that disconnected drive. After swapping out everything hardware wise, compiling custom kernels (on the raspberry pi btw :) ), even falling back to the raspberry kernel which uses different USB drivers (dwc_otg instead of dwc2) I could not get rid of the disconnects - only delay it. Newer kernels seemed to improve the situation but what finally solved the issue for me was an old USB 2.0 hub i put in front of the SSD-USB adapter. Problem solved.

Thats about it. In case you are wondering what i am using the raspi for: This weblog you are reading right now is running on it (beside other things like a local wiki), for over six month by now, while only using 2-5W, fairly stable :)