Keep your GPG keys secure with a bootable Alpine Linux USB

There are no shortage of articles that criticize OpenPGP and its implementations like GnuPG. Common arguments against OpenPGP and its implementations include the reliance on long-lived keys that complicates key management and precludes forward secrecy, the view that the protocol is outdated and no longer reflects modern cryptographic practice, and the claim that this complexity manifests as poor usability that leads ordinary users to make mistakes. There are also voices from the academic world that raise similar criticisms, notably in the “Why Johnny” studies: “Why Johnny Can’t Encrypt” (1999), “Why Johnny Still Can’t Encrypt” (2006), “Why Johnny Still, Still Can’t Encrypt” (2015) and “SoK: Why Johnny Can’t Fix PGP Standardization” (2020). (However, there is a recent poster with the title “Johnny, after 35 years, you will finally be able to use PGP!”)

On the other hand, there are also voices in defence of OpenPGP. This article provides a thoughtful rebuttal to the common criticisms above, arguing that cryptographic primitives do not necessarily become insecure with time and longevity can instead indicate robustness. Furthermore, it notes that many critiques conflate protocol flexibility and extensibility with insecurity or incorrectly attribute problems of specific implementations and user interfaces to the OpenPGP standard itself. Similarly, it should be considered that calls to replace OpenPGP often implicitly favor centralized or opaque systems that trade away decentralization, user control and long-term flexibility. See also the response to criticisms from one of the GnuPG authors. It should be noted that there are ongoing efforts to modernize the OpenPGP specification, including RFC 9580, as well as the independent LibrePGP specification. This has raised the concern, however, that the existence of multiple, partially incompatible specifications may lead to fragmentation and interoperability issues within the ecosystem.

Given the difficulty of using OpenPGP, is GnuPG worth it? GnuPG assumes that you are willing to take responsibility for your own encryption, and the system is only as effective as your ability to manage keys and trust correctly. If all you need is encrypted messaging, end-to-end encrypted communication tools that deliberately hide this complexity, such as Signal or GNU Jami, are likely a better fit, as are encrypted email services like Proton Mail or Tuta. In certain cases, especially in the Free Software community, a GPG key is practically a necessity to participate meaningfully. Outside of these cases, if after considering your threat model you find that your cryptography needs are more complex and you are willing to invest time in understanding how OpenPGP works, then GnuPG remains a remarkably flexible and powerful tool.

Here are a few good resources that explain how OpenPGP and GnuPG work behind the scenes, particularly the structure of keys and the role of subkeys. This overview gives a clear overview of the structure of an OpenPGP certificate and the relationships between the primary key and its subkeys. For a more visual and intuitive treatment, this illustrated explanation walks through how GnuPG keys are actually a bundle of key pairs through numerous diagrams. Finally, OpenPGP for application developers provides an in-depth, specification-oriented perspective that is particularly useful if you want to integrate OpenPGP into an application.

A widely recommended way to use GnuPG securely is to separate the certification key from the subkeys and to generate the entire key hierarchy on an air-gapped system. The reason for this is that the certification key, whose purpose is to certify your identity and sign subkeys, has very different security requirements from the other subkeys, since losing access to it is equivalent to losing your cryptographic identity. In contrast, subkeys can be rotated or revoked independently, and are therefore suitable for routine use. Therefore, many users place these subkeys onto a physical security token, which are designed to make daily cryptographic operations convenient whilst significantly raising the bar for key exfiltration.

In this section, we will install Alpine Linux onto a bootable USB stick to generate the GPG keys in order to reduce the exposure of the secret key material. Alpine Linux is well suited for this purpose because it is a minimal distribution with a relatively small attack surface, and it supports a diskless mode where the entire system runs from RAM rather than persistent storage. It also uses APK as its package manager, which is notable for its declarative approach to system state via a configuration file. As an alternative to running from USB, the same system can also be installed onto an SD card and used on a Raspberry Pi for a truly air-gapped setup. For the security token, I am using a Nitrokey device because both the hardware and software is Open Source and it has undergone third-party security review by Cure53, although a lower-cost alternative is Gnuk, which is Free Software providing firmware for smart cards but requires manual configuration and hardware assembly, making it more suitable for technically experienced users.

First, download an ISO for Alpine Linux from their website. Select the “extended” version, if it is available for your computer’s architecture, since it provides a more complete base system and reduces the need for immediate network access during setup. The image can then be flashed onto a USB stick using either dd(1) on UNIX-like systems or a tool such as Rufus on Windows. Below, the former method is shown.

Plug in your USB stick and identify the correct device node under /dev/ using lsblk or fdisk -l. You can find the correct USB stick by comparing the size, system partitions and manufacturer or label information of each listed device. These utilities are provided by the util-linux package, although BusyBox comes with a more limited version of fdisk. See the lsblk(8) man page for options that control which columns are displayed. Now run the following command, where /path/to/alpine-linux.iso refers to the path of the Alpine Linux ISO image you downloaded and /dev/sdX is the device node for the USB stick. Make absolutely sure the selected device is the right one. If you choose the wrong device, this will immediately overwrite that device and all data will be lost irreversibly.

dd if=alpine.iso of=/dev/sdX bs=4M conv=fsync

In the above command, the bs=4M option sets the block size to four megabytes, which generally improves write performance by reducing the number of system calls performed during the copy operation. The conv=fsync option ensures that all buffered data is flushed to the physical device before dd exits, reducing the risk of data corruption if the USB stick is removed too early. On systems that use the GNU implementation of dd, the optional status=progress flag can be added to display live progress during the write operation, but this option is a GNU-specific extension and is not guaranteed to be available on other implementations.

The image that we just flashed is an installation environment used only to prepare a second bootable USB stick that will contain the offline system. Keep the USB stick plugged in and reboot the system to boot from this installation medium. On most systems this requires selecting the USB device from the firmware boot menu, which is typically accessed via a key such as F2, F10, F12 or Escape during startup. Once the system has started, log in at the console as root (no password is required) and run the setup-alpine script in order to configure the running system. Ensure that networking and a package repository mirror is set up so that packages can be later fetched. When the script reaches the disk configuration stage, answer none to all disk and storage questions.

Now plug in the second USB stick. Install the required tooling by running apk add dosfstools util-linux. Identify the correct device node exactly as before using lsblk or fdisk -l and make absolutely sure it is the new USB stick. Then run the following command, where /dev/sdX should be replaced with the device node corresponding to the second USB stick.

fdisk /dev/sdX <<EOF
  o
  n
  p
  1
  2048
  -0
  t
  0c
  a
  w
EOF

The o command creates a new empty DOS partition table, n creates a new partition, p selects a primary partition, 1 chooses the first partition slot, 2048 aligns the partition start to a safe boundary, -0 tells fdisk to extend the partition to the end of the device, t changes the partition type, 0c selects FAT32 LBA, a marks the partition as bootable and w writes the changes to disk. If using the interactive interface, you can press m to see help for these one-letter commands. Note that the -0 shorthand is supported by fdisk from util-linux but not by the BusyBox version, in which case you must manually enter the final sector number. Be aware that once w is entered, all changes are written permanently and any existing data on the device becomes irrecoverable. As fdisk only defines the partition layout and does not create a file system, the new partition must be formatted with a FAT file system using mkdosfs -F 32 /dev/sdX1, where /dev/sdX1 refers to the first partition that was just created on the USB stick.

After setting up the file system, run the setup-bootable script to install a minimal, bootable Alpine Linux system onto it, following the procedure described in the Alpine wiki at Create a Bootable Device. In this step, the bootable system files are copied directly from the installation environment onto the target USB stick. First determine where the installation media is mounted by running the mount command. It is typically available as a read-only file system under /media/usb. Adjust the source path as needed and run, for example, setup-bootable -v /mediaresolution/usb /dev/sdX1, where /dev/sdX1 refers to the previously created partition on the second USB stick. Once the copy completes, the second USB stick contains a bootable Alpine Linux system with a writable file system that will serve as the basis for the air-gapped environment. Do not reboot the computer yet.

The extended Alpine installation image does not include GnuPG and the paperkey package, which we will later use to back up our keys onto paper, is only available from the testing repository. To get around this, we will fetch these packages in advance using the installation environment and populate a local cache on the second USB stick to make these available offline. First mount the second USB stick by running mount /dev/sdX1 /mnt, where /dev/sdX1 is the partition created earlier. Then run setup-apkcache and select a directory on the mounted USB stick, such as /mnt/cache, as the cache location. While still running from the installation environment by running setup-apkcache and selecting a directory on the second USB as the cache location. Any packages downloaded will now be stored on the second USB and can later be installed without network access when booted into the offline system.

Modify the file /etc/apk/repositories to add the testing repository, which is required for the paperkey package. The file should look similar to the following example, adjusted to match the Alpine release you are using. You can change http to https if the mirror you selected earlier supports the HTTPS protocol to avoid network tampering during package retrieval.

https://dl-cdn.alpinelinux.org/alpine/v3.20/main
https://dl-cdn.alpinelinux.org/alpine/v3.20/community
@testing https://dl-cdn.alpinelinux.org/alpine/edge/testing

Now we need to download the required packages. Use apk add gnupg paperkey@testing. The file /etc/apk/world should look similar to the following, with existing entries preserved and the new packages added as separate lines.

alpine-base
busybox
dosfstools
gnupg
paperkey@testing
util-linux

List the contents of the cache directory you set earlier (probably /mnt/cache) using the ls command to confirm that these packages are present. If anything is missing, you can inspect or manage the cache using the apk cache command, as described in the Alpine Linux Wiki. Once everything is there, you can shutdown the system, unplug the installation USB stick and boot into the second USB stick to continue with the setup of the air-gapped system.

For unattended or repeatable installs, Alpine Linux supports answerfiles, which pre-define the responses normally entered during setup-alpine. Create a file with the contents in the example below, then run setup-alpine -n -f answerfile; the additional -n flag is required to explicitly skip setting a root password, because without it setup-alpine will still stop and prompt interactively for a master password even when an answerfile is provided. This answerfile configures the system to run entirely without network access and directs Alpine’s package manager to use the previously prepared local cache on the USB stick.

# Use US layout with US variant.
KEYMAPOPTS="us us"

# Set hostname to 'alpine'.
HOSTNAMEOPTS=alpine

# Set device manager to mdev.
DEVDOPTS=mdev

# Set up the network interfaces.
INTERFACESOPTS="auto lo
iface lo inet loopback
"

# Set nameserver.
DNSOPTS="-d internal 192.168.0.1"

# Set timezone to UTC.
TIMEZONEOPTS="UTC"

# Set HTTP/FTP proxy.
PROXYOPTS=none

# Use local APK repository.
APKREPOSOPTS="/media/usb/apks"

# Do not create any non-root users.
USEROPTS=none

# Do not install SSH.
SSHDOPTS=none

# Do not install NTP.
NTPOPTS=none

# Set up diskless installation.
DISKOPTS=none

# Set up USB for configuration and APK cache storage.
LBUOPTS="usb"
APKCACHEOPTS="/media/usb/cache"

Modify the file /etc/apk/world to include the lines gnupg and paperkey@testing, as shown previously. Once this file is updated, run apk add or apk fix without any arguments from the installation environment. This causes APK to resolve the system state defined in /etc/apk/world using, where available, the packages in the configured local cache. If an online package mirror was accidentally enabled during setup, you can add the --no-network flag to these commands to prevent APK from contacting them and resolve strictly against the local cache.

Generate your GPG key with gpg --expert --full-gen-key. You will be presented with a different key types using various cryptographic algorithms. Compared to the Rivest–Shamir–Adleman (RSA) cryptosystem, elliptic-curve cryptography (ECC) provides strong security with much smaller and faster keys, which is a desirable property if you plan to make paper back ups of your keys. GnuPG also offers Kyber as of version 2.5.19, which is a post-quantum cryptography algorithm, but note that Kyber produces significantly larger keys than either RSA or ECC. Nevertheless, you may want to consider this option if you intend to keep long-term sensitive data and are concerned about harvest-now-decrypt-later attacks.

The --expert option enables more key types and allows you to explicitly select the capabilities assigned to each key. By default, after selecting your preferred key type, GnuPG generates a single key with multiple capabilities. However, since we will later create a dedicated signing subkey, you should remove the capabilities from this key so that it is limited to certification. When prompted for parameters such as key size or elliptic curve, the default values are appropriate unless you have a specific reason to choose otherwise. You should enter 3y when asked for the expiry date to set it three years into the future.

Eventually, you will be asked to enter a password to protect your GPG keys, which will act as a last line of defence in case the key material is ever exposed. Ideally, you choose a strong password that is also memorable. Because we will later set a short PIN on the security token for everyday use, the length of this password and the time required to type it are not significant concerns. A good option is to use a passphrase comprised of multiple words. You may use the word list I created for this purpose, which contains 1,296 common English words chosen to avoid spelling differences between American and British English. This yields a little over 10 bits of entropy per word, so a passphrase of about nine randomly chosen words is sufficient to exceed 90 bits of entropy, which is likely good enough to withstand a brute-force attack for all practical purposes.

Due to the importance of remembering this passphrase, you should keep a backup in case you forget it, since losing the passphrase is equivalent to permanently losing access to the keys. To do this securely, you can follow these instructions to split the passphrase into multiple shares, which you should store in separate locations. Each share on its own reveals no useful information, and the original passphrase can only be recovered when all shares are combined.

When you run gpg --list-keys, you should see a single certification key along with its fingerprint. We will now add each of the subkeys. Run gpg --expert --edit-key FPR, where FPR is the fingerprint of the key you just generated; you may also use your name or e-mail address instead and GnuPG will select the correct key automatically if the identifier is unambiguous. This drops you into an interactive prompt, where you can type help to see the available commands. Use the addkey command to create subkeys, repeating the process to add one subkey for encryption, signing and authentication, depending on your needs. When you are finished, the key listing should show a distinct certification key and separate subkeys for each capability you configured.

Since Alpine Linux was configured to run entirely from RAM, changes made during a session would normally be lost on reboot; the lbu utility exists to allow persistant changes trough a local backup which stores these changes in a backup file which is loaded on startup. The configuration for this utility lives in /etc/lbu/lbu.conf, where you can control options such as the number of retained backups and whether the backup is encrypted. Some interesting options are the number of backups and setting encryption for the backups. The encryption uses OpenSSL’s AES-256-CBC algorithm by default, but, as of the time of writing, it does not use PBKDF2, which you will be warned about when encrypting or decrypting the backup file.

The list of paths included in and excluded from the backup is defined in /etc/apk/protected_paths.d/lbu.list. To ensure that your GnuPG keyring persists across reboots, add the root user’s home directory by running lbu include /root. You may also choose to store other important secrets in this location, encrypted with GnuPG, so that the USB stick can act as a recovery point for rebuilding your computing environment if necessary. Conversely, some files under the GnuPG home directory (typically located at .gnupg) can be excluded from the backup using lbu exclude, such as the random_seed, which is frequently regenerated, lock files (ending in .lock or beginning with .#lk) and socket files (such as S.gpg-agent). You can review the effective include and exclude set at any time by running lbu list. Once you are satisfied with the selection, persist the changes by running lbu commit, which writes the backup archive to the location configured earlier during setup.

The following are the paths that I include and exclude in the backup.

+root/
-root/.ash_history
-root/.gnupg/public-keys.d/pubring.db.lock
-root/.gnupg/S.keyboxd
-root/.gnupg/S.gpg-agent.ssh
-root/.gnupg/S.gpg-agent.browser
-root/.gnupg/S.gpg-agent.extra
-root/.gnupg/S.gpg-agent
-root/.gnupg/random_seed

Once the keys have been generated, it is essential to create proper backups for recovery if you lose access to the USB stick, including at least one offsite backup in case a single incident affects both the USB stick and the first backup. In addition to the secret keys, this backup should also include important auxiliary data such as configuration files, public keys, owner trust values and revocation certificates, since these are required to fully restore GnuPG to its original state. Simply copying the ~/.gnupg directory alone will not do, as the structure of the directory and file formats may change in future GnuPG versions. Instead, it is better to use the official key export options, since the export format is designed to be portable across GnuPG versions and even compatible with other OpenPGP implementations, making it the more robust long-term backup strategy.

Run the following commands to export the relevant key material and trust data to the home folder. Note that exporting secret keys is sufficient, since the secret key material already contain a copy of the corresponding public key material, which GnuPG can use to reconstruct the public key during import. The --export-options backup flag instructs GnuPG to include metadata required to restore the key or keys later using GnuPG.

gpg --export-options backup --export-secret-keys --armor > secret-keys.asc
gpg --export-ownertrust > ownertrust.txt

Connect an additional storage device dedicated to backups and mount it at /mnt. As before, identify the correct device using lsblk or fdisk -l to avoid saving the backup to the wrong disk. Once the device is mounted, use lbu package $(date +%F)-backup.apkvol to create a backup archive in your home directory containing the files that would normally committed by lbu commit. The difference between the two commands is that lbu package allows you to explicitly specify the output location. After the backup archive is created, encrypt and sign it using the command below, where FPR is the fingerprint of your key. GnuPG will use for signing the default key specified in the configuration file gpg.conf or the first key in the secret keyring if this is not set. You can override this by adding the --local-user option and specifying the fingerprint of the key that you wish to use.

gpg --encrypt --sign --symmetric --recipient FPR "$(date +%F)-backup.apkvol"

Although the secret key material inside is already encrypted, the additional encryption layer helps hide metadata such as filenames and key identifiers as an extra layer of defence should you accidentally drop the backup storage medium during transport. The reason that you should not simply asymmetrically encrypt the archive is that doing so creates a chicken-and-the-egg problem: you would need your GnuPG key in order to access the backup that contains your GnuPG key. However, having both symmetric and asymmetric encryption enabled allows you to decrypt the archive either with the passphrase alone or, if it is still available, using your security token. When decrypting the backup, GnuPG will automatically verify the signature if you have your public key in the keyring. After creating the encrypted archive, copy only the encrypted version to an external USB device, securely delete or shred the unencrypted archive, use the eject command to safely unmount the backup media, and then physically disconnect it.

Do the same for your second backup device and then take that second copy to an offsite location. Keep in mind that your security is only as strong as the weakest link: if you store any backups on a network-enabled device, you undermine the purpose of an air-gapped system, since an attacker could compromise your keys through the backup instead. Each time you make a change to one of your keys, repeat the backup process in the following order. First, update the on-site backup medium and then take that updated medium offsite. Next, bring the previously offsite backup medium back on site and update it as well, keeping it locally afterwards. By always updating the on-site copy first and then rotating which physical device is stored offsite, you ensure that there is always a current offsite backup at all times.

In addition, you may also want to make a paper backup of your keys. Because NAND flash can silently lose stored charge over time as electrons leak from floating gates causing undetectable bit flips during reads, it is not the best option for long-term data retention. In comparison, paper can last even hundreds of years when stored in good conditions, making it a good option for long-term secrets. This is useful as a method of last resort, should all other backup options become inaccessible.

To create a paper backup of your secret key, run the command below, where secret-key.gpg is the export of the secret key created earlier. This produces a text file containing the minimal data required to reconstruct the secret key, without including the public key material.

paperkey --secret-key secret-key.gpg --output paperkey.txt

In case you want to store other sensitive data on paper, you can first encrypt it with GnuPG and then use my hexpress script to format it into a readable representation suitable for printing.

To print these, you can install CUPS and use it to print via USB or over a local network, but it is important to consider that many modern printers are network-connected devices, which has the potential to lead to unintended data exposure. Alternatively, writing the backup by hand is a safer alternative, though it is more error-prone.

The resulting paper should ideally be stored in a fireproof and waterproof safe, and placing it inside a sealed plastic freezer bag can provide additional protection against moisture damage. To restore the data, you can either scan the written backup, bearing in mind that a network-connected scanner has similar risks to a networked printer, or manually transcribe it back into a digital file.

The following command can be used to reconstruct the original secret key material, where public-key.gpg contains the corresponding public key and paperkey.txt is the file produced earlier by paperkey. The public key can be obtained from wherever you chose to publish or store it previously, such as your own website, a public keyserver or another system.

paperkey --pubring public-key.gpg --secrets paperkey.txt --output secret.gpg

You can now insert your security token into the air-gapped machine and transfer the subkeys onto it using GnuPG’s smart-card support. In general, this involves entering the interactive key editor with gpg --edit-key FPR, where FPR the fingerprint of your key, selecting each subkey in turn, and using the keytocard command to move it onto the device. During this process, GnuPG will prompt you to choose which slot on the token (signature, encryption or authentication) should hold the selected subkey. When done, type quit and enter N when prompted to save your changes. Refer to the documentation for your chosen security token for more information.

Once a subkey has been transferred, it is removed from the local secret keyring, but the changes remain in memory unless you run the save command, which will save the changes to disk. Even if you accidentally saved after moving the keys to the card, however, fret not. Since we configured Alpine Linux to run entirely from RAM, so long as you do not commit these changes using lbu commit, the secret keys will be restored on next boot. You can also restore the secret keys from the backup created earlier if necessary.

Next, you need to move the public key material off the air-gapped system. Export the public key with gpg --export FPR > public-key.gpg and copy that file onto a separate USB stick, where FPR is the fingerprint of your key. You can shutdown the system and boot into the machine where you intend to regularly use GnuPG. You will need to install GnuPG using the system’s package manager.

GnuPG comes with its own minimal implementation of a CCID driver, but the build time flags for this support are not always enabled by Linux distrobutions, which means you will have to rely on an external driver. This is also needed for some devices that do not implement the protocol in a standards-conforming manner. If this is the case, use the search functionality of your system’s package manager to locate the packages for PCSC lite and CCID and install these.

Plug in the USB with the export of the public key and import it with gpg --import public-key.gpg. Now insert the security token into the main machine and run gpg --card-status. GnuPG will automatically create so-called “smart-card stubs” in your local keyring. These stubs are references that tell GnuPG that the private key operations for those subkeys must be delegated to the smart card.

From now on, GnuPG will transparently use the security token for signing, decryption and authentication. You only need to access the certification key for certain key management tasks, such as creating new subkeys, revoking subkeys, extending expiration dates, signing other people’s keys or changing user IDs. In these cases, you should plug in the USB stick with the Alpine Linux system and boot into it to complete the tasks.

In the course of this article, we built an air-gapped environment using a minimal Alpine Linux system running from a bootable USB stick, used it to generate a GnuPG key offline, separated the certification key from its subkeys and transferred the subkeys onto a security token for daily use. We also created several independent backups to allow restoring the secret keys in case of loss of access, including a paper backup suitable for long-term storage.

This significantly reduces the exposure of the certification key, at the cost of additional friction when performing certain infrequent key-management tasks. For those new to GnuPG, becoming comfortable with this workflow and understanding the OpenPGP protocol takes time and deliberate practice. Nonetheless, if you decide to adopt this setup, it provides a practical way to use GnuPG reasonably securely.