Somehow, over the past few years, I have ended up with ten Raspberry Pi single board computers scattered around the Palatial Overclock Estate, all running 24x7, most on uninterruptible power supplies, almost all headless. And that doesn't count the ones in the basement that I get out from time to time for ongoing projects, like my IPv6 testbed or my differential GPS rover.
Having lost one of them to a failed micro-SD card, the Pi's boot media, it occurred to me that maybe I should start thinking about a way to back them up. You would think this was a solved problem. But not so much.
The Wrong Solution
The most common mechanism - and the one I had been using - was to simply to copy the entire raw disk image on the micro-SD card to a handy backup disk as a flat file using the Linux dd command. Then, in the unlikely event that the need arises, restore it by doing a reverse image copy to an unused micro-SD card using a similar dd command.
But as common as this approach seems to be on the Salmon of Knowledge, it turns out to be uglier than it sounds. Even though those micro-SD cards from SanDisk and Samsung both claim to contain sixteen gigabytes, they aren't actually identical, differing very slightly in terms of the actual number of logical blocks they contain. You can dd a smaller image to a larger card, but obviously not vice versa. This is all beside the fact that a full raw disk image copy takes a long time.
I addressed this in the past in two ways.
First: restoring, for example, a sixteen gigabyte image to a thirty-two gigabyte card. Sounds simple, but it's not scalable: when I make changes to the system, I have to make another image copy. This one will be thirty-two gigabytes. So if I have to restore it, it will be to a sixty-four gigabyte card, even though I was wasting most of that thirty-two gigabyte card; the image backup includes all of the blocks on the card I was backing up whether they were used or not.
Second: I went on Amazon.com and ordered some micro-SD cards identical in brand, model, and capacity to the one I was backing up. Also a simple solution, as long as I can find those cards, but it ignores the growing collection I have of unused micro-SD cards.
The Right Solution
Thanks (as usual) to the web site Stack Exchange, and specifically the user goldilocks from which I drew inspiration, I have written a set of shell scripts that use the rsync utility to make a file-by-file incremental backup of a Raspberry Pi. And to format an unused micro-SD card of any suitable capacity and restore those files to it, recreating the boot media. Which I've tested.
The first backup takes a long time because it is a full backup. Subsequent backups - done after I make a change - take almost no time at all. The approach I took was to run the backup on the Pi itself a.k.a. online (rsync can also be run remotely, but I elected not to do that). I bought a one terabyte solid state disk (SSD) with a USB interface. I hook it up to the Pi while it is running, mount the SSD, and run the backup script. Formatting and restoring to an unused micro-SD card is done on one of my desktop Linux servers a.k.a. offline.
The Gory Details
These scripts can be found here: https://github.com/coverclock/com-diag-bin . They are licensed under the GPL. When I install these scripts on a system, I typically clone the repository, then create soft links from a bin directory to the scripts I need in the repo directory. During that process I drop the .sh suffix from the script name. So in the repo the script is called pilocalbackup.sh but in the bin directory it is just pilocalbackup.
In the examples below, /dev/sdx is a stand-in for the device name of the micro-SD card from the Raspberry Pi, and /dev/sdy is a stand-in for the device name of the backup SSD. Your mileage may vary. The standard Raspbian micro-SD card will have two partitions, the boot partition in /dev/sdx1 and the root partition in /dev/sdx2. There are files in both partitions that have hard coded references to these partition numbers.
I always assume I'm restoring to a two partition Raspbian card, but I can backup from a Noobs card for which Raspbian was selected at install time; A Noobs card will have something like seven partitions, five of which are unused in the Raspbian instance. The conversion from Noobs to Raspbian requires some editing of files in both the boot and root partitions of the card to change the boot partition to 1 and the root partition to 2. If you are restoring a micro-SD card to create a duplicate of an existing system, you might also want to change the hostname and IP address of the Raspberry Pi that you are restoring. This is shown below.
The mount point names I use below are purely my own personal convention. I use /mnt for the backup SSD, /mnt1 for the boot partition (1) on the micro-SD card, and /mnt2 for the root partition (2) on the micro-SD card. The backup script backs up both partitions, and the restore script restores both partitions.
All of the scripts necessarily have sudo commands embedded in them, so I encourage you to inspect them carefully; sudo is only used when necessary.
The README.md for the repo has a bunch of examples, including dealing with full raw image files. Below I'll show just a few germane cases. The repo also has a lot of other unrelated but useful (to me, anyway) scripts.
Determine the Raspbian Release from Various Media
cat /etc/debian_version # This is online on the Pi itself.
or
sudo mount /dev/sdx2 /mnt2
cat /mnt2/etc/debian_version # This is an offline Pi micro-SD card.
sudo umount /mnt2
or
sudo mount /dev/sdy1 /mnt
cat /mnt/pi/framistat/root/etc/debian_version # This is the backup.
sudo umount /mnt
Backup Local Files Online On A Pi Using rsync
sudo mount /dev/sdy1 /mnt
pilocalbackup /mnt/pi/framistat # This is the default.
sudo umount /mnt
or
pilocalbackup # This uses the default.
sudo umount /mnt
Restore Files Offline To An Unused Micro-SD Card Using rsync
piimageformat /dev/sdx doodad
sudo mount /dev/sdx1 /mnt1 # This is the boot partition.
sudo mount /dev/sdx2 /mnt2 # This is the root partition.
sudo mount /dev/sdy1 /mnt # This is the backup drive.
pilocalrestore /mnt/pi/framistat /mnt1 /mnt2
sudo umount /mnt /mnt1 /mnt2
Check, Verify, And Repair a Raspberry Pi Image Offline
piimagecheck /dev/sdx /dev/sdx1 /dev/sdx2 # These are the defaults.
or
piimagecheck /dev/sdx # This uses the defaults.
Customize a Raspberry Pi Image Offline After Restoringsudo mount /dev/sdx1 /mnt1
sudo vi /mnt1/cmdline.txt # Change the boot partition.
sudo umount /mnt1
sudo mount /dev/sdx2 /mnt2
sudo vi /mnt2/etc/fstab # Change the / root and /boot partitions.
sudo vi /mnt2/etc/dhcpcd.conf # Change the static IP address.
sudo vi /mnt2/etc/hostname # Change the host name.
sudo vi /mnt2/etc/hosts # Change the host name resolution.
sudo umount /mnt2
No comments:
Post a Comment