Do you still use USB thumb drives? Flash or SD or MicroSD cards? Portable hard drives?
Even in this age of ubiquitous cloud, there’s still a ton of bits and bytes being carried around in people’s backpacks, for a variety of reasons. IT folk need portable media for those crash-cart, no-network situations where they have to perform CPR on a server or switch. Others use portable drive for alternate boot environments (e.g., Bootcamp on a Mac). People working with digital media often have a “working set” of files they’re currently working on, or depending on what they’re doing they may have their entire portfolio on one volume. Some of this can be archived in the cloud but current projects sometimes demand more storage than a laptop can spare. And not everyone is in cloud all the time in all locations.
Additionally, in this era of single-card systems, Micro-SD cards are everywhere. When is the last time you backed up your Raspberry Pi? If you’re doing some work that would be painful to recreate, are you confidant that you can easily recover from a backup?
In this tutorial we’re going to show you a way to make external media backups as painless as possible. Our goal is simple: plug it into a USB interface and the backup is automated. As the user, you don’t have to do anything other than plug the device in – no running of a script or clicking a button. You plug it in, the backup happens, it goes to your cloud vault, and you’re notified when the backup is done and it’s safe to unplug.
How? Read more!
The Magic of udev
udev is a device manager for Linux. One of its useful features is that it can be programmed to automatically take action when certain events happen, and we’re going to leverage that to fire off a backup script.
In this tutorial, we’re not going to specify which cloud backup solution you’re using because there’s a galaxy of options. Instead, we’re going to call a dummy script (cloud_backup.sh) and you can modify it to insert whatever you’re doing – a B2 API call, an rclone invocation, working with a min.io bucket, etc.
How It Works
First, we’re going to find a way to uniquely (or semi-uniquely) identify your USB. Then when it’s mounted, the system will automatically create a symlink to the device and then pass that symlink to our backup script.
Note that we’re going to backup a raw image of your USB drive, which you can then mount or put back on the device. This is done because the backup will succeed regardless if the drive is encrypted without any manual intervention. If the drive is encrypted, the backup image will be encrypted as well.
Uniquely Identifying Your USB
I’m working with a desktop Debian 11 system that has abundant USB 3.0 and USB 2.0 ports. In this example I’m going to use a Gorilla 32GB USB but the principles are the same regardless if we’re talking about USB thumb drives, external hard drives, or flash cards. Any Linux system with a USB port should work. Paths might be slightly different on other Linux distros.
Insert your USB device (no need to mount it) and run
lsusb -tvv
You’ll get something like this:
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M ID 1d6b:0003 Linux Foundation 3.0 root hub /sys/bus/usb/devices/ /dev/bus/usb/006/001 /: Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M ID 1d6b:0002 Linux Foundation 2.0 root hub /sys/bus/usb/devices/usb5 /dev/bus/usb/005/001 /: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M ID 1d6b:0002 Linux Foundation 2.0 root hub /sys/bus/usb/devices/usb4 /dev/bus/usb/004/001 |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M ID 8087:8001 Intel Corp. Integrated Hub /sys/bus/usb/devices/4-1 /dev/bus/usb/004/002 /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M ID 1d6b:0003 Linux Foundation 3.0 root hub /sys/bus/usb/devices/usb3 /dev/bus/usb/003/001 /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/14p, 480M ID 1d6b:0002 Linux Foundation 2.0 root hub /sys/bus/usb/devices/usb2 /dev/bus/usb/002/001 |__ Port 12: Dev 6, If 0, Class=Mass Storage, Driver=usb-storage, 480M ID abcd:1234 LogiLink UDisk flash drive /sys/bus/usb/devices/2-12 /dev/bus/usb/002/006 /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M ID 1d6b:0002 Linux Foundation 2.0 root hub /sys/bus/usb/devices/usb1 /dev/bus/usb/001/001 |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M ID 8087:8009 Intel Corp. /sys/bus/usb/devices/1-1 /dev/bus/usb/001/002
This is telling you that there are multiple USB hubs and ports. The one we’re interested is the “Mass Storage, Driver” one. And the system is nice enough to even provide us with the path: /sys/bus/usb/devices/2-12
I know this is going to shock you, but apparently some manufacturers cut corners when implementing electronics standards. Or perhaps the standard has a different idea of “serial number” than “unique identifier”. The best, proper approach is to use udevadm to query the device and get the ID_SERIAL field and build a rule around that.
However, checking many devices, I find that they generally take one of these approaches:
- do not implement the ID_SERIAL field
- implement it with something like “1234”
- use a short value (e.g., “861F-CC42”) which is not worldwide-unique but rather model unique
Some vendors do burn serials into their removable media but this is obviously a manufacturing hassle and apparently the market doesn’t care. I’ll show you how to build a rule based on serial numbers, but you probably won’t be able to use it, so we’ll use an alternate method.
Identifying with udevadm
Issue a udevadm command similar to the one below, replacing the last part of the path with information appropriate to your system. As you can see above, we’re on Bus 2/Port 12, which makes it easy to find with bash tab-completion in /sys/bus/usb/devices.
Here I’m working with a 32GB Gorilla USB thumb drive. Let’s go for the gold and see if we have a WWID to work with:
# udevadm info -q all /sys/bus/usb/devices/2-12|grep -i seri E: ID_SERIAL=USB_Flash_Disk
I guess not. So we’ll use a different udevadm command to display all the attributes.
# udevadm info -ap /sys/bus/usb/devices/2-12
Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.
looking at device '/devices/pci0000:00/0000:00:14.0/usb2/2-12':
KERNEL=="2-12"
SUBSYSTEM=="usb"
DRIVER=="usb"
ATTR{authorized}=="1"
ATTR{avoid_reset_quirk}=="0"
ATTR{bConfigurationValue}=="1"
ATTR{bDeviceClass}=="00"
ATTR{bDeviceProtocol}=="00"
ATTR{bDeviceSubClass}=="00"
ATTR{bMaxPacketSize0}=="64"
ATTR{bMaxPower}=="300mA"
ATTR{bNumConfigurations}=="1"
ATTR{bNumInterfaces}==" 1"
ATTR{bcdDevice}=="1100"
ATTR{bmAttributes}=="80"
ATTR{busnum}=="2"
ATTR{configuration}==""
ATTR{devnum}=="16"
ATTR{devpath}=="12"
ATTR{idProduct}=="1000"
ATTR{idVendor}=="090c"
ATTR{ltm_capable}=="no"
ATTR{manufacturer}=="USB"
ATTR{maxchild}=="0"
ATTR{power/active_duration}=="147776"
ATTR{power/async}=="enabled"
ATTR{power/autosuspend}=="2"
ATTR{power/autosuspend_delay_ms}=="2000"
ATTR{power/connected_duration}=="147776"
ATTR{power/control}=="on"
ATTR{power/level}=="on"
ATTR{power/persist}=="1"
ATTR{power/runtime_active_kids}=="1"
ATTR{power/runtime_active_time}=="147500"
ATTR{power/runtime_enabled}=="forbidden"
ATTR{power/runtime_status}=="active"
ATTR{power/runtime_suspended_time}=="0"
ATTR{power/runtime_usage}=="1"
ATTR{product}=="Flash Disk"
ATTR{quirks}=="0x0"
ATTR{removable}=="removable"
ATTR{rx_lanes}=="1"
ATTR{speed}=="480"
ATTR{tx_lanes}=="1"
ATTR{urbnum}=="573"
ATTR{version}==" 2.10"I’ve trimmed it because after that it’s going to walk up the USB tree and report controllers, etc. You only want the first, blank-line-delimited section.
We’ll try to grab some of these fields to make the identification as unique as possible. Of course, if you only have one USB drive you plug into a certain computer, this is not as critical as if you have a set.
Here is the final rule, which we’re going to put in /etc/udev/rules.d/95-local.rules (split into multiple lines for formatting here, but this should all be one line)
SUBSYSTEM=="usb", ACTION=="bind", ATTR{manufacturer}=="USB", ATTR{product}=="Flash Disk",
ATTR{idProduct}=="1000", ATTR{idVendor}=="090c", SYMLINK+="gorilla_drive",
RUN+="/usr/local/bin/backup-removable.sh gorilla_drive"OK, a lot to unpack:
- Rules in /etc/udev/rules.d are processed alphabetically so prefix your rules with a high number to make sure they’re processed last and not overridden. BTW, there are >100 rules used by the system in /lib/udev/rules.d.
- Be careful to use == and not =. == is a test for equality and = is an assignment. We use != above as a test for “not equals”. The syntax is unforgiving so be careful about quotes, +=, etc.
- The SUBSYSTEM just lets udev know what we’re talking about.
- ACTION is set to “bind”. This is a later step than “add”. If you use “add” your device will be removed automatically, and it’ll appear if nothing is happening. If you set udev_log=debug in /etc/udev/udev.conf and watch daemon.log after plugging in, you’ll see this happen.
- The ATTR links are ways to uniquely identify the product. If we had a serial, we could just use ATTR{idSerial}=”whatever_serial”. Instead, we’ll say “if manufacturer, product, idProduct, and idVendor match, it’s a match”.
- The SYMLINK clause specifies that /dev/gorilla_drive will be created
- The RUN clause says “run /usr/local/bin/backup-removable.sh” and passes it the gorilla_drive argument. (Note that this is a literal string being passed, so backup-removable.sh will be called with “gorilla_drive” not “/dev/gorilla_drive”).
The Backup Script
This is where you come in. What do you want to do with your backup?
Here is a quick example:
#!/bin/bash
DEVICE=${1} # e.g., "gorilla_drive"
if [ ! -f /dev/${DEVICE} ] ; then
echo "ERROR! No such device: /dev/${DEVICE}"
# perhaps send a notification here
exit 1
fi
dd if=/dev/${DEVICE} of=/backups/{DEVICE}.$(date '+%Y%m%d') bs=1MB
# and now call something like this, or minio, scp, rsync, whatever
rclone sync /backups b2:some_backet
echo "backup of $DEVICE is complete" | mailx -s "backup of $DEVICE is complete" someone@example.com
# or perhaps something like:
# aplay /usr/share/sounds/gnome/default/alerts/sonar.ogg
# or send an email to your phone's SMS
# or call an API
# etc.Trying It Out
For this example, my backup-removable.sh script is very simple:
#!/bin/bash DEVICE=$1 echo "`date` backed up $DEVICE" >> /tmp/backup_log.txt
After I created 95-local.rules, I ran:
systemctl restart udev
This may not be necessary but can’t hurt. Then I inserted by device.
Immediately I saw:
# ls -l /dev/gorilla_drive lrwxrwxrwx 1 root root 15 Jan 17 12:35 /dev/gorilla_drive -> bus/usb/002/025 # cat /tmp/backup_log.txt Mon Jan 17 12:35:45 PST 2022 backed up gorilla_drive
Restoring
If you backup using this method and need to restore, it’s as easy as:
dd if=/backups/some_image of=/dev/gorilla_drive
Or if you want to grab only some files from the backup:
mkdir /mnt/thumb mount -o loop gorilla_drive.backup.20220117 /mnt/thumb
If you’re using an encryption solution to mount (e.g., truecrypt), check the documentation. It’ll be something like:
truecrypt gorilla_drive.backup.20220117 /mnt/thumb
That’s about as painless as we can make thumb drive backups. If you’re too lazy to plug your drive in and have it automatically backed up…well, stay away from laundromats!
Related Posts:
I also enjoy using LEBs! I have an empire of little guys for VPNs, larger guys for apps, and big guys for databases and storage. When I'm not in front a screen I'm into German Shepherd dogs, quality knives, target shooting, theology, tabletop roleplaying games, and forest hiking.
I enjoy writing technical articles here on LowEndBox to help people get more out of their systems. You can find me on LowEndTalk @raindog308
Very cool share, thank you! I knew you could create backups like this, but never knew using USB to trigger.