If you run a VM, you should patch and update it regularly.  New vulnerabilities come out all the time, and even if an update doesn’t address a security problem, it fixes bugs or provides new features.

But of course, every update has a risk.  Even if the update itself doesn’t cause any system problems, it might cause issues with other software you’re running (or your own code).  Personally, I can’t remember the last time I had an issue after running an apt upgrade.  Definitely not true for a full OS version upgrade, where distros introduce a lot of changes.  Nevertheless, with every patch, there’s a risk.

Of course, you can back out an individual apt upgrade.  For example, something like

apt install nginx=1.18.0-6ubuntu14

would roll back to that specific version of nginx.  But it can be time-consuming to determine the old version and the dependencies.

There’s a Better Way

In a lot of big organizations who have extensive VM farms and regular patching programs, they protect themselves from bad patches by doing a VM-level snapshot.

If the patching goes bad, reverting it is only a click and a reboot away.

This technique is widely used and very effective.  And if you’re running something like Proxmox at home, or your provider offers snapshots, you can do this, too.

The core command in Proxmox is

qm snapshot "${VMID}" "${SNAP_NAME}" --description "${SNAP_DESC}"

So you could easily write a script (to run on your Proxmox host) to do a snapshot of the VM and then ssh into the VM and do apt upgrade and apt upgrade.

I’d recommend keeping your snapshot name something formulaic so you can have an automated cleanup script to clean up old snapshots.  You don’t want to be running your update script everything and piling up snapshot after snapshot until the disk is full.

But There’s a Catch

How do you know a snapshot is bad?  Or rather, how soon would you know?

Remember that a snapshot is an all-or-nothing fallback.  If you snapshot on Monday at 1am, then you can take the whole VM back to Monday at 1am.  But what if only one part of it is having problems?  What if you have 10 apps and only 1 is broken, and by Tuesday afternoon when the problem is discovered, you’ve done a lot of work in the other apps?

The reason the snapshot-then-patch method is widely used in IT is that these organizations have automated testing of the applications and infrastructure.  In theory, immediately after the VM is patched, a full suite of tests are run and any problems are determined.

But you may not have that.  Still, having a VM snapshot is a valuable tool to have because it gives you flexibility.  If a problem arises, you can choose whether to get into reverting a single package or set of packages, or do a quick snapshot failback.