Yesterday on LowEndTalk, member @idroid007 posed a question that probably never occurs to most VPS subscribers:

What if your VPS provider’s storage gets corrupted?

This happens less frequently today than it did in the spinning rust (SATA) era.  Head crashes were inevitable with spinning disk – indeed, I think decades from now, people will look back at those drives and wonder how they worked at all, given the extreme precision needed for the technology to not fail.  SSDs wear out but they are less sensitive.

But they do wear out.  And even if the drive hardware is fine, there can still be other issues.  There could be an issue with the RAID controller, or the RAID software.  There could also be something done at the wetware level – a sysadmin fat-fingers a command, a datacenter tech yanks the wrong drive out for maintenance, or a malicious actor damages the system.

So what happens then?

The answer is…it depends on the provider.

In the best case scenario, the provider has snapshots and will swiftly restore your data to a new storage subsystem and reboot your VM.  But in our world of ultra-cheap LowEndBoxes, that’s asking a lot.

The second best case scenario is that the provider does in-house backups, so they’ll provision a new VM and then undertake a restore.  This wouldn’t be nearly as fast as a snapshot, because the provider is going to have to spin back terabytes of data for everyone who’s been affected.

This happened to me once.  Back probably 10 or more years ago, a provider had a problem with faulty power supplies and it zapped a whole bunch of servers.  They had backups, so after a long wait – I think it was a week or so – I got my VM back.  Considering I was paying probably $5/month, this was fine.

What’s most likely, however, is that you’ll be given a brand new VM, your data at the provider is lost, and it’s up to you to restore.

This may sound shocking, but reading your Terms of Service carefully.  Most providers do not offer backups.  Margins are razor thin when VMs are selling for $2/month and expecting instant recovery – or even any recovery at all – is asking too much.

This is why you should always have a backup / disaster recovery plan in place!  Anything that’s important on your VPS should be backed up somewhere that is completely unrelated to the provider.  If you have service at provider A in New York, backup to provider B in Amsterdam (or to your own home).

You could also consider a storage VPS.  $5/TB is a popular price point.  Or using some kind of self-hosted S3 storage or BackBlaze’s B2.  If you use the latter, be sure to investigate CloudFlare’s bandwidth alliance.  With any cloud-hosted service (storage VPS, self-hosted, or SaaS), be sure to calculate the cost of a restore.  Hopefully you’ll never need it, but if you do, you don’t want to be paying 9 cents a GB to stream all your data back.

Good luck and stay safe!

raindog308

raindog308 is a longtime community LETizen, technical writer, and self-described techno polymath. With deep roots in the *nix world, he has a passion for systems both modern and vintage, ranging from Unix, Perl, Python, and Golang to shell scripting and mainframe-era operating systems like MVS. He’s equally comfortable with relational database systems, having spent years working with Oracle, PostgreSQL, and MySQL.

As an avid user of LowEndBox providers, raindog308 runs an empire of LEBs, from tiny boxes for VPNs, to mid-sized instances for application hosting, and heavyweight servers for data storage and complex databases. He brings both technical rigor and real-world experience to every piece he writes.

Beyond the command line, raindog308 has a life-long love of German Shepherd Dogs, high-quality knives, target shooting, theology, tabletop RPGs, playing guitar, and hiking in deep, quiet forests.

His goal with every article is to help users, from beginners to seasoned sysadmins, get more value, performance, and enjoyment out of their infrastructure.

You can find him daily in the forums at LowEndTalk under the handle @raindog308.