Automated Hass.io Backups + Recovering From Disaster

June 20, 2018

Automated Hassio and Node-RED Backups

Today on the hookup we’re going to talk about one of the most important things in the IT world, backups, and how to manage them automatically so you won’t have to think about them!

Backups are not a sexy subject, but they are really important.  Whether it’s a hard drive crash, a flash memory failure, or just carelessly deleting or saving over an important file , at some point we’ve all been burned by not having a recent backup of our work.  The best backup systems have three things in common: they happen often, and they happen automatically, and they are redundant, meaning it’s best to have a backup of your backups.  Today we’re going to set up some automations in node red and home assistant that check all of those boxes.

Lets talk first about backing up your backups.  The most common way I’ve seen people back up their home assistant configurations is using github.  I’m not a huge fan of this solution, and here’s why: Github is public, any files that you upload can be viewed by anyone on the internet.  This is great for the main purpose of github, which is a place to publish your open source code, but it’s not great when you might have sensitive credentials in your files.  Correct use of the secrets.yaml and git.ignore files is absolutely crucial to ensuring you don’t publish your sensitive data for the entire world to see.  You should know that there are nefarious bots combing github constantly searching for sensitive information that people accidentally uploaded… if it hits github servers even for a few seconds, you should assume that your sensitive information has been compromised.

I also don’t like that when backing up to github there is no local copy of the backup, aside from the one actually on the home assistant drive.  I’m fairly positive that github isn’t going to just randomly shut down their servers, but as a reoccurring theme on my channel you might have noticed that if at all possible I like to keep things local.

Instead of github, I would highly recommend that you store your backups on dropbox.  Not only does dropbox give you a local copy of your files, but it is also cannot be accessed publicly on the internet.  If you’re already using dropbox and you don’t want to waste your storage space on your home assistant backups, there’s no reason you can’t sign up for another dropbox account and use the free 2GB to store your backups.  While you’re at it, you might as well use my dropbox referral link in the description to create your new account so I can get some of that sweet sweet free storage.

 

Lets walk through the process of creating a new dropbox account and linking it to your hassio instance.  The first thing you need to do is add Daniel welch’s addon repository to your hassio addons page.  That link is in the description.  Once you’ve done that you’ll have the option to install the hassio dropbox sync addon, go ahead and click install, and then we’ll head on over to dropbox to create a new account

(don’t forget to use my referral link from the description).

Enter your email address and choose a password to sign up.  It will bring you to a screen to ask you to download the dropbox program, but you don’t need to.  Just click on the dropbox icon in the upper left to get to your web interface.  On the next screen you can just hit “end tour”.  Then click on the face icon in the upper right corner and select settings.  Here you’ll need to verify your email so you can qualify for a developer key.

Next you’ll need to navigate to the developer console for dropbox by going to this url https://www.dropbox.com/developers/apps which is also posted in the description.  Click on create app.

On this next page you need to select dropbox API, name your app and decide whether you want to give it full access to your dropbox.  Since I created a brand new dropbox account for this I chose just to give it full access.  Otherwise it will create a folder inside a new folder in your drobox called apps, and your configuration will be slightly different.

Click on agree to terms and press create app.

Next you’ll see a page with a bunch of options that you don’t need to touch.  All you need on this page is to generate an access token, click that button and copy your token.

Next we’ll head back to our dropbox sync addon and put in our access token and also tell it which folder to backup to.  I made a folder called hassio backups.  You can also set a “keep last” parameter here, which will control how many snapshots will be kept locally on your SD card.

(finish this using screenshots)

Now that we’ve completed our addon’s config section we need to click the start button to activate it.  It’s important to note that clicking start doesn’t actually sync your backups, to do this, you need to run a service call… more on that in a minute.

Hassio has a great feature that allows us to create and name snapshots by calling a service.  You’ll recall that one of the most important parts about a good backup system is that they are automated.  We’re going to use node-red to handle this automation.  Here’s what the flow looks like.

To start, we need to schedule how often we want this backup to happen.  To do this I’m using one of my favorite node red custom nodes, the light scheduler node… I’ll include that package name in the description so you can add it using palette manager.  I chose once a week, specifically on Monday since the weekend is typically when I’m messing with my home assistant setup.  After running through a switch node that filters out only the ON payload I send it into a service call that begins a hassio snapshot formatted with the current date.

Because the snapshot is far from an instant process I’ve added a conservative delay time of 1 hour from the creation of the snapshot to the firing of the service to upload our new backup files to dropbox.  Once the service is called your dropbox sync addon does it’s thing and uploads your new snapshot to your dropbox.  Now we have a great backup system that we never need to think about… sort of.

There is one downside to this system that I haven’t been able to solve yet:  On my instance of home assistant each snapshot is around 225 megabytes, which means that I’ll only be able to store 8 full snapshots on dropbox before I need to go in and delete them.  The dropbox sync addon will delete old local copies of your snapshots if you want it to using the “keep last” parameter in the configuration, but it can’t touch the files in your dropbox.  This means every 2 months you’ll need to manually delete old files from your dropbox like a caveman.

Let me stop you there past Rob.  Did you ever stop to question why each backup is 300 megabytes?  I certainly haven’t written 300 megabytes of code, in fact, all the important time consuming parts of my home assistant configuration are text files.  So why are my backups 300 megabytes?

Examining the archived backup file revealed some interesting problems with my home assistant setup, and maybe with yours.  Located within your config folder is a file called home assistant v2 (dot) db.  This is your home assistant database file where event and state changes are stored.  How big is yours? Mine was a hulking 250 megabyes on a system that was only a few weeks old.  Now, you may want this massive record of your home’s events and states, but for the most part I couldn’t care less.  All I want is to have a 24 hour history graph and the ability to restore previous states after a reboot.

To prevent your database from storing ALL of your home’s happenings for all eternity you need to add a few things to your configuration file.

The recorder component determines which events and states get added to your database, and also controls how long they stay there.

By adding purge keep days, and purge interval we can modify how many days of information are kept in the database.  Using purge interval 1 my home assistant will prune my database file every day, and by using purge keep days 1 it will only keep the last 24 hours of states when a purge is called.  This means at any point I have between 24 and 48 hours of events and states in my database.  This keeps the file at a much more reasonable 10 megabyes, which is acceptable to me.

The second culprit was node-red.  If you’re using the notorious BDG version of this addon when you take a snapshot backup of it you are actually producing a backup of the install, not of your flows.  Your configuration and flows for node red are actually located in your share folder, which we can backup using a partial snapshot by adding the share folder to our partial snapshot

If you’re using Francks version of Node RED your files are located in the config folder and will automatically be backed up when the homeassistant folder is included in your partial snapshot.

By making these few small changes we should be able to get our snapshots down in the 15 megabyte range, which means our 2 gigabytes of dropbox space should last nearly 2.5 years before we need to manually delete old back up file.  Hopefully we’ll never have to use these files, but now we have a weekly, automatic, redundant backup that we can depend on if we do need them.

The process of restoring your home assistant is actually pretty straight forward.  There’s an option to restore a partial snapshot in hassio, but I’d actually recommend a different method.  All you need to do is install a fresh copy of hassio on your sd card (I would recommend having a second SD card so you can keep your original installation intact in case you need to go back to it), go into your hassio menu and install your various addons including the samba share addon so you can access your folders from your PC.

Once you’ve got samba added, go ahead and replace the contents of your new config folder with the config folder from your backup image (a dot tar file can be opened just like a zip file using winrar).  And if you use node-red make sure the addon is stopped and replace the contents of your node-red folder either in /share if you’re using the Notorious BDG version or /config if you’re using Franck’s version and restart home assistant.

Once your home assistant restarts you should be 100% back up and working.  I tried this out on my own system and was able to go from a blank SD card to a fresh home assistant install with my exact same configuration in less than 90 minutes, which seems like a decently long time until you realize that about 70 minutes of that was waiting for hassio and various addons to install.

I know no one wants to spend their precious time messing around with a backup system, but for me imagining the pain of reconfiguring my home assistant from scratch gives me serious nightmares.  Hopefully you were able to get some useful information from this video.  Next week we’ll be back to building stuff.  If you enjoyed this video, please consider subscribing, and as always, thanks for watching to the hookup.

Links

Dropbox referral link: https://db.tt/iDkPuSSn6Q
Daniel Welch’s addon repository: https://github.com/danielwelch/hassio-addons
Dropbox developer console: https://www.dropbox.com/developers/apps
Light Scheduler: https://www.npmjs.com/package/node-red-contrib-light-scheduler
Node-RED Backup Flow: https://github.com/thehookup/backups/blob/master/backup_flow.txt

Follow me on Twitter: @TheHookUp1
Support my channel:
Patreon: https://www.patreon.com/thehookup

Music by www.BenSound.com

Related Posts