Hi Nicholas.

You must include the pertinent directories in your backup settings in order to guarantee a thorough backup of your Docker configuration and containers using tklbam-backup. Here are some pointers for doing so:

1. The directories related to Docker that you want to backup should be identified. Your description suggests that these folders should be:

• /run/docker
• /var/lib/docker

2. Modify your tklbam-backup configuration to include these directories. You can typically find the configuration file at '/etc/tklbam/backup.d/[backup-name].conf'. Edit the file and add the directories to the list of directories to be backed up.

3. Run the backup command after saving the configuration file's modifications. By doing this, you can be guaranteed that the backup will retain the folders that you specify, together with your containers and Docker information.

You ought to be able to gather the files required for a whole backup of your Docker setup by including these folders. It's crucial to read the tklbam-backup documentation and take into account any unique needs or suggestions made by the program.

Hi Nicholas. My excuse is that I've had my head down, deep in dev mode (working on the new upcoming release; v18.0). I hope this essay makes up for the delay...

Anyway, as Vivek noted, to include the directories that you note, you would need to add them to the backup config. Although I would generally recommend that you just use the overrides file: /etc/tklbam/overrides (as noted in the TKLBAM FAQ). That is the default place to store your local modifications and reduces risk of your changes getting overwritten.

Having said that, looking at the directories you've listed, most should NOT be in your backup! See the FHS (Filesystem Hierarchy Standard) for more context (I'm writing off the top of my head, so better to head you to a reference).

/run

/run is runtime files. It should be tmpfs by default and as such all the files contained are destroyed at shutdown and need to be recreated at boot. So as a general rule, nothing in /run should be backed up.

If you have an issue with files missing from /run, then my guess is there are services not running that should be. Happy to assist you to troubleshoot it if you want to share some details.

/usr

With the exception of /usr/local, the whole /usr file tree is package managed (or at least should be). So (/usr/local notwithstanding) changes within /usr should only ever be done by the package manager (unless you know exactly what you are doing). As such, there should not be any volatile user generated data in there to back up (on the basis that TKLBAM should reinstall the same package set as currently installed - more on that below). By default TKLBAM includes /usr/local.

/var/cache

As the name suggests, /var/cache is (or at least should be) only variable (volatile) application cache data. Whilst being worth using disk space for (usually to save time), by FHS convention, it should be easily recreated/regenerated/redownloaded/etc. All applications that use /var/cache should silently re-create their cache file structure and contents. It's worth being aware that unfortunately, not all apps always abide by policy. E.g. I'm not sure if it's still the case, but apt used to whinge if you remove the /var/cache/apt/archives/partial/ directory.

Including /var/cache in your backup is unlikely to cause any issues as if there are any issues with it, it should just get invalidated and cleared by the application. On the other hand, it's unlikely to be of any real value in a backup. So up to you. Personally, if there is something in particular you want to include the cache of, I'd just explicitly include that/those, rather than everything. Also, be sure to exclude /var/cache/tklbam & /var/cache/duplicity from your backup! Including those locations will almost certainly cause some weird behavior and/or errors and/or massive backups! FYI, to exclude stuff from backups, prefix a '-' in the override. E.g. to include all of /var/cache except those 2 I mentioned above add this to your overrides file:

/var/cache
-/var/cache/tklbam
-/var/cache/duplicity

Also, another sidenote/pro tip: the quick and dirty way to clear the TKLBAM cache:

rm -r /var/cache/tklbam/*
rm -r /var/cache/duplicity/*

(Don't delete the top level directories, just the files & dirs in them)

/var/lib

/var/lib is an interesting one as once you deviate from Core, then there is often stuff there that you'll want to include in your backup. We add the ones that apps use (to the best of our abilities, please always report bugs when stuff doesn't restore cleanly when it should!). That's one of the things that in the TKLBAM profiles, we include most stuff that you should need by default and include most of the places (by the FHS) that you should be storing stuff. I suspect you may be right to want to include the Docker one (/var/lib/docker), the others I'm not convinced about.

Assuming that you're not including anything from /usr (see above), then backing up the dpkg data is a bad idea! The dpkg data (i.e. /var/lib/dpkg/info) should stay with the package management. Similar with apt (assuming you meant /var/lib/apt/lists), although the lists subdir specifically is rebuilt with 'apt update', so probably wouldn't actually cause an issue if you included it in a backup (but why would you want to?). FWIW when you first install, /var/lib/apt/lists is empty, clearing it out makes the ISO a bit smaller and it's overwritten when you run 'apt update' anyway.

Re systemd (/var/lib/systemd) after having a quick poke around in there, I don't think you want that in your backup either! It looks like systemd internals that are best left with the system IMO.

As I said above, /var/lib/docker may well be good to backup when using Docker. But assuming that's where it stores it's data, I suspect that it may be quite big. Having said that, if all your containers come from Docker Hub (or some other docker repository that you can access) then perhaps you could reduce the size of your backup by only including some (or perhaps even none?) of the Docker var/lib directory?

Installing Docker on backup restore

Having said all of that, I suspect that the reason why you want to include those is because of Docker. By default TKLBAM won't install third party software, but restoring to your existing server should work (if you include /var/lib/docker in the backup). As per my hint buried above, if you do a restore to a clean server (i.e. on Core) then Docker won't be installed. It's not a default part of Core so would need to be installed, but because it's not in the TurnKey or the Debian repos, TKLBAM won't install it - as other user installed packages from TurnKey or Debian repos would be. So off the shelf, what you suggest is probably the only obvious workaround, especially if you had never heard of the FHS before.

TBH, it's a use case (installing from 3rd party repo) that I have often considered, but never actually done anything about (I've never bothered doing it for myself and no one else has ever asked).

The way to go I reckon would be to include a hook script. You could also leverage that to do some dumps of data from your docker containers if you wanted, or perhaps better still, get your containers to use an external storage location for data that you then include in your backup? Unless you are a coder, then it may seem a little daunting, but I'm more than happy to assist as best I can.

FWIW in /etc/tklbam/hooks.d there is a "fixclock" hook script (that auto updates the clock before backup/restore) that is pretty simple and serves as something of an example. There is also an explict example script there too, as well as a couple more in /usr/share/tklbam/contrib/ (not sure how good they are...).

Sing out if you want any more input/suggestions.

Also, whilst I haven't gotten very far with it, I've been looking at building a docker host appliance, as well as leveraging that to be able to produce appliances that include apps via docker. As you're likely aware, there are a number of apps that only support install via docker. They're a pain for us and end users to maintain any other way - personally I prefer native apps when possible, but I get docker is a thing and I get that it does smooth some edges.

So this is a topic that i anticipate I will be looking more closely at.