Sorry, if your server is gone (e.g., terminated EC2 instance) nobody can help you. Next time either save an escrow key somewhere[s] safe or don't set a passphrase.

Don't misunderstand, we'd love to help if we could, but we can't. The encryption key for your backup was generated locally on your server not ours. We designed passphrase protection to use special cryptographic countermeasures to make typical cracking techniques (e.g., dictionary attacks) very difficult even for someone with access to massive amounts of computer resources.

Note, if the system you backed up is still available, just log into it as root and change the passphrase (you don't need to know the old passphrase):

tklbam-passphrase

No, but if you backup more frequently (e.g., hourly instead of daily), we strongly recommend creating full backups more frequently - daily or weekly instead of monthly (I.e., the default).

The reason is that long backup chains are inefficient and more vulnerable if something goes wrong as links in the chain depend on one another.

If you backup daily, and do a full backup monthly, your backup chains will consist of a full backup with at most 31 incremental backups linked to it. But if you backup hourly, by the end of the month your backup chain could consist of up to 744 incremental backups, all of which have to be downloaded and extracted when you restore.

To configure automatic hourly backups with a full backup every 7 days:

mv /etc/cron.daily/tklbam-backup /etc/cron.hourly
chmod +x /etc/cron.hourly/tklbam-backup

echo full-backup 7D >> /etc/tklbam/conf

By default, a full backup will happen if the last full backup is older than 30 days. Between full backups, all backup sessions are incremental.

We recommend enabling the daily backup cron job so that daily incremental backups happen automatically:

chmod +x /etc/cron.daily/tklbam-backup

You can override the default by setting the full-backup parameter in the tklbam configuration:

# create a full backup every 14 days
echo full-backup 14D >> /etc/tklbam/conf

By adding a negative database override to /etc/tklbam/overrides:

# exclude drupal7 database
echo -mysql:drupal7 >> /etc/tklbam/overrides

# exclude sessions table in drupal8 database
echo -mysql:drupal8/sessions >> /etc/tklbam/overrides

Or on the command line:

tklbam-backup -- -mysql:drupal6/page_cache

By default ALL databases are backed up so adding a negative database override excludes only that database or table from the backup.

Excluding a table only excludes its data. The schema is still backed up as long as the database is included.

Specifying a positive database override changes the default behavior so that only the database or table specified in the override is included in the backup.

You can mix positive overrides with negative overrides.

A full backup is a backup that can be restored independently of any other backup. An incremental backup links with the last backup before it and only includes changes made since the previous backup.

Backup chains are links of backup sessions which start with a full backup, and then a series of incremental backups, each recording only the changes made since the backup before it. Incremental backups are useful because they are fast and efficient.

Restoring an incremental backup requires retrieving the volumes of all incremental backup sessions made before it, up to and including the full backup that started the chain. The longer the backup chain, the more time it will take to restore.

By adding an override to /etc/tklbam/overrides:

echo /mnt/images >> /etc/tklbam/overrides

Or on the command line:

tklbam-backup /var/www/*/logs

Make sure you understand the implications of doing this. For example, if you add a directory handled by package management this may break package management on the system you restore to.

By adding a negative override to /etc/tklbam/overrides:

echo -/var/www/*/logs >> /etc/tklbam/overrides

Every TurnKey appliance that TKLBAM supports has a corresponding backup profile, which is downloaded from the Hub when you initialize TKLBAM. The profile is used to calculate the list of system changes we need to backup. It usually describes the installation state of a TurnKey appliance and contains a list of packages, filesystem paths to scan for changes and an index of the contents of those paths which records timestamps, ownership and permissions.

You can also generate your own custom profiles with the following command:

tklbam-internal create-profile

The backup profile is stored in /var/lib/tklbam/profile and contains the following text files:

1. dirindex.conf: a list of directories to check for changes by default. This list does not include any files or directories maintained by the package management system.
2. dirindex: appliance installation state - filesystem index
3. packages: appliance installation state - list of packages

Users can override which files and directories are checked for changes by configuring overrides (See below).

By default TKLBAM is designed to work with S3 automatically, and this is the easiest and safest option. In manual mode, TKLBAM can also work with non-S3 storage addresses, but this complicates usage and carries additional risks which you should make sure you understand first to avoid data loss.

Of all non-S3 manual storage targets, the local filesystem is the simplest options because you don't need to mess around with authentication credentials.

So assuming you want to store your backup at /mnt/otherdisk:

tklbam-backup --address file:///mnt/otherdisk/tklbam/backup
tklbam-escrow /mnt/otherdisk/tklbam/key

And restore like this:

tklbam-restore --address file:///mnt/otherdisk/tklbam/backup 
               --keyfile=/mnt/otherdisk/tklbam/key

Not as easy as the Hub-enabled "automatic" mode, but still easier than a conventional backup process. Linux supports mounting most types of storage devices (e.g., external harddrive, local network file share) to the filesystem though this can require extra configuration at the operating system level.

One of the main disadvantage of using a local storage target, besides the more complicated setup and maintenance process is that you won't be able to restore/test your backup in the cloud, or from a VM running in another office branch (for example).

Also keep in mind that a physical hard disk, even a RAID array, provides much lower data reliability than the 11 nines (99.999999999%) of Amazon S3

For this reason we recommend users use local backups to supplement cloud backups (e.g., providing fast local access).

See also:

• Do I have to use Amazon S3 for storage?

One of my favorite ways to do this:

# step 1: generate a backup dump
tklbam-backup --dump=/tmp/mybackup

# step 2: interactively review the dump's file contents & disk usage
cd /tmp/mybackup
apt-get install ncdu
ncdu

# step 3: add includes or excludes, go back to step 1, rinse, repeat
vim /etc/tklbam/overrides

# Everything perfect?
tklbam-backup --upload-raw=/tmp/mybackup

By default, TKLBAM will automatically determine what paths and databases need to be backed up on a given TurnKey system according to the backup profile it gets from the Hub. The default profile tracks changes to the user-servicable, customizable parts of the filesystem (e.g., /etc /root /home /var /usr/local /var /opt /srv) while ignoring changes in areas maintained by the package management system.

You can "override" the default backup profile configuration by specifying overrides, either on the command line, or preferably by editing the /etc/tklbam/overrides configuration file.

Yes. For example, let's say your default TKLBAM backup is several gigabytes in size and you'd like to create a lighter 100 MB backup that will be updated more frequently and take less time to update/restore:

cp -a /etc/tklbam /etc/tklbam.light

echo -/var/www/\*/logs >> /etc/tklbam.light/overrides
echo -/home/liraz/bigfiles >> /etc/tklbam.light/overrides
echo -mysql:mydatabase/bigtable >> /etc/tklbam.light/overrides

export TKLBAM_CONF=/etc/tklbam.light

mkdir /var/lib/tklbam.light
export TKLBAM_REGISTRY=/var/lib/tklbam.light

tklbam-init
tklbam-backup

For convenience you may want to create a script that sets the TKLBAM_REGISTRY and TKLBAM_CONF environment variables:

cat > /usr/local/bin/tklbam-backup-light << EOF
#!/bin/bash
export TKLBAM_CONF=/etc/tklbam.light
export TKLBAM_REGISTRY=/var/lib/tklbam.light

tklbam-backup
EOF

chmod +x /usr/local/bin/tklbam-backup-light

Yes. Here are a couple of recommended ways to do this:

1. Create a separate backup with an empty backup profile:

   export TKLBAM_REGISTRY=/var/lib/tklbam.srv
   export TKLBAM_CONF=/etc/tklbam.srv
   
   tklbam-init --force-profile=empty
   tklbam-backup --skip-packages --skip-database -- /srv
   

2. Use the --raw-upload option

   This lobotomizes TKLBAM so instead of creating a system level backup it just backs up the directory you specify. In other words, --raw-upload turns TKLBAM into a directory-level backup tool rather than a system-level backup tool.

   For example, let's say you have a collection of big files at /srv that you don't want to include in your system backup (e.g., because you don't want to bloat your backup).

   So you configure an overrides to exclude the /srv directory from your default backup and create another TKLBAM backup just for the big files:

   echo -/srv >> /etc/tklbam/overrides
   export TKLBAM_REGISTRY=/var/lib/tklbam.srv-raw
   tklbam-backup --raw-upload=/srv
   

   Later, you'll need to use the --raw-download option to restore:

   tklbam-restore --raw-download=/srv <your-backup-id>
   

   If you don't use the raw-download option, TKLBAM will assume you are trying to restore a system-level backup and you'll get an error.