TKLBAM: Migration to v14.x from earlier versions of TurnKey Linux

There have been some significant changes in Debian Jessie (what TurnKey Linux v14.x is based on) from previous versions. This page is aimed at providing a general overview and some info on a couple of common issues. It is not exhaustive and you may well encounter other issues.

If you have strange things happen after migrating from an earlier version of TKL to v14.x then please post on the support forums and someone will be along to try to help ASAP. Also if you are a Hub user (or AWS Marketplace subscriber) then please also feel free to try our support portal (accessible when logged into the Hub).

It is suggested that you read through this entire page before you start. Then either follow the Suggested Workflow or the Semi-manual/Staged TKLBAM Migration. Either way you will likely still need to make some Manual Adjustments for Specific Software.

Suggested migration workflow

It is suggested that you migrate from older (pre v14.x) versions of TurnKey in a staged process, something like this:

  1. Note the specific points as detailed towards the bottom of this page.
  2. Audit restored data in new server - check what is working and what isn't.
  3. Resolve issues noted in your audit, documenting as you go.
  4. Do the final migration.

1. Initial restore and audit

It is recommended that you do not touch your production server (where your backup comes from) until the migration is complete and you are 100% satisfied.

First check that your latest backup is fairly recent. Assuming you haven't made any significant changes to your appliance within the last month or so; a backup from within that timeframe should be adequate for our initial purposes. Then restore your data to a disposible test server (e.g. a local VM or a fresh AWS server, etc) and do a full audit. Take careful note of what is working and what isn't. Once you have the audit complete, take some time to do some research on anything you are unfamiliar with.

When searching for documentation/hints on specific issues, generally Google is your friend. Keep in mind that under the hood TurnKey is built on Debian, so generally stuff that applies to Debian, also applies to TurnKey. v14.x is based on Debian Jessie aka Debian 8.

Please note that when appliances include software that is installed from upstream (e.g. TurnKey installs WordPress directly from WordPress themselves) that usually, the whole application is included in your backup. So if you migrate your v13.0 WordPress site (let's pretend it's running WordPress v3.2) to a v14.2 WordPress appliance, you'll still be running the old version of the software. In other words, even though TurnKey v14.2 comes with WordPress v4.4, your old 3.2 site will overwrite it.

As most of our appliances are LAMP based, I'm using PHP as an example, but this generally applies to other software too. If you are coming from v13.x, and have not activiely updated the software, most PHP applications running on v13.x should still work ok as the difference between PHP in v13.x (PHPv5.4) and v14.x (PHPv5.6) are not too significant. However, there is a chance that older software that ran fine on v13.x may be a bit glitchy on v14.x, or perhaps not work at all.

If the software you are using, isn't properly supportted on the newer version of PHP (or whatever language/platform) then you may also need to upgrade the software itself (e.g. WordPress). Even if you don't need to, you may wish to update to a newer version. Newer versions often included new features, and ongoing security fixes.

If you are updating third party software from upstream, the first thing to do is read the docs. Projects (almost) always provide migration instructions or scripts from one version to another. Before you do anything it's well worth reading through any relevant documentation you can find. Unless you are experiencing issues during your audit, generally I recommend that you make upgrading third party software an additional step later, rather than tackling it now...

2. Work through issues found during audit

Consider this initial run purely a test. That way you can be fearless in your your tweaking. If you make something worse, or things get messy (e.g. trial and error without clean resolution) then you can just trash your test server and start again with a fresh restore (to a fresh server if you've broken things too much).

Make sure you document everything you do; especially things that work. Try to focus on one problem at a time and keep working until each solution is complete. If you end up starting a fresh, any independant issues you have already completely resolved (which don't have consequences for other issues), don't need to be reapplied at this point. Any issues you have partially resolved, or need to be redone (to reveal other issues which you still need to address) can be redone using your docs as reference.

3. Final migration

You've worked through all issues uncovered in your audit and documented their resolution. Back on your production server, where relevant and possible; put it in "maintenance mode" or otherwise lock it (so no new content is added). Then run a final backup.

Restore your new backup to a fresh new server. Follow your docs to complete the migration so everything is working well. Take it out of "maintenance mode" (etc).

Once you are 100% happy, then do any final tweaks (e.g. configure DNS to map to the new machine etc) to put it into production and do a full TKLBAM backup of your new server:

tklbam-backup --full-backup now

When you are ready you can destroy the old server. Note that some like to leave the old server running for a little while, just in case. Also it is recommended that before you destroy the server; within the Hub set "max backups" to 1 and force a final full backup. That will clear all the previous backups and leave you with a final legacy backup for archive/historical purposes.

TKLBAM semi-manual/staged migration

If you have a bit of an idea where the important files you need to restore are located, a good alternative is a manual (or semi-manual) restore. This may short circuit things and mean less work in the long run. It is ideal for migrating from very old servers, or heavily customised or particularly complex setups. However TKLBAM can still be a very useful way of transferring your data to the new server and make the process semi-manual, rather than fully manual. It is still advisable to follow a somewhat similar workflow to what is noted above.

Instead of doing a full restore though, just do a download and dump:

mkdir /tklbam-dump
tklbam-restore BACKUP_ID --raw-download=/tklbam-dump

Using tklbam-restore, you can continue to restore specific parts of your backup using the --limits= switch. E.g. to restore all of /var/www/ and it's contents, plus a MySQL database named 'DATABASE_1', but no packages, you could use this line:

tklbam-restore /tklbam-dump --skip-packages --limits="/var/www mysql:DATABASE_1"

Alternatively, to skip packages and restore everything except /etc (and it's contents) and /boot (and it's contents), use this line:

tklbam-restore /tklbam-dump --skip-packages --limits="-/etc -/boot"

Please see the tklbam-restore man page for more info.

If you break something, you can roll back with tklbam-restore-rollback. Please note though that TKLBAM only stores one rollback. So it may be best to try to restore everything you need/want in one go. A good option may be to start manually making TKLBAM backups of this new server at each step of the process.

You can also manually restore files from the /tklbam-dump directory if you wish. The downloaded files should be in locations which are relative to the download directory. E.g. files that come from /var/www should be found in /tklbam-dump/var/www. Please note that by default, permissions will not be preserved when manually moving files, so permission adjustments may be required.

Either way, you'll still need to decide what to move and what not to, but it may make a manual migration a little easier. It would certainly be easier than having to do a full manual migration!

Specific software

Webmin (all appliances)

As of v14.0 Webmin is now hidden behind stunnel. This means that if you restore from an earlier version you'll need to make some changes before Webmin will work again. In the Webmin server config file (/etc/webmin/miniserv.conf) make sure that you have these entries (add them or change them as appropriate):

port=10000
ssl=
listen=10000
inetd_ssl=1
bind=127.0.0.1
sockets=
no_resolv_myname=0
ipv6=0

Also double check that stunnel config (/etc/stunnel/stunnel.conf) has this (most likely right at the end):

[webmin]
accept  = 12321
connect = 127.0.0.1:10000

If you have configured SSL certificates for Webmin (and/or Webshell), then you will most likely also need to configure stunnel to use them instead. E.g. adjust the lines in /etc/stunnel/stunnel.conf that says this:

cert = /etc/ssl/private/cert.pem

Apache (inc LAMP and about 70% of the library)

Apache config has had some significant changes, although the one that will break things is actually TurnKey related. We hardened the SSL in v14.0 and moved the default certificate location. We also included all the additional SSL/TLS config together (in /etc/apache2/mods-available/ssl.conf).

The easiest way to avoid the most significant issue, is to exclude Apache's ssl.conf file from your restore. You can do that like this:

tklbam-restore --limits="-/etc/apache2/mods-available/ssl.conf"

If you have a third party CA cert then this should not affect you (although you can move your cert if you desire). If you are using self signed certs, then simply removing reference to the old cert from your Apache vhost site files (in /etc/apache2/sites-available) should do the trick. E.g. here is the update in action on the Trac appliance.

There are other changes too that should probably be made (although Apache should still work without them). See links below for more info.

More info:

MySQL

Nothing of significance should have changed with regard to MySQL so it expected that the majority of users will be fine. However, due to a bug in PHPMyAdmin, we have switched to the lighter weight MySQL web UI; Adminer. 

You are welcome to continue with PHPMyAdmin if you'd prefer, but you'll need to disable Adminer and tweak PHPMyAdmin to use a sub directory. And get used to using PHPMyAdmin from https://YOUR_DOMAIN.COM/phpmyadmin (i.e. as a subdirectory of your site, rather than via a specific port).

Additionally (I don't think it's related, but not sure) some users have reported that their MySQL root password does not work post restore. That can easily be reset by re-running the MySQL inithook:

/usr/lib/inithooks/bin/mysqlconf.py

Fileserver

Samba has gone from v3.x in all previous versions of TKL to v4.1/v4.2 in v14.x so again some significant changes. A helpful community member has solved the main issues and posted a clear outline in the forums here.

The file access WebUI has changed too. Unfortunately, user accounts which you may have set up in old version of fileserver WebUI, may need to be recreated. OTOH the new WebUI uses Samba users, so existing users can automatically use it with their existing account and password.

MediaWiki

As of v14.2 MediaWiki is installed direct from the developers (rather than a Debian package). Instructions for migrating from v14.1 to v14.2 are documented. There are no confirmed instructions for previous versions, however it should be very similar.