Do you have an aging TKL Mayan EDMS server running v16.1 (v3.5.5) and want to upgrade
to v17.1 (v4.2.7)? If so this is the post for you! Read on to find out how you can migrate 
to the new version. 

This guide will help you move your Turnkey Mayan EDMS v16.1 to a new Turnkey Mayan EDMS v17.1 install. Assuming you did not upgrade your v16.1 install this will upgrade your Mayan EDMS from v3.5.5 to v4.2.7. You can then go on and upgrade your v4.2.7 to v4.3.4 by following my other post "TKL Mayan EDMS v17.1 - Upgrade v4.2.7 to v4.3.4" if you would like

I timed this from start to finish and this process for me took an hour. Your time could very so plan down time accordingly give your self more than enough time. Since we will not be changing the old server roll back to should quick if you do have issues and have to abort. 

This guide assumes:

• You have made backups of your current server
• You have a new v17.1 server installed and updated
• You are logged into both the servers as root
• You have not made any custom apps or changes to the install other than using the app.
• You have enough space to copy the backup file plus extracting.
• If your new/old servers are virtual machines, make a snapshot!
   

Overview:

• 1. On your v16.1 server
  • 1.1. Verify your config files
  • 1.1.1 - Check for missing config.yml
  • 1.1.2 - Generate new config.yml
    • 1.1.2a - Backup and remove old file
    • 1.1.2b - Restart services
    • 1.1.2c - Copy from backup file
    • 1.1.2d - Restart services, check for errors
  • 1.2 Stop Services
  • 1.3 Backup database
  • 1.4 Tar/Zip files
• 2. Transfer backup file to new server
• 3. On New Server
  • 3.1 Stop Services
  • 3.2 Take note of passwords being used
    • 3.2.1 - Database 
    • 3.2.2 - RabbitMQ (Celery Broker)
    • 3.2.3 - Redis (Celery Results Backend)
  • 3.3 Remove current media folder
  • 3.4 Untar/zip files
  • 3.5 Update migrated config.yml
    • 3.5.1 - Update Celery Broker to RabbitMQ
    • 3.5.2 - Update Celery Result Backend
    • 3.5.3 - Update Database Password
    • 3.5.4 - Update Lock Manager Backend 
  • 3.6 Check Mayan Home Directory
  • 3.7 Create User Settings
  • 3.8 Move document directories to new locations
  • 3.9 Purge/Load database
  • 3.10 Update the environment file
  • 3.11 Migrate database static files to new version
  • 3.12 Restart supervisor
  • 3.13 Check that all is well.
• 4. Clean up
  • 4.1 Remove transferred tar file
  • 4.2 Remove v16 database backup and log files
  • 4.3 Remove v16 backup config 
  • 4.4 Copy over new config from backup
    • 4.4.1 - Stop services
    • 4.4.2 - Copy config file from backup
    • 4.4.3 - Restart services, check system
    • 4.4.4 - Check to see if config files are the same
    • 4.4.5 - Remove backup config file
  • 4.5 Migration Complete!
     

Details:

# Part 1 - On your v16.1 Server

# (1.1) Verify your config files
# In this step you need to ensure you have a good working config.yml file.

# (1.1.1) - Check for missing config.yml
# I have noticed that for whatever reason the config.yml file only had one line in it
# a logging line, however things still just worked. This is because most of important 
# setting are being used in the supervisor config file `/etc/supervisor/conf.d/mayan.conf` 
# and the rest are used are the defaults. If this is your case or you don't have a
# config.yml file read on. If you have complete config.yml file go on to step 1.2


# (1.1.2) - Generate new config.yml
# Mayan will create a backup config file every time is starts successfully. We can use
# this feature to create a working config.yml file for us. Only use these steps if you do
# not have a backup file or your file is blank or only has one line.

# (1.1.2a) - Backup and remove old file
mv /opt/mayan-edms/media/config_backup.yml /opt/mayan-edms/media/backup.cfg

# (1.1.2b) - Restart services
supervisorctl restart all

# (1.1.2c) - Copy from backup file
cp /opt/mayan-edms/media/config_backup.yml /opt/mayan-edms/media/config.yml
chown mayan:mayan /opt/mayan-edms/media/config.yml

# (1.1.2d) - Restart services, check for errors
supervisorctl restart all



# (1.2) Stop Services
systemctl stop supervisor.service


# (1.3) Backup database
# Run pg_dump to dump the database, we will make sure we include clean/create to 
# drop the database objects before restore when we restpre the database on the new
# server

pg_dump --dbname=mayan \
  --clean \
  --create \
  --format=plain \
  --file=/opt/mayan-edms/media/mayan16_db_backup.sql


# (1.5) Tar/Zip files
# Tar the media directory of the v16 server. 
tar -zcvf ~/mayan16_media_files.tar.gz /opt/mayan-edms/media/


# Part 2 - Transfer backup file to new server
# Use whatever means to transfer the file mayan16_media_files.tar.gz  in the root
# home directory to the home directory of root on the new v17.1 server. You can use
# scp if the servers are on the same network or sneakernet (thumb drive)


# Part 3 - On New Server
# Once the file mayan16_media_files.tar.gz is on the new server all the commands here will
# be performed on the new v17.1 server. You may want to close the sessions to the old 
# server just so you don't delete everything on your v16 by accident. And even if you
# did delete them you have a current backup or snapshop right????


# (3.1) Stop Services
systemctl stop supervisor.service


# (3.2) Take note of passwords being used
# We will reuse these passwords in the current config.yml file so we will make a copy the
# file, and note the passwords you will need to carry over from the v16 config file
cp /opt/mayan-edms/media/config.yml ~/v17_config.yml

# Open up the config file and note each password you will need. Come back to this 
# file later when you need to update your passwords
nano ~/v17_config.yml


# (3.2.1) - Database
# Look for key "DATABASES:" will be under that
# "PASSWORD: PASSWORD_WILL_BE_HERE"

# (3.2.2) - RabbitMQ (Celery Broker)
# Look for:
# "CELERY_BROKER_URL: amqp://mayan:PASSWORD_WILL_BE_HERE@127.0.0.1:5672/mayan"

# (3.2.3) - Redis (Celery Results Backend)
# Look for:
# "CELERY_RESULT_BACKEND: redis://:PASSWORD_WILL_BE_HERE@127.0.0.1:6379/1"


# (3.3) Remove current media folder
# Danger! Be sure you are on the NEW server, no ooppsies, there be dragons!
rm -R /opt/mayan-edms/media/


# (3.4) Untar/zip files
tar -xzvf ~/mayan16_media_files.tar.gz -C /


# (3.5) Update migrated config.yml
# We need to update the migrated config to use passwords from the new server, 
# update the celery broker to use RabbitMQ, and use redis for the locking manager backend
# Use the backed up version of the config file we made at `~/v17_config.yml` 
# for the passwords needed.

nano /opt/mayan-edms/media/config.yml 

# (3.5.1) - Update Celery Broker to RabbitMQ
# Change this:
# CELERY_BROKER_URL: redis://:<old_password>@127.0.0.1:6379/0
# 
# To This, use the password you noted from step (3.2.2)
# CELERY_BROKER_URL: amqp://mayan:New_Password_Goes_Here@127.0.0.1:5672/mayan

# (3.5.2) - Update Celery Result Backend
# Change this:
# CELERY_RESULT_BACKEND: redis://:<old_password>@127.0.0.1:6379/1

# To this, use the password you noted from step (3.2.3)
# CELERY_RESULT_BACKEND: redis://:New_Password_Goes_Here@127.0.0.1:6379/1

# (3.5.3) - Update Database Password
# Change this:
# DATABASES: / PASSWORD: <old_password>

# To this, use the password you noted from step (3.2.1)
# PASSWORD: New_Password_Goes_Here

# (3.5.4) - Update Lock Manager Backend 
# Change this:
# LOCK_MANAGER_BACKEND: mayan.apps.lock_manager.backends.file_lock.FileLock
# LOCK_MANAGER_BACKEND_ARGUMENTS: {}

# To this, use the password you noted from step (3.2.3)
# LOCK_MANAGER_BACKEND: mayan.apps.lock_manager.backends.redis_lock.RedisLock
# LOCK_MANAGER_BACKEND_ARGUMENTS: {'redis_url':'redis://:New_Password_Goes_Here@127.0.0.1:6379/2'}


# (3.6) Check Mayan Home Directory
# Ensure mayan user has home directory, if it does skip this step
mkdir /home/mayan
chown mayan:mayan /home/mayan


# (3.7) Create User Settings
# Only if they do not exist already, if they exist skip this step
su - mayan -c "mkdir --parents /opt/mayan-edms/media/user_settings/"
su - mayan -c "touch /opt/mayan-edms/media/user_settings/__init__.py"


# (3.8) Create and Move document directories to new locations
# From 3.5 to 4.X the default location of files changed. Need to move them to the new locations

mv -v /opt/mayan-edms/media/document_storage/ /opt/mayan-edms/media/document_file_storage/
mv -v /opt/mayan-edms/media/document_cache/ /opt/mayan-edms/media/document_file_page_image_cache/


# (3.9) Purge/Load database
# Next we need to load the v16 database to new server. This command will not output anything
# will redirect output to log file, and actual log file as well. You should inspect each 
# for errors.

psql --set ON_ERROR_STOP=on --dbname=postgres \
     --file=/opt/mayan-edms/media/mayan16_db_backup.sql \
     --log-file=/opt/mayan-edms/media/v16_db_import_log.txt


# (3.10) Update the environment file
# As we only need one line now all settings are in the config file in the media folder
mv /etc/mayan/env /etc/mayan/env.backup
echo "export MAYAN_MEDIA_ROOT=/opt/mayan-edms/media" > /etc/mayan/env


# (3.11) Migrate database static files to new version
ENV=/etc/mayan/env
BIN=/opt/mayan-edms/bin
su - mayan -c ". $ENV && $BIN/mayan-edms.py performupgrade"


# (3.12) Restart supervisor
systemctl start supervisor.service

 
# (3.13) Check that all is well
# Everything should be running. Check all users, workflows and documents
supervisorctl status


# Part 4 - Clean up
# Once you are sure everything is well so some clean up. We will remove the tar file,
# and copy over your config.yml with the config_backup.yml file as that should have all
# the updated settings, and remove any other files

# (4.1) Remove transferred tar file
rm ~/mayan16_media_files.tar.gz


# (4.2) Remove v16 database backup and log files
rm /opt/mayan-edms/media/mayan16_db_backup.sql
rm /opt/mayan-edms/media/v16_db_import_log.txt


# (4.3) Remove backup config files 
rm /opt/mayan-edms/media/backup.cfg
rm ~/v17_config.yml


# (4.4) Copy over new config from backup
# Because Mayan will read the config.yml file for settings and anything not in the file
# will be default values, (like new settings added since the config file was migrated
# ove), those default values will not be stored in the migrated config.yml file. to fix 
# this we will use the config_backup.yml file. As said before when Mayan is launched
# successfully it will dump all the running settings (including new ones) in the
# config_backup.yml file. So this file is the last known all working settings. We 
# can use this to update our config file.


# (4.4.1) - Stop services
supervisorctl stop all

# (4.4.2) - Copy config file from backup
mv /opt/mayan-edms/media/config.yml /opt/mayan-edms/media/config.yml.backup
mv /opt/mayan-edms/media/config_backup.yml /opt/mayan-edms/media/config.yml

# (4.4.3) - Restart services, check system
supervisorctl start all

# (4.4.4) - Check to see if config files are the same
diff /opt/mayan-edms/media/config.yml /opt/mayan-edms/media/config_backup.yml

# (4.4.5) - Remove backup config file
rm /opt/mayan-edms/media/config.yml.backup


# (4.5) Migration Complete
# You are done! If you found this helpful leave a comment, or you run into trouble let 
# me know as well. These guides take many days to write and test so let me know if they 
# are helpful.