Restoring from a large (7GB+) backup fails

Ken Wingerden - Thu, 2011/09/15 - 03:35

One of our servers has over 7GBs of backup data, mostly due to content (images, videos, etc) loaded into the content management system. When attempting to launch a new VM from a backup of this system with hub-launch, the restore seems to be running smoothly, but eventually stalls. After manually running the restore from the command-line, it was clear why the hub-launch was just stalling - the root '/' partition ran out of space.

Restoring duplicity archive from s3://s3-us-west-1.amazonaws.com/tklbam-gkp7dhx45incclo2
Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20110914T195922Z.sigtar to local cache.
Copying duplicity-full.20110914T195922Z.manifest to local cache.
Last full backup date: Wed Sep 14 19:59:22 2011
Traceback (most recent call last):
  File "/usr/lib/tklbam/deps/bin/duplicity", line 1252, in <module>
    with_tempdir(main)
  File "/usr/lib/tklbam/deps/bin/duplicity", line 1245, in with_tempdir
    fn()
  File "/usr/lib/tklbam/deps/bin/duplicity", line 1199, in main
    restore(col_stats)
  File "/usr/lib/tklbam/deps/bin/duplicity", line 539, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/patchdir.py", line 522, in Write_ROPaths
    ITR( ropath.index, ropath )
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/lazy.py", line 335, in __call__
    last_branch.fast_process, args)
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/robust.py", line 37, in check_common_error
    return function(*args)
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/patchdir.py", line 575, in fast_process
    ropath.copy( self.base_path.new_index( index ) )
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/path.py", line 416, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib/tklbam/deps/lib/python2.6/site-packages/duplicity/path.py", line 595, in writefileobj
    fout.write(buf)

IOError: [Errno 28] No space left on device
Apparently duplicity downloads and unpacks the backup to the /tmp directory, which happens to live on the root partition (the way TKL Ubuntu Server is configured). 

I attempted a quick-and-dirty solution (or so I thought) by logging into the newly launched server and running the following commands:

	mkdir /mnt/tmp
chmod 777 /mnt/tmp
chmod +t /mnt/tmp
rm -rf /tmp
ln -s /mnt/tmp /tmp
tklbam-restore [backup id]

but I ran into the same problem.

I am hoping that there is a solution that would allow me to continue to use hub-launch without having to perform the restore manually, but would live with a quick-and-dirty manual solution that actually works.

Thanks!

-Ken

Forum: 
Support
Tags: 
tklbam