Bill Carney's picture
Bill Carney - Sun, 2013/10/13 - 16:34

I am livid right now.  After having some issues with a virtual machine running 30+ WordPress installations on a TKL LAMP installation, I concluded my only option was to restore a backup from Thursday.  I restored the backup from the Hub.  To my horrror I found that only eight of the 30 databases were restored!  The rest are now GONE!

My faith in this product has been shattered.  I can no longer rely upon it.  What is the point of a "backup" product that DOESN'T BACK UP DATA?!?!?

Forum: 
Support
Tags: 
lamp
tklbam
mysql
data
Liraz Siri's picture

Liraz Siri - Mon, 2013/10/14 - 19:32
Sorry to hear about the trouble you're having. I understand your frustration. Unfortunately everything can fail so put your faith in testing and test your backups before you need them. In any case, your data might not actually be gone. You may just be having a problem with the database restore process. Take a look at the backup logs to see what databases and database tables were being backed up.

Also, we're updating the Squeeze package archive today with a new version of TKLBAM (v1.3) that should make it easier to diagnose the issue you're having. With the new version try downloading your backup to a raw directory like this:: 

tklam-restore --raw-download=/tmp/mybackup your-backup-id
Then take a look into /tmp/mybackup/myfs to see if your databases are still there. If there's a bug we need to look into then I'll need more specifics than what you have provided to isolate and fix it.
Bill Carney's picture

Bill Carney - Mon, 2013/10/14 - 23:41

 

Unzip Turnkey LAMP stack as a VM on my local machine, ran in VMWare Fusion.
 
Set up VM using same root PWs that I used on my original machine
 
Skipped TKLBAM initialization, will do that later.
 
Installed Security Updates
 
Logged in via Shell in a Box
 
Init using "tklbam-init (snip)"
 
apt-get upgrade tklbam-restore
 
Attempted Restored using
tklbam-restore 11 --raw-download=/tmp/mybackup --time 2013-10-08T06:52:51
 
resulted in:
 
Copying duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg to local cache.                                                                                          
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg failed (attempt #1, reason: IncompleteRead: IncompleteRead
(0 bytes read, 3776027 more expected))                                                                                                                                                    
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg failed (attempt #2, reason: IncompleteRead: IncompleteRead
(0 bytes read, 3776027 more expected))                                                                                                                                                    
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg failed (attempt #3, reason: IncompleteRead: IncompleteRead
(0 bytes read, 3776027 more expected))                                                                                                                                                    
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg failed (attempt #4, reason: IncompleteRead: IncompleteRead
(0 bytes read, 3776027 more expected))                                                                                                                                                    
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg failed (attempt #5, reason: IncompleteRead: IncompleteRead
(0 bytes read, 3776027 more expected))                                                                                                                                                    
Giving up trying to download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg after 5 attempts                      
BackendException: Error downloading s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-new-signatures.20131008T065251Z.to.20131010T063557Z.sigtar.gpg                                
Traceback (most recent call last):                                                                                                                                                        
  File "/usr/bin/tklbam-restore", line 440, in <module>                                                                                                                                   
    main()                                                                                                                                                                                
  File "/usr/bin/tklbam-restore", line 383, in main                                                                                                                                       
    get_backup_extract()                                                                                                                                                                  
  File "/usr/bin/tklbam-restore", line 379, in get_backup_extract                                                                                                                         
    downloader(raw_download_path, target)                                                                                                                                                 
  File "/usr/lib/tklbam/duplicity.py", line 138, in __call__                                                                                                                              
    Duplicity(opts, '--s3-unencrypted-connection', target.address, download_path).run(target.secret, target.credentials)                                                                  
  File "/usr/lib/tklbam/duplicity.py", line 86, in run                                                                                                                                    
    raise Error("non-zero exitcode (%d) from backup command: %s" % (exitcode, str(self)))                                                                                                 
duplicity.Error: non-zero exitcode (23) from backup command: duplicity --restore-time=2013-10-08T06:52:51 --archive-dir=/var/cache/duplicity --s3-unencrypted-connection s3://s3.amazonaws
.com/tklbam-jfdkqj3btc5boko2 /tmp/mybackup                         
 
 
I'll give it a try later at a different location with a faster connection; where I'm at now this process took two hours.
Bill Carney's picture

Bill Carney - Tue, 2013/10/15 - 02:30

Every restore I tried this evening generated 500 Internal Server errors.  I am unable to restore any backup.

 

I tried restoring the backup from above a second time again to no avail:
 
Copying duplicity-new-signatures.20131013T192312Z.to.20131014T071023Z.sigtar.gpg to local cache.                                           
Last full backup date: Thu Oct  3 07:21:54 2013                                                                                            
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-full.20131003T072154Z.vol1.difftar.gpg failed (attempt #1, reason: BotoSer
verError: BotoServerError: 500 Internal Server Error                                                                                       
<?xml version="1.0" encoding="UTF-8"?>                                                                                                     
<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>09486C2B454E8D2D</Request
Id><HostId>wfjfRKDOqRyIfwQMtkBiceXHdNB3sxYpFwrjhXNQYyS50+ZlDw+WE0gsOjnL3C8s</HostId></Error>)       
 
 
I tried a restore from 25 September.  This time I did not run any updates upon first boot to try to save some time.  Still no joy.
 
tklbam-restore 11 --raw-download=/tmp/mybackup --time 2013-09-25T06:35:37
 
Copying duplicity-new-signatures.20131013T192312Z.to.20131014T071023Z.sigtar.gpg to local cache.                                           
Last full backup date: Thu Oct  3 07:21:54 2013                                                                                            
Download s3://s3.amazonaws.com/tklbam-jfdkqj3btc5boko2/duplicity-full.20130827T072409Z.vol1.difftar.gpg failed (attempt #1, reason: BotoSer
verError: BotoServerError: 500 Internal Server Error                                                                                       
<?xml version="1.0" encoding="UTF-8"?>                                                                                                     
<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>532B6B0FC7570F29</Request
Id><HostId>I9SD5W8nMDgslFHbadvg/ANlKFuk6Q4dSkuwsfzvwkdE+px6V0+e5AIG7wDsUdDV</HostId></Error>)  
Liraz Siri's picture

Liraz Siri - Tue, 2013/10/15 - 05:01

Well, it looks like you can't reach Amazon S3. Either that or S3 is really unreachable which would be the first time I've seen that. Do you have a proxy/firewall on the network that might be interfering with your access to AWS?

Bill Carney's picture

Bill Carney - Tue, 2013/10/15 - 05:36

No, I was able to connect with S3 yesterday when I first tried the restore (the one that didn't restore all the MySQL databaes).  I'm attempting this at home with my non-firewalled residential connnection.  It worked yesterday, not sure why it's not working today.  I even tried restoring a backup from August, yet the restore functions insist on going through this Oct 3 backup, which causes this 500 error.

Bill Carney's picture

Bill Carney - Tue, 2013/10/15 - 17:51

Is there a way to delete or skip a backup when restoring?  It's this nearly 1GB full backup I made on Oct 3 that appears to be causing my restore issues.

Jeremy Davis's picture

Jeremy Davis - Sun, 2013/11/10 - 14:46

Incremental backups rely on the full chain of backups (up to and including the previous full backup). So if the backup in question is part of the chain then the only way to avoid it is restore from a date prior to the problem one... In other words, if the backup from that date is part of an incremental backup chain then you can only ignore it if you go for a backup prior to that date (i.e. Oct 2 or before...). 

If it is a full backup then it doesn't matter. 

Add new comment