digg_de's picture
digg_de - Mon, 2015/05/04 - 11:55

Hi,

once a week my Turnkey LAMP hangs up at night. In this time there are no user or application processes in work. How can i check what causes the hang-up? Which logfiles can give me a help? Can i enable more logging to find the problem?

Thanks for help!
Bye g.g.

 

Forum: 
Support
Jeremy Davis's picture

Jeremy Davis - Mon, 2015/05/04 - 16:50
And there are many different monitoring options. Apps that you can install like monit and third party web based apps too. If you keep in mind that TurnKey is Debian under the hood then that should help you research options and what would suit your needs best.
digg_de's picture

digg_de - Mon, 2015/05/04 - 21:26

Thanks again for your fast answer! I now installed monit with monitoring service for rsyslog, apache2, mysql, ssh, postfix, proftp and webmin. Hopefully i will find the problem?

Bye g.g.

OnePressTech's picture

OnePressTech - Tue, 2015/05/05 - 09:01

Hi GG,

Dumb question... you say that the LAMP stack "hangs once a week" but that there are no "user or application processes in work". The term "hanging" implies the server was doing something noticeable then stopped doing that noticeable thing. To do something noticeable requires some software to do something. So your LAMP stack can't "hang" and have no user or application processes running...

How did you detect the LAMP stack was "hanging"?

Could you restate your problem, how you detect it, and how it appears visibly. It will help us better suggest the root cause of your problem. For example, if your software is doing a large backup via CRON job on an AWS micro-instance then you will chew through your CPU allocation and Amazon will shut it down for a while. Having Monit running won't be much help in that instance because Amazon will shut it down as well (in all other cases Monit is an excellent suggestion Jeremy :-)

Cheers,

Tim (Managing Director - OnePressTech)

digg_de's picture

digg_de - Tue, 2015/05/05 - 10:41

Hi Tim,

yes you are right. Normally, if there is no application running the LAMP stack can't hang.

I detect the following:
- VMware Player: no input more possible in the "Turnkex Linux Configuration Console"
- SSH: no more reaction, no login possible, server not reachable
- APACHE: no more reaction, server not reachable
- ProFTP: no more reaction, server not reachable
- Postfix: no email messages were sent
- Cron-Jobs: no jobs were executed

The only way is now to power off from the VMware player. After restarting everything works fine. The last entries in the syslog are these:

May  3 04:09:01 lamp /USR/SBIN/CRON[24565]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
May  3 04:17:01 lamp /USR/SBIN/CRON[24577]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
May  3 04:39:01 lamp /USR/SBIN/CRON[24580]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))

I'm looking now for a possibility the find the causes or what's happens between 04:39:00 and 05:09:00?

The LAMP Stack is not running in Amazon Cloud. It runs on our own VMWare Server System. There are more VM's running in the same time. All other VM's works fine.

No, i can not restate the problem. I have to wait for the next hang up.

Additionally i contact the administrator of our VMware Server to find out, what happens between 04:39:01 and 05:09:00 on 3rd of may.

Thanks for help!
Bye g.g.

OnePressTech's picture

OnePressTech - Tue, 2015/05/05 - 10:58

Hi GG,

Ok...knowing that the scenario in question involves a vmware player changes the whole diagnosis process.

Check your disk space. If you have been doing a lot of debugging and the log files have chewed up a lot of your disk your swap file might not be able to get enough diskspace to operate. The fact that it hangs once a week would point me to logfiles and diskspace first. What happens is that if you are running with lean RAM in the VM there is a heavy reliance by the O/S on the swapfile. If this is a reproducible once a week hang I would just log in once a day and run TOP and write down the resource utilisation keeping an eye on logfiles and disk space.

Cheers,

Tim (Managing Director - OnePressTech)

digg_de's picture

digg_de - Tue, 2015/05/05 - 11:02

Hi Tim,

at first, thanks for your help!!!

Yes, it is a dedicated server. Yes, i will check all the logs on the server, but i think disk space is not the problem.

No, it is not reproducible one a week. It was an estimation. But i will check it and will come back with more informations.

Thanks g.g.

digg_de's picture

digg_de - Tue, 2015/05/05 - 19:30

The only process on the server i found in this time slot was a backup process of all VMs. I excluded temperally the Turnkey LAMP VM from these general backup. In all other logs on the server is no hint for my problem, no problems with sufficient disk space or other issues. ... Now i have to wait for the next hang up ...

OnePressTech's picture

OnePressTech - Wed, 2015/05/06 - 00:56

Do you have TKLBAM configured to do a remote backup of the LAMP appliance to AWS or other external location? If so, is TKLBAM scheduled to run around the same time as you VM backup?

Cheers,

Tim (Managing Director - OnePressTech)

digg_de's picture

digg_de - Wed, 2015/05/06 - 07:23

No, i dont use TKLBAM.

Add new comment