Martin's picture

Hello,

I tried to set-up autosuspend with turnkey nexcloud. The server should go to energy save when no user is logged in to nexcloud.

It works so far with Webmin at port 12321. But the https port 443 is not recognised by autosuspend. Although the port is listed "Establised" with lsof -i P -n.

Any help is highly appreciated.

Martin

Forum: 
Jeremy Davis's picture

Apologies on such a slow response...

Anyway, you'll need to provide a bit more info if you want a hand with this. Firstly, I assume that you're using a v16.x TurnKey appliance. Also, you mention "autosuspend", but don't give any further details. I assume that you mean this autosuspend? Perhaps installed from the Debian repos via apt?

If I understand correctly, you are wanting network port access to trigger wake up? My quick browse of the autosuspend; available wakeups docs suggests that OOTB, that's not an option with 'autosuspend'. However, AFAIK ethtool (should be installed by default on all TurnKey servers) can support wake up from a range of events. So perhaps the 2 can be strung together to achive your desired outcome?

The Debian WoL (Wake on LAN) wiki page has some info, although that mostly focuses on the "MagicPacket" method. The Arch wiki (Arch is a completely different Linux OS, but often has the best, most up to date technical info about common Linux tools) has a great WoL page that may be of value? Regardless though, any of the network related, non-"MagicPacket" methods, you'll likely need to ensure that you have some ARP rules set up in your router (so your router knows where to direct the packets intended for your machine).

So bottom line is, that without knowing much more about what you've tried and what config you've already done, I'm really only just guessing... So if you can provide more info, I'm happy to do what I can to help...

Martin's picture

Sorry for being not precise.

- It should be TurnKey 16.0. I downloaded the file   "turnkey-nextcloud-16.0-buster-amd64.iso" in December 2020 and installed it. I do not know where to find the version of the TurnKey applicance.

- I installed autosuspend from the Debian repository

- Wakeup with ethtool and a magic packet from a windows client works well.

- I think the issue is that https connections are not always active but only occasionaly set the port as established. Autosuspend also checks periodically for an active connection. So there is a high probabilty that a coincidence of the two situations never happen and autosuspend is not able to recognise the connection.

I am not a programmer. Just a not really experienced linux user. With ideas and proposals of the autosupend github forum and some snippets from the web  I set up the bash scipt below. As apache2 continously writes its connection to a log file the scipt could monitor if there is still activity writing on the log file and thus an active NextCloud connecion. It works now fine for me.

Although I do not understand why the file /var/log/apache2/access.log is empty but the log is written in in /var/log/apache2/other_vhosts_access.log.

The method does not work for the webmin https connection as it is not written in the apache2  access log. Is there any other log file of webmin I could monitor for an active connection?

Thanks Martin

----------------

#!/bin/bash

# Get parameters from command line
LogFile=$1
NoActivity=$2
Debug="off" ; # preset Debug to off
Debug=$3 ; # Debug if 3rd parameter equals debug

# Get system time and time of last log file file modify
SystemTime=$(date +%s)
FileModify=$(date -r $LogFile +%s)
TimeElapsed="$(($SystemTime-$FileModify))"

# Check if no activity
if [ $TimeElapsed -gt $NoActivity ]
    then
        Return=1
    else
        Return=0
fi

#Debug
if [ "$Debug" = "debug" ]
then
    echo "----------debug CheckActivty.sh-----------------------"
    echo $LogFile
    echo SystemTime $SystemTime
    echo FileModify $FileModify
    echo TimeElapsed $TimeElapsed
    echo NoActivity threshold $NoActivity
    echo ReturnValue $Return "(0 = Nextcloud active / 1 = Nextcloud inactive)"
    echo "---------------------------------"
fi

exit $Return
Jeremy Davis's picture

You've provided plenty of info now, thanks.

It's irrelevant now (you'll see why below) but FYI you can get the "TurnKey version" from an appliance like this:

turnkey-version

In the case of your server, you'd get this:

turnkey-nextcloud-16.0-buster-amd64

As you'll note, it's the same as the ISO file name, hence why I said it was irrelevant... :)

Anyway, great work on your progress. FWIW I wasn't even aware that waking a computer from sleep without a "Magic Packet" was even possible! So thanks for educating me! :)

Re your questions on the Apache log. It might be nice to think about how we might tidy this up a little, although TBH, it doesn;t really cause too much pain I don't think?! Anyway, AFAIK it's pretty much the default config that ships with Debian, plus the Nextcloud Apache config that we provide that makes it log the way that it does.

At the top of /etc/apache2/sites-available/default-ssl.conf you'll find the following:

<IfModule mod_ssl.c>
	<VirtualHost _default_:443>
		ServerAdmin webmaster@localhost

		DocumentRoot /var/www/html

		# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
		# error, crit, alert, emerg.
		# It is also possible to configure the loglevel for particular
		# modules, e.g.
		#LogLevel info ssl:warn

		ErrorLog ${APACHE_LOG_DIR}/error.log
		CustomLog ${APACHE_LOG_DIR}/access.log combined

Note that the last 2 lines there are defining the default log locations. There's the access.log! :) But if you poke around in the Apache config a bit more, you'll find another CustomLog defined in /etc/apache2/conf-available/other-vhosts-access-log.conf (/etc/apache2/conf-enabled/other-vhosts-access-log.conf is a symlink to this file, as only the files in /etc/apache2/conf-enabled are read - it works similar with sites-available and sites-enabled). Here's other-vhosts-access-log.conf:

# Define an access log for VirtualHosts that don't define their own logfile
CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

So that sets the other_vhosts_access.log file to be a "catchall" for all vhosts (aka sites) which don't explicitly define a access log location. This overwrites the previously set default access.log file (although the file itself is still created, albeit empty). As we don't define a log location for the CustomLog in the Nextcloud site file (/etc/apache2/sites-available/nextcloud.conf) it falls back to logging in the other_vhosts_access.log file.

Re Webmin, you should find the Webmn logs (or more precisely symlinks to them) in /var/log/webmin. Although please note that Webmin (and Webshell too) are hidden behind stunnel (stunnel is a light weight SSL tunnel that we use as a SSL termination frontend for Webmin). So clients connect to stunnel (on port 12321) which then connects to localhost:10000 (which is what Webmin listens on). So I'm not 100% sure that scanning the Webmin logs will give you the info that you are looking for. By default stunnel has logging disabled, but you could enable it if you wanted which I suspect would likely provide the info you want.

FWIW personally, I reckon that the best way to manage this would be actually monitoring the conenctions, rather than the logs of conenctions. You can get the info about connections using one of the built-in tools. 'ss' works quite well, e.g. something like this (ss -tu) might give you good start?!:

root@lamp ~# ss -tu
Netid              State              Recv-Q               Send-Q                             Local Address:Port                               Peer Address:Port                
tcp                ESTAB              0                    0                                      127.0.0.1:webmin                                127.0.0.1:35892               
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:38902               
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:39616               
tcp                ESTAB              0                    0                                  192.168.1.109:12321                              192.168.1.56:46326               
tcp                ESTAB              0                    0                                      127.0.0.1:35892                                 127.0.0.1:webmin              
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:39776             

Running this on my test (LAMP) server (192.168.1.109) shows that from my laptop (192.168.1.56) I have multiple connections (to my test server). 3x SSH connections and Webmin. If you filter those results so that it only shows remote connections, then you would have something of value IMO.

As we know that the local external IP address (in my case 192.168.1.109) if we collect all the active connections to/from that IP would be a good start. Although 127.0.0.1 is localhost so actually connections between anything and that are internal so have no interest to us here. We can use grep to filter those lines in/out. This should give us just the relevant lines. E.g. something like this (grep's -v switch excludes matches):

ss -tu | grep 192.168.1.109 | grep -v 127.0.0.1

Here it is on hte data from above:

root@lamp ~# ss -tu | grep 192.168.1.109 | grep -v 127.0.0.1                      
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:38902               
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:39616               
tcp                ESTAB              0                    0                                  192.168.1.109:12321                              192.168.1.56:46326               
tcp                ESTAB              0                    0                                  192.168.1.109:ssh                                192.168.1.56:39776  

So if I close all the connections, it should return nothing (FWIW I normally connect to my servers via SSH, as it gives a nie native user experience - so this step required me to log in via the actual VM window):

root@lamp ~# ss -tu | grep 192.168.1.109 | grep -v 127.0.0.1

Yep - it worked! :) The beauty of doing it that way is that even if you install a new service, it will still monitor the activity of the new server without you needing to update your list of log locations to monitor.

FWIW, here's a vaguely similar script similar to what you are using, but this one leverages ss. I also took the liberty of writing it in the style I tend to use and will add some notes at the end.:

#!/bin/bash -e

[[ -z "$DEBUG" ]] || set -x
TIMEOUT=${TIMEOUT:-30} # seconds

connections=$(ss -tu | grep 192.168.1.109 | grep -v 127.0.0.1) || true

if [[ -z "$connections" ]]; then
    sleep $timeout
    exit
fi
exit 1

Notes:

  1. It's always good practice to set 'e' for all bash scripts. If you set 'e' (either at the top as I did or via 'set -e') then your script will exit (with exit code 1) if it encounters an error. Otherwise it will ignore the error and just keep executing the next line.
  2. Rather than having a commandline debug option, I prefer to use an env var (sometimes I include both). The beauty of using a 'DEBUG' env var is that if you have lots of scripts that you string together, to debug you can just set DEBUG and they all will have debug enabled. Also, I just set 'x' rather than have an explicit debug output. You can add some as well if you wish, but usually the verbose output that 'x' gives is enough (I'll post the output below).
  3. I set a timeout here, although you could collect it as input if you rather. FWIW, the way that I've done it means that you can also preset this with an env var too if you want. I.e. if $TIMEOUT is set, that will override the built in default (of 30 seconds).
  4. This line captures the output of the ss command I gave above. Obviously you'll need to change the external IP of your server. Note that it has the '|| true' (OR True) on the end to ensure that the line never errors. If grep does not find a match it will return a non zero exit code so with no connections and without the '|| true', the script will "exit 1" on this line.
  5. Test if $connections is empty.
  6. (If no connections) Wait $TIMEOUT seconds.
  7. (If no connections) Exit with exit code 99. I used that instead as '1' is the default error code and we've set 'e', Using an alternate non-zero exit code here to signal that there are no connections (i.e. '99') allows you to easily differentiate between the desired result (exit codes of either '0' or '99') vs actual errors (likely '1', but could potentially be others).

I didn't put an 'exit" (or even an 'exit 0') at the end as it's redundant and overly verbose IMO (the script will always exit '0' unless there is an error).

Also, as promised, here is the script run via an SSH session (with just the SSH session connected). Note that I set $DEBUG and a custom TIMEOUT, plus gather the exit code afterwards (as you can see, I called my script 'test'):

root@lamp ~# TIMEOUT=1 DEBUG=y ./test; echo $?
+ TIMEOUT=1
++ ss -tu
++ grep 192.168.1.109
++ grep -v 127.0.0.1
+ connections='tcp     ESTAB   0        200        192.168.1.109:ssh      192.168.1.56:44354   '
+ [[ -z tcp     ESTAB   0        200        192.168.1.109:ssh      192.168.1.56:44354    ]]
0

Now run from the VM window (with no network connections):

root@lamp ~# TIMEOUT=1 DEBUG=y ./test; echo $?
+ TIMEOUT=1
++ ss -tu
++ grep 192.168.1.109
++ grep -v 127.0.0.1
+ connections=
+ [[ -z '' ]]
+ sleep 1
+ exit 99
99

Anyway, I've rambled enough now... :) Hope that's of some value. Good luck with it all and hopefully chat another day...

Martin's picture

Jeremy,

Hope this is not against the etikette of this forum. But thank you really for your valuable solution provided.

Just for refernce (I also referenced this in the autosuspend forum on Github)

It works great and gave me some learning on Bash programming as well. So I could slightly adapt it to the code given below for more flexibility. I only need to add the IP adress of the server as a parameter on the command line. I am pretty sure this can be derived out of the linux system automatically by more advanced linux users. Then the script is fully flexible.

Thanks again for great help and support!

--------

#!/bin/bash -e

# start with /<path>/TestConnection.sh <"ServerIP> [y]   (y for debug)

ServerIP=$1  #get server IP from command line
DEBUG=$2     #get if debug from commeand line
#TIMEOUT=$3  #not really necessary
TIMEOUT=1

echo ----- test for active connections at $ServerIP

# if DEBUG is zero, set to echo commamds (-x)
[[ -z "$DEBUG" ]] || set -x
TIMEOUT=${TIMEOUT:-30} # seconds

# look for active external onnections of $Server by ss -tu and grep lines containing $Server,
connections=$(ss -tu | grep $ServerIP | grep -v 127.0.0.1) || true # set to true if grep has an error

# If length connections = zero, then no connection, set return value non zero for autosuspend
if [[ -z "$connections" ]]; then
    sleep $TIMEOUT
    echo ----- no connections  return value = 99
    exit 99
fi

# active connection return value = 0
echo ----- connections to $ServerIP return value = 0
exit 0

 

Jeremy Davis's picture

Thanks for your kind words and posting back with your updated script. I'm also really glad that my post was useful to you. Chat some more another day hopefully! :)

Add new comment