It looks like when your server was loading all results from it's database, it used too much RAM. As per the message, it tried to allocate another ~16MB but that exceeded the allowed memory usage (of ~530MB).

My assumption is that it hit the limit allowed in the php.ini file. Note that there are 2 files, one for Apache and one for CLI. In this case you'll want to edit the CLI file (because when you hit the error you were running from the commandline) but it may pay to double check the Apache limit too.

In a v16.x appliance, the 2 files should be:

/etc/php/7.3/apache2/php.ini
/etc/php/7.3/cli/php.ini

As per the file path, the one you want to edit is the second one and look for the 'memory_limit' line and bump it up higher. It sounds like it's ~500MB currently, but needs to be higher. I would suggest adding an extra 64MB and see how that goes.

Alternatively, if you just wish to temporarily remove the PHP RAM limit while you run the command you're trying to run, you should be able do it ad hoc like this:

php -d memory_limit=-1 /opt/observium/check-errors.php

Any changes in PHP config require restart (and as in our default config PHP runs as an Apache module, that means restarting Apache).

That should do the trick...

I've had a quick look and I can recreate the issue (again) - even on the latest release. Here's what I did to fix it. Hopefully it works for you.

Enable the 'non-free' repo:

sed -i "\|non-free| s|^#||" /etc/apt/sources.list.d/sources.list
sed -i "\|non-free| s|^#||" /etc/apt/sources.list.d/security.sources.list

Update package lists and install 'snmp-mibs-downloader':

apt update
apt install snmp-mibs-downloader

That patches some of the mibs files on install and that seemed to resolve the issue for me?!:

root@jed-test-observium /opt/observium# ./discovery.php -u

  ___   _                              _
 / _ \ | |__   ___   ___  _ __ __   __(_) _   _  _ __ ___
| | | || '_ \ / __| / _ \| '__|\ \ / /| || | | || '_ ` _ \
| |_| || |_) |\__ \|  __/| |    \ V / | || |_| || | | | | |
 \___/ |_.__/ |___/ \___||_|     \_/  |_| \__,_||_| |_| |_|
                     Observium Community Edition 20.9.10731
                                  https://www.observium.org

-- Database is up to date.

But I also downloaded mibs too:

download-mibs

And all still seems well:

root@jed-test-observium /opt/observium# ./discovery.php -u

  ___   _                              _
 / _ \ | |__   ___   ___  _ __ __   __(_) _   _  _ __ ___
| | | || '_ \ / __| / _ \| '__|\ \ / /| || | | || '_ ` _ \
| |_| || |_) |\__ \|  __/| |    \ V / | || |_| || | | | | |
 \___/ |_.__/ |___/ \___||_|     \_/  |_| \__,_||_| |_| |_|
                     Observium Community Edition 20.9.10731
                                  https://www.observium.org

-- Database is up to date.

I hope that helps.

PS it turns out that restarting Apache isn't/wasn't required anyway... I guess PHP only calls SNMP as a separate process?!

I'm not sure why their default install works and ours didn't? We do a pretty default install. As for the mibs changes, the snmp-mibs-downloader package page says:

This package ships the IETF RFCs containing MIB files and a script which extracts them to be used by Simple Network Management libraries. The script can be used to update some MIBs to the latest version or to download extra vendor MIBs.

I know that Observium includes a whole heap of mibs (in /opt/observium/mibs/) but AFAIK it also relies on system mibs too.

Anyway, glad that's got you going. Please report back if you have any further issues or questions.