You are here
Seth Berrier - Wed, 2021/11/10 - 16:59
I am experiencing very strange behavior on one of my new LAMP servers. This may not be Turnkey specific but I figure I'd start here and see if you all have any ideas!
Here's what I'm observing:
- When making outgoing web requests, the connection sometimes just times out or stalls indefinitely with no response.
- Repeating the request immediately afterward usually causes it to succeed.
- I'm not certain, but the problem looks like it might only happen with TLS connections (HTTPS)
-
This seems to be related to multiple requests going to the same server.
- Running an 'npm install' is a good example, a few requests in and it will suddenly stall and eventually (after a good 5 minutes) time out
- If I SIGTERM the process and try again it gets a little further and then times out again
- As of now, I cannot get any 'npm install' to complete successfully as there are SOOO many timed-out connections (hundreds).
- 'apt-get update' is another good example, often fails and has to be re-run (I can usually coax this into completing)
I'm kind of at a loss to explain this. There is a chance it might be related to the larger network and not just my server but I wanted to do my best to explore software solutions before I contact the network administrators.
Any thoughts?
-----------------------
Here's an example showing this behavior. It stalls at deb.nodesource.com (while running the nodesource repo setup script) and after I cancel it and try again manually, it works fine. This one is VERY consistent:
+ apt-get update Hit:1 http://security.debian.org buster/updates InRelease Hit:2 http://deb.debian.org/debian buster InRelease Ign:3 http://archive.turnkeylinux.org/debian buster-security InRelease Ign:4 http://archive.turnkeylinux.org/debian buster InRelease Hit:5 http://archive.turnkeylinux.org/debian buster-security Release Hit:6 http://archive.turnkeylinux.org/debian buster Release 0% [Connecting to deb.nodesource.com] *much time passes* ^C berriers@games ~$ sudo apt-get update [sudo] password for berriers: Hit:1 http://security.debian.org buster/updates InRelease Hit:2 http://deb.debian.org/debian buster InRelease Get:3 https://deb.nodesource.com/node_14.x buster InRelease [4584 B] Get:4 https://deb.nodesource.com/node_14.x buster/main amd64 Packages [768 B] Ign:5 http://archive.turnkeylinux.org/debian buster-security InRelease Ign:6 http://archive.turnkeylinux.org/debian buster InRelease Hit:7 http://archive.turnkeylinux.org/debian buster-security Release Hit:8 http://archive.turnkeylinux.org/debian buster Release Fetched 5352 B in 1s (3716 B/s) Reading package lists... Done
Forum:
NPM Install Example
Here's a log of me trying to run 'npm install' on a complex node.js project (aren't they all). This one does give back some error info that makes it look to me like it's struggling to resolve the server name in some way, despite having contacted it repeatedly just moments ago:
It looks like the forum ate your 2nd post
Unfortunately, it looks like the forums ate your second post.
Regardless, I note from the apt output that it's the deb.nodesource.com URL that is causing the issue. And you also note that it only seems to be a problem with https. My suspicion is that it's related to the recent expiry of the Let's Encrypt root certificate.
Following a little more online research, it seems that nodesource uses Akami CDN and they appear to have been affected. Apparently Akami had some plans to mitigate it but it seems that nodesource is still affected. I reckon that the reason why it's intermittent is because they have round robin DNS and some nodes have been fixed, but others haven't, so it depends whether you hit a broken one or a fixed one.
I suggest that you have a browse through the relevant GitHub issue. Hopefully someone has posted some potential workarounds that you can try.
I suggest that you
Add new comment