RyanF's picture

When connecting to the ruTorrent web interface of a Turnkey Torrent Server I get the following message:

"Torrent list not yet available, connection to rTorrent not established."

the log shows the following:

[02.08.2016 17:42:24] WebUI started.
[02.08.2016 17:42:24] No connection to rTorrent. Check if it is really running. Check $scgi_port and $scgi_host settings in config.php and scgi_port in rTorrent configuration file.

Sure enough, when I check in 'Bootup and Shutdown' in the webin I see that rTorrent isn't running. clicking 'Start Now' yeilds:

Executing /etc/init.d/rtorrent start ..

Starting rtorrent.
rtorrent started successfully.


but afterwards it still shows rTorrent isn't running and the web interface still wont connect.

Rebooting the machine has no effect. This is a fresh install of the Turnkey Torrent Server on a 32BG thumb drive in a USB2 port, and the only modifications i've made were to the rtorrent.rc file. The server worked as expected for several reboots after the changes to this file until this error occured.

Ive tried completely reinstalling the server and I get the same error after rebooting a few times. Deleting the rtorrent.lock file hasn't helped either.

What am I missing?

Forum: 
Tags: 
Jeremy Davis's picture

It does appear that there is an issue with our torrentserver appliance. You are the 2nd person (see this bug on our issue tracker) in as many days to raise the alarm.

Unfortunately I'm pretty snowed under at the moment so won't be able to look into it further until next week at the earliest. In the meantime you can try troubleshooting as per what I posted yesterday (on the bug). Otherwise if you want this up and running ASAP, you may need to follow the bug reporter's lead and implement something yourself (perhaps as per my post a few minutes ago).

 

Hi guys,

As said before. I´m experiencing the same problem.

It was working fine after the installation.

I added a network interface and changed the hostname, and it sotped working.

As RyanF pointed out, it looks like rtorrent won´t start.

This is auth.log:

Aug 10 19:29:25 TORRENTSERVER sshd[534]: Server listening on 0.0.0.0 port 22.
Aug 10 19:29:25 TORRENTSERVER sshd[534]: Server listening on :: port 22.
Aug 10 19:29:30 TORRENTSERVER su[604]: Successful su for rtorrent by root
Aug 10 19:29:30 TORRENTSERVER su[604]: + ??? root:rtorrent
Aug 10 19:29:30 TORRENTSERVER perl: pam_unix(webmin:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=  user=root
Aug 10 19:29:30 TORRENTSERVER su[604]: pam_unix(su:session): session opened for user rtorrent by (uid=0)
Aug 10 19:29:30 TORRENTSERVER su[604]: pam_unix(su:session): session closed for user rtorrent
Aug 10 19:29:31 TORRENTSERVER webmin[561]: Webmin starting
Aug 10 19:31:29 TORRENTSERVER smbd[881]: pam_unix(samba:session): session closed for user nobody
Aug 10 19:31:29 TORRENTSERVER smbd[882]: pam_unix(samba:session): session closed for user nobody
Aug 10 19:35:21 TORRENTSERVER smbd[1228]: pam_unix(samba:session): session closed for user nobody
Aug 10 19:35:21 TORRENTSERVER smbd[1227]: pam_unix(samba:session): session closed for user nobody

 

My system:

turnkey-torrentserver-14.1-jessie-i386.iso

installed on VMware Workstation 11 with Workstation 5.x hardware compatibiliy

Please tell me if you need more data.

panchuz

Jeremy Davis's picture

The problem seems to be between rTorrent and some issue with hardware and/or specific torrent files and/or specific tracker sites. This means that I have been unable to reproduce the issue locally. I also note that there are a number of bug reports on Debian. And until I can reproduce the issue I have no idea how to resolve it.

One suggestion is that you uninstall the Debian rtorrent package and install a newer version of rtorrent from upstream and see if that solves the problem. If not, then the developer(s) may be able to help you pin point the issue?

Alternatively, you could install something else (e.g. Transmission headless) and use that for handling torrents instead. FWIW that's what we've decided to do and will re-release the appliance with Transmission instead of rTorrent at some point in the near future.

Lionel Harris's picture

So the appliance is broken on a fresh install when selecting to install updates.  In addition if you don't elect to install updates or have an existing install, it breaks itself on Automatic Update.  I read the bug report and it looks like you guys have realized this and you're deciding to move to transmission.  

I'd like to say that the only reason I was attracted to this distro was because it was rtorrent/rutorrent.   I prefer the ruTorrent web interface and for me 14.1 has been pretty damn solid up until this issue, I've not had a buggy experience as mentioned in the github bug.  While 14.0 had some issues, I would think that would be expected becuase if I remember correctly, that was the first release with rtorrent/rutorrent.  

I'm very sad to see you guys go in this direction and hope you'll reconsider, I think you guys have given up too easily.  I myself have never used Transmission and just Googled it to look it over... the top hit was that Transmision for Mac was just been infected with Malware a second time in six months.  While this really doesn't apply to a linux server and I would imagine your testing would catch something like malware within your own distro, I would rather just avoid Transmission for now.

Jeremy Davis's picture

Thanks tons for your feedback and perspective. We love feedback, even if it's not flattering! :)

Apologies in advance on the extensive essay below.

Perhaps let me give you some context. One of our aims is to provide ease of use. But as fundamental components, we aim to provide stability and security. Unfortunately in it's current state the rTorrent Debian package is not providing the former. And the most obvious and immediate way to work around that (using an upstream source install) is to reduce the probability of fulfilling the latter (no facility for auto security updates). The latter would also require the end user to engage in manual rebuilds and updates when rTorrent upstream releases security updates.

After spending significant time researching, it appears to me that even were we to install from upstream (putting aside the points above) that we still not be able to provide consistent stability across user platforms. rTorrent seems to be quite sensitive to hardware and software idiosyncrasies. As can be seen from their bug tracker, there are tons of seemingly random edge case bugs that effect some users but seem difficult to reproduce reliably by devs.

The current appliance uses rTorrent from the Debian package repository. TBH I'm not 100% sure why the auto security upgrade breaks it, but it does appear to for some users. FWIW I'm not one of them, I haven't been able to reproduce the issue myself but we've had enough reports now to be clear that there is a problem!

In fairness, as I mentioned elsewhere, I do not use the TKL Torrentserver appliance myself. FWIW my personal torrentserver uses Transmission and has been running reliably for about 4 years now. I have personally tried rTorrent/ruTorrent a couple of times over the years but always run into one issue or another. It appeared that this had changed (hence us introducing it for the v14.0 release), but it seems perhaps not as much as I had anticipated...?

Whilst the (built-in) Transmission web UI is not as powerful as ruTorrent, for general purposes (adding/removing torrents etc) it should be sufficient. Also Transmission is not as resource friendly as rTorrent but it is rock solid stable. It also has a pretty cool full UI client (that runs on Windows and Linux at least, not sure if it works with Mac too?)

WRT to your discovery of the Tranmission Mac upstream release malware infection, that's a reflection of our current times, the weaknesses in upstream publishing workflows and the lack of proper due diligence by end users. It's also symbolic of the popularity of Tranmission on Mac.

Unfortunately, these days downloading random programs from random websites (no matter how official they appear); without some sort of validity checking is fraught with danger. If you use randomly downloaded software which you haven't (or aren't able to) check properly, it's a matter of when you'll be infected, not if! Reality is that the same issue could potentially effect any upstream binary (including rTorrent) where their release procedure is insufficient (e.g. providing only MD5 checksums, not using PGP signed checksum files, etc) and/or end users diligence is limited (e.g. not checking SHA sums against PGP signed sum files).

As mentioned above, whenever possible, we use Debian packages, direct from the Debian archives. Behind the scenes, Debian build all their own binaries from source code (no reused random binaries). The Debian apt package system also integrates PGP package signing and verification into the build, download and install processes. Is it infallible? Of course not, but the bar is much higher and because it all happens automatically it is always used.

To put all that aside and address your final point ("you guys have given up too easily"): From my perspective, I need to keep a handle on ~100- different appliances and ensure that they all remain stable and secure. That's a pretty big task and there are plenty of gaps already! It often feels like running flat out just to stay in exactly the same place!

My experiences so far with rTorrent have been ok at best and woeful at worst. I understand that that is not everyone's experience. I know that we all have our favourite software and more often than not have objective reasons for our preferences (although I would argue, more often than not they are more subjective than we'd like to admit). I'm also sure that there will be some potential users that may not use our appliance if/when we switch to Transmission (or something else).

But for me; having an appliance that "just works", is reliable, stable and secure is the best possible situation. TBH the work to migrate to Tranmission has already been done so I am in no rush to reconsider rTorrent. However I am open to being convinced to change my mind. Be aware though, it will require you (or someone) to take on the Torrentserver appliance as a developer and provide the time and effort to resolve the current issues in a reasonable (decided by me) and reliable way. Ideally I'd also like a commitment to maintain it (at least the rTorrent component) into the future.

That may seem like a pretty big ask (and it is), but I would argue, no bigger ask than requesting that we reconsider moving to Transmission! :)

Thank you Jeremy for your reply.

Not able to reproduce? Really?

Feels like having caught a Picachu... :)

I like Transmission. I have used it in my router for a long time. So I will be happy to give the appliance another try when the change is done.

On the other hand, if any of you guys still want to have some fun, I can send you the VM (the Picachu!) using wetransfer.

Please let me know.

 

 

 

 

panchuz

cpugenesis's picture

Hi Guys.

I've installed my own transmisison servers in the past on ubuntu and centos and had stability issues on both, so thought I'd try turnkey / rtorrent instead.

In setup it started to work then started to see this issue:

 "Torrent list not yet available, connection to rTorrent not established."

For me, even when I ran rtorrent itself, I could see the rtorrent console but still saw the above error in rutorrent.

 

The below has got it up and running for me, allthough need to do some extensive testing now:


* Edit rutorrent config
 nano /var/www/rutorrent/config/config.php
**SET scgi config TO 
 $scgi_port = 5000;
 $scgi_host = "127.0.0.1";

*Edit rtorrent config
 nano /var/lib/rtorrent.rc

**Remove SCGI_LOCAL
 scgi_local       = /var/run/rtorrent/rpc.socket
**Add SCGI_PORT
 scgi_port = 127.0.0.1:5000

** NOT NEEDED - Later changed back to 0.0.0.0 and worked fine
** CHANGE bind
 bind 127.0.0.1

**Add new line for max file size
***Fixed an error I had with download not starting below.
 set_max_file_size = 300G

 


When launched via etc.d it does some stuff with the rpc.socket file we no longer use
 chmod: cannot access '/var/run/rtorrent/rpc.socket': No such file or directory
 chown: cannot access '/var/run/rtorrent/rpc.socket': No such file or directory
just commented these out in the /etc/init.d/rtorrent file
nano /etc/init.d/rtorrent
         #chmod ug=rw,o= $SOCKET
         #chown rtorrent:www-data $SOCKET

Sync'd daemon up
 systemctl daemon-reload

Re-Set permissions on rtorrent folder
chown -R  rtorrent:rtorrent /var/lib/rtorrent/
chmod -R 777 /var/lib/rtorrent/

Also noticed rtorrents download folder was set to permission for root, so changed that too

 chown -R rtorrent:rtorrent /srv/storage
 chmod -R 777 /srv/storage


When I run service rtorrent start I see the below error still, however, rutorrent works and connects successfully.
 Warning: Unit file of rtorrent.service changed on disk, 'systemctl daemon-reload' recommended.

 

 

 

### OTHER ERRORS ###

* Download not starting

  >file_list: Failed to prepar
 XMLRPC Error: Found a file exceeding max file size

Add max file size to rtorrent.
   nano /var/lib/rtorrent/.rtorrent.rc
   add below line
 set_max_file_size = 300G

 

* ERROR Could not lock session directory
rtorrent: Could not lock session directory: "/var/lib/rtorrent/.session/", Permission denied

This was despite ls -o showwing drwx and rtorrent owner, but cown / chmod above fixe dit.

**FIX - Re Set permissions with chown / chmod as above 

#### DEBUGGING ###


* Run rtorrent application as user rtorrent, with interactive output in shell
 su - rtorrent -s /bin/bash -c "rtorrent"

 

* Run rtorrent as root

 cp /var/lib/rtorrent/.rtorrent.rc ~/

 rtorrent

 

* Check permissions on rtorrent folder

      su - rtorrent -s /bin/bash -c "rtorrent"

 

 

 

Jeremy Davis's picture

As noted above, it is planned that the next release of the Torrentserver appliance will use Transmission instead of rTorrent. There have been some disagreements about whether that's a good idea or not. Perhaps if we can get some community maintainers then we could reconsider that? Or perhaps at some point in the future, we could consider having 2 different appliances (one with rTorrent, one with Transmission)?

Whilst performance wise, rTorrent seems to be one of the best (especially for really high numbers of torrents). However, my experience with Transmission over the years has been that it's flawlessly reliable.

cpugenesis's picture

Hi Jeremy

I'd like to put a third option in the mix if I may...Deluge.  I have set this up on a Ubuntu Server OS since my last post and that has been running solid for me.  Also has a great web UI and has a decent set of features.

One option is to maintain multiple branches but then it is more maintainence and will muddy the water on choice...Just depends on what the target user base is I guess - and if it's focused around the users trusting what they are given is the best option for their scenario or if it's imperitive that the users choose their package.   

To start out with I'd say gather the options with benefits for each, and make sure the goal is clear i.e. do we need to focus on the best all round torrent, something for casual downloaders, seeding stations, or a separate entity for each.

Not sure where my installations of transmission and rtorrent have gone wrong (on different host systems with different stability issues on each)  but sounds like more community maintaners are needed and we could do with some test branaches going out there to see what actually is most stable.  

I'd be willing to get involved in the comminity.

Thanks

Jeremy Davis's picture

Thanks for taking the time to post your suggestion. And apologies on my essay like response...!

And yes you certainly may! We're always happy to have feedback. Even in the cases where it's not flattering! :)

TBH I have never used Deluge, but it does seem to be a fairly popular choice. Having said, after doing some online research (sounds much more impressive than "after googling"!) that a number of users report that Deluge is good, but when using it with numerous torrents it has high hardware usage (particularly CPU) whereas Transmission is much lighter weight (although still not as efficient as ru/rTorrent when concurrently connecting to 1000s of torrents). On the flipside, apparently it is very aggressive so generally achieves quite good speeds (probably the reason for the increased resource usage...).

You seem to have a great grasp on the complexities of the situation. In some respects, allowing users to choose their preference would be awesome, but I agree that it does increase the maintenance overhead, plus probably makes it hard for newer users to choose.

Perhaps in the future, we could consider providing an appliance that pre-includes all the options. Then users could enable/disable the different clients as they decide? That would require increased initial development work plus increased ongoing maintenance but would be particularly cool IMO. I think it's probably too much work to get done within the window we have for v14.2 dev.

Our current appliance is not really designed for a specific torrent usage scenario. And TBH I'm not sure what usage scenario most of users are fulfilling. It is probably best described as trying to be an "all-rounder" (as best as can be achieved).

Regardless, yes we do need more community maintainers. The more the merrier! FWIW all our code is on GitHub (here is the current Torrentserver appliance code). If you'd like to fork that and have a play, please be my guest. I'd love to hear how you go and am more than happy to give you assistance and pointers if you get stuck or have questions.

Off on something of a tangent, but hopefully give you some additional insight. Historically one of our biggest issues with community contributions has been our versioning regime and build process. At the moment, all appliances must be the same version. So that means we either need to re-release an appliance (as the same version - which is pretty dirty) or rebuild and version bump the entire library (which is way to labour intensive to do too often).

One of my main aims for v14.2 is to provide support for sub-versioned appliances. That way we will be able to be much more responsive to community input. I.e. we can individually rebuild appliances as community members tweak and improve them, rather than having to wait for the next release. As there is still an overhead, we will probably still build appliances in batches, although I think it would be good to have an arrangement where we schedule regular build days every so often or once there are 10 appliances in the queue. That way we could have a maximum time that a tweaked server will need to wait before it's rebuilt.

cpugenesis's picture

No worries, an essay response is all good if you're providing the imformation to justify, it which you definitely have :)

I havn't tried hammering my deluge torrent yet so it will be interesting to see what that looks like.  Sounds like a balancing act, balancing resource usage with features and benefits of each client.  Your research seems to all be pointing towards Transmisison and justifing that move.  

Defo would be nice to see an all in one, one day.   I think we could have a checklist for each client with a 'Good For'  / 'Not So good For' points and highlighting the bits around deluge cpu usage etc.  There could be a default client as our recommended all rounder with the option to switch on/off others as you mentioned.  

The only flip side is some poeple might want to keep it compact so not have them all installed with all the dependancies.  Be good to explore if we can package them up so can be installed as part of the switch on process with an option to remove for those streamlining.     

Cheers on the community part - I'll defo have a play - and thanks very much for the offer :)

Sounds like a lot to keep on top of trying to keep them all sync'd up etc.  Do you currently take a snapshot off all the branchs to get a selection of community members to do some extended testing pre release then? Sounds like a pain if one needs a rebuild in the current form.  I bet you can't wait to have some form of sub versioning hehe.  Is it in a modular form where you incorporate the main base and system updates in one and then add the necessary packages and config for each appliance?

 

 

Jeremy Davis's picture

It'd be great if you get a chance to have a play and give us some input. But no hard expectations. See how you go and as I say, please feel free to ping me if you need pointers. There is a fair bit of documentation but it's not all great and some of it is a bit out of date.

As to your specific point about not including too much cruft; my inclination would be to include required config for all options (they're small text files), but only have one client actually installed. If users wished to use one of the other clients then we could include install scripts which take care of that.

As for our current release workflow; when we are working on a release (as we are currently) we have a small internal core team (myself and 2 other guys - but all of us only working on it part-time). We also have a couple of really committed long serving community members who help out immensely as they can afford the time. We also have other more casual community members assist here and there (often with their "favorite" appliance) but that ebbs and flows. In the past we have also had a company that uses TurnKey lots internally; donate some of their juniors to assist with testing.

We build the release from the current HEAD of master on all relevant repos. After testing (or sometimes even after release), we go back and tag HEAD (or on the odd occasion if new commits have been merged since release; the relevant commit) with the release version. For example, the master branch of Torrentserver appliance build code now includes Transmission (as per previous comments). But if you look at the 14.1 tag you can see that it still includes r/ruTorrent. FWIW we also have a "common" repo, which is where shared components come from (e.g. the fileserver components which the Torrentserver appliance includes). That is also tagged with the version. E.g. Common at the 14.1 tag is currently about 30 commits behind HEAD.

Also one of our team has developed an appliance testing framework which we hope to put into production for this release. Hopefully that will both decrease the time and effort required for testing, plus improve the quality of the release.

Michael's picture

This worked for me, thanks!!

 

cpugenesis's picture

Cheers some good info :)  Will have a gander,

 

Cheers

Jeremy Davis's picture

I look forward to your input and feedback! :)
Michael Laporte's picture

Hey, sorry if this is reviving a necro'd thread... but I noticed that in my case the problem only happens after a reboot and is resolved with a

service rtorrent stop

followed by

service rtorrent start

I have since added a script that does just this into /etc/cron.daily hoping it resolves the problem in a way and saves me the 2 minutes it takes to do it manually  :D.  The server only reboots during a power outage and I go on it only from time to time, so daily is probably adequate for me.

I have no idea why it doesn't start up properly when the system boots up and have given up trying to figure it out, especially given this simple workaround.... hope it works for others.

Cheers,

Mike

Jeremy Davis's picture

Thanks for sharing your work around. It sounds like it's not starting properly when the system tries to start it on boot. Perhaps it's some race condition at boot time? Perhaps you could tweak the initscript (/etc/init.d/rtorrent) and see if it can be improved?

Regardless, as you have probably read, the next version will use Transmission. We may revisit an rTorrent appliance again, but not for now...

Add new comment