Cmd Line Guy - Thu, 2010/11/25 - 03:28

When it comes to downloading large files such as the Turnkey VM or ISO image, I much prefer using wget for a couple of reasons:

1) Very often the computer on which I'm running the web browser is not the same as the one to which I want to download the file; very often that system is in an open ssh terminal on my desktop.
2) It has a very useful auto-restart feature in case of an interruption.

Would it be possible to provide friendlier URLs for downloads on this site? wget doesn't really like the redirect URLs that the TurnKey Linux website uses.

Thanks.

Forum: 
Support
Liraz Siri's picture

Liraz Siri - Thu, 2010/11/25 - 05:29
I like using wget too, and in fact I use it all the time with the sourceforge links.

Like so: 

wget http://downloads.sourceforge.net/project/turnkeylinux/turnkey-lamp/11.0rc-lucid-x86/turnkey-lamp-11.0rc-lucid-x86.iso
Also, we're going to be experimenting soon with hosting a few images directly on a CDN and paying for the bandwidth with advertising on the download page. If it pays for itself we'll move from sourceforge to hosting the images directly. A nice bonus is that the CDN is much faster than sourceforge. I downloaded an ISO image to one of our servers at 30MB/s!
Jeremy Davis's picture

Jeremy Davis - Thu, 2010/11/25 - 10:07

Here in Australia most people have download quotas but my ISP (and a few others) mirror SourceForge so its quota free. Besides downloads from SourceForge max out my connection (possibly because its mirrored by my ISP) so faster is no use to me!

Liraz Siri's picture

Liraz Siri - Thu, 2010/11/25 - 20:53
I didn't realize there was a special situation with SourceForge in Australia. Don't worry about it, we'll keep SourceForge updated as a mirror as long as it's useful to you and maybe others in a similar situation.

We probably wouldn't want to drop SourceForge entirely anyhow as hosting all build types on the CDN would be expensive.

Adrian Moya's picture

Adrian Moya - Thu, 2010/11/25 - 15:21

TKLDevEnv's tklpatch-getimage command does it's magic to list and download files from sourceforge. If there are plans for moving them, please drop me an email so I can prepare the code for the changes. Maybe in that time we could agree in a better way to get the images (but did I told you I was proud of my code in tklpatch-getimage? wink)

Look at this beauty (tklpatch-getimage --update):

echo "Updating image list, please wait" 
wget -q -O /tmp/imagelist http://sourceforge.net/projects/turnkeylinux/files/ 
cat /tmp/imagelist | sed -n 's|.*href="\(http://sourceforge.net/projects/turnkeylinux/files/.*\.iso/download\).*|\1|p' | sort | uniq > /usr/local/share/tklpatch/baseimagelist 
COUNT=`wc -l < /usr/local/share/tklpatch/baseimagelist` 
echo "Done. $COUNT images found." 
exit 0

Liraz Siri's picture

Liraz Siri - Thu, 2010/11/25 - 17:10
Don't worry guys, we're not going to delete anything from sourceforge so all the existing links will continue to work indefinitely. Also, transferring to sourceforge is pretty easy from "the cloud" so if there is demand for it we could continue to publish new releases there. It's no big deal.

BTW JedMeister, the CDN we are going to be experimenting with (e.g., SimpleCDN) has a presence in Sydney so that might work just as well for you.

Here is a test link: 

wget http://e1.simplecdn.net/testbucket.turnkeylinux.org/turnkey-lamp-11.0rc-lucid-x86.iso
Jeremy Davis's picture

Jeremy Davis - Sun, 2010/12/12 - 03:04

I perhaps should have started a new topic but thought I'd just do a quick post here as its somewhat relevant to discussion (although only vaguely relevant to the OP).

I've been thinking, what about having TKL images also available as torrents? I for one would be happy to seed TKL images and I'm sure there are plenty of other TKL users who would do likewise. This could be a low cost and alternative way to distribute TKL and another way that users could give back to the project (by seeding the TKL images).

What do you think?

Liraz Siri's picture

Liraz Siri - Sun, 2010/12/12 - 18:38
Torrents work best for very popular files which hundreds/thousands of people will want to download simultaneously. A CDN served HTTP download will usually provide a better user experience in other cases. But it might be a good idea to give it a shot, maybe run a test case with just LAMP...

Rik Goldman - Sun, 2010/12/12 - 22:31

I put the Ampache material we produced on a tracker and seeded for a month. I thought it was a grand idea and applauded me profusely. I did get one download over the course of the month, all told. Granted, I could have shopped around for trackers a little.

Jeremy Davis's picture

Jeremy Davis - Mon, 2010/12/13 - 02:15

but I think the main problem with torrents is a sort of chicken-egg type scenario.

They don't work as an effective download option unless a lot of people are seeding it, but you won't get a lot of people seeding it unless there are quite a few downloading it. Hence your note on popularity Liraz.

If TKL were to have a tracker and regularly updated scrape available on the torrent download page so it would be obvious to downloaders the situation - that would avoid or at least reduce the potential for a poor user experience. I don't know but I would suspect that some may use it and support it just as a way of reducing load/costs to TKL, sort of like a way of giving back, by taking a little bit. As long as we could have at least a dozen or so consistent seeders it should be a relevant option (even if not the preferred one). I'm not sure how the tracker/quota thing all works, but if it gets off the ground, perhaps there is a way of giving karma for seeders? Not that I probably need too much more karma :)

I can provide at least 2 seed locations for starters (from home and from work) and it sounds like Rik may be able to have his arm twisted, and perhaps could convince his work to seed it too. Then if both you dev guys seed one each from home and one from work then we have 7 seeders already.

Rik Goldman - Mon, 2010/12/13 - 04:21

I absolutely have the resources to seed. I guess I would have to learn how to mirror. But I'll commit what I've got. Only limitation I see is bandwidth.

Can't seed from work, but can have students contribute by seeding at home. Get the message across that if they already know about torrents, they know the wrong things; if they don't know, they'll learn legit use.

Jeremy Davis's picture

Jeremy Davis - Mon, 2010/12/13 - 06:34

That's a great thought, getting the students involved. Yes I think it's fantastic to be able to teach them about the legitimate and legal use of technology. They could have a go at the TKL Torrent Server appliance and/or look at how it could be tweaked/improved etc or even look at alternatives to how it could be done. Lots of great learning opportunities!

Ok then, I'll have to grab the bull by the horns then and get a bit proactive and try this experiement out.

Add new comment