On 10/13/14 13:32, stephen Turner wrote: > Do you or will you have a feature within toybox to do a torrent like > download using wget or similar command line tool?
I could add a bittorrent client, sure. But probably not until after 1.0. I haven't done wget yet because about half the web has already switched over to requiring https, so you these days you can't download from twitter or github or kernel.org without some form of https support. (I'd been planning to use "stunnel" but nobody's providing a reasonable embedded implementation of it. I've pestered dropbear about it for years, but no...) > I found a program > that looks very appealing and i'm hoping its functionality will make it > into your application at some point. > > It splits the data for a single file/location and apparently also > accepts multiple locations/mirrors. > > http://axel.alioth.debian.org/ > > https://docs.oseems.com/general/operatingsystem/linux/command-line-download-accelerator Oh this old gem. Way back in the http 1.0 days, the mosaic web browser would open multiple simultaneous connections to download web page elements faster. Their stated rationale was each connection had setup/teardown costs, but the actual problem was that busy websites would apportion the bandwidth between each connection, and if you did multiple connections in parallel you'd get multiple shares of the bandwidth. This turned into an arms race between web browsers for how _many_ parallel connections you could do, and once they were well into double digits it led to websites putting in firewall rules to block browsers that did this. (The solution was http 1.1 and pipelining. Penalizing sites for abusive connections scaled back when masquerading when mainstream, but for a while there it was common to account bandwidth per IP instead of per-connection in fancy load balancers and such.) Most of the references to this have fallen off the web, but here's something from that era: http://chimera.labs.oreilly.com/books/1230000000545/ch11.html The technique you're using does not actually speed up downloads, since a single connection can go somewhere around 100 megabytes/second on a gigabit lan. This does not make the server, the client, or the routers in between go faster. What that technique _does_ is get multiple shares of bandwidth from busy websites, at the expense of other users, by essentially "voting twice". The only actual benefit from it is your "apparently also" (which does not sound like the primary use when phrased like that). If this technique were to become common again, webmins would probably add firewall rules to treat it as DOS and block your IP from accessing the site at all for minutes at a time. (Some of the oldtimers still hold a grudge. And it's trivial to detect.) That said, an _actual_ bittorrent client would be a reasonable thing to add, sure. > Thanks, > Stephen Rob _______________________________________________ Toybox mailing list [email protected] http://lists.landley.net/listinfo.cgi/toybox-landley.net
