-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The way things currently work, mike's system rsyncs the core tree of downloads from the primary mirror grid, then generates the .torrent files from the individual downloads themselves once the checksum validation completes. After that, ooodev and depshtrike.com pull the .torrent files from mike's setup via rsync.
The largest of the .torrent files I have for openoffice is the 3.1 all platforms DVD at 344kbyte. The largest .torrent I have for openoffice that isn't a cd/dvd iso is OOo_3.2.0_Solaris_Sparc_install-wJRE_de.tar.gz.torrent at 33kbyte. Considering we're representing files that exceed 100mbyte in size without even trying here, I don't really see the need to attempt to conserve space on the scale that we would actually see conserved. The entire collection of .torrent files for 3.2.1 with the current system is 5.16 MB (5,415,847 bytes). On 30/08/2010 4:01 PM, Peter Pöml wrote: > Hi Harold, > > Le 30.08.2010 à 20:40, Harold Feit a écrit : >> It's technically mike's side that handles torrent generation currently, >> not ooodev.org. > > Thanks for the insight. > >> If you have an rsync folder that I can use to automatically load the >> .torrent files from your side, it can be set up fairly trivially to have >> the torrents authenticated on the depthstrike.com trackers within 2 >> hours of generation automatically. > [...] >> It is currently not a good idea to accept all torrent infohashes without >> discrimination, especially for this project. The load spikes are >> potentially system crippling. > > Thanks for the insight. Is there another way of excluding illegitimate > access? Like, allowing only torrents with certain names (instead of bt info > hashes)? I assume the load spikes would be from illegitimate users, who try > to use the trackers for some other stuff? Or how does it happen? infohashes only. The announce protocols of the bittorrent protocol prevent any other methods of excluding illegitimate access from working. > > Maybe I mentioned it before: there are no .torrent files that might be > mirrored; the content served is generated in realtime from a database. The > content does not primarily exist in torrent form on disk; torrents are just > one thing that can be made of them. And I would like to avoid the overhead of > writing torrent files for all files to disk, although that would certainly be > possible. The .torrents only need to be committed to disk at generation time, and only need to contain the info dictionary of the torrent, and no other part of it for authentication purposes (I actually strip all other information out of the .torrent file in the uploader that works on my site). > > Would it help if you mirror the file tree itself? I can provide a script that > does this without consuming bandwidth and disk space (Google knows it by its > name "null-rsync"); you'd have a seemingly identical copy of the tree, just > that the files would be filled with zeroes. > > Or the process that runs over the file tree and calculates the hashes > (including the bt info hash) could issue a HTTP or XMLRPC request to your > server to let it know about a btih to be included. That sounds easy, and it > would mean that new info hashes end up in your tracker without major delay. I have a php based upload script that could work with this that I use with my regular users. If you upload with this properly, the necessary portion of the .torrent can be stored on disk in my system and not actually commit anything to disk on the mirrorbrain side. Hard to believe I forgot about this considering I wrote the damn thing. Nice and oldschool HTTP POST too. > > Principally, you server could mirror .torrent files by downloading them via > HTTP though. But that doesn't sound too attractive (albeit certainly okay for > a small number of files. An ugly method sure, but workable. > > Thanks! > Peter > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMfAcWAAoJEF8nceBm0DUaX5QH/A/Uol9+cdf7BufVdCUJ5ncq NRL6QpB0YfLvgI6xRsHcrpnHtz78iK0batmzPCq92qs3JEvFWFFK7WHzlknoAIca ux/TmM0Thw5iWWeLHNhNAQ7JXsoVzJiSoa39IuKUeFZ+8accC+yklJx0eW4g7KOj nT1KM+kvjqQqecx2eNa064ksWe1khXnNi7Kqi7W21cv2mH+O+a6SI7g3hSKgR/Ud PVWLsPA3pl5ZbP7kKSPF6bUACZJXZVcXchD8Gj+HxFNk/E8PIOcgtpczjyoPvwHL H7hrtIGpYbvP07DZ+t1eO9wZ4zqe7btiBm54y7pCf0mAw/hGEG8MLOa/aSNy88Y= =MQ5j -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
