What about breaking the tables up in parts and distributing them through torrents?
On Feb 9, 2010, at 11:11 PM, Robin Wood <[email protected]> wrote: > On 10 February 2010 04:17, Harley Green <[email protected]> > wrote: >> Glad to hear it went so well for the first time! >> This problem sounds like something perfect for a reliable multicast >> file >> distribution mechanism! Just one host needs to have direct access >> to the >> drive with the tables and >> loops through the entire set streaming over a multicast capable >> network. >> Then anyone who wants to get the tables just has to listen in and >> wait for >> the cycle to complete. >> As more people get it more sources can stream to different >> addresses sending >> either different portions of the table set or starting at different >> offsets. >> --Harley > > That would have been OK but most people brought USB disks not laptops > and someone was explaining to me how USB is dumb so has to have the > host handle most of the processing load. Throw 2 or more disks on and > the overhead goes way up. > > Disk duplication was suggested but that wouldn't work because of > different size drives, people only wanting parts and people wanting > updates after other tables have been added. > > The best I can come up with without any research is to get some esata > or external hot plug IDE caddies and a bunch of screwdrivers so people > who brought enclosed USB disks could pop them out and put them in a > caddy. Don't know how receptive people would be to voiding their > warranties. Some would, some wouldn't. > > I think putting the seeds on an internal RAID 0 rather than my > external USB disk would also speed things up a lot, even with the > current external USB model. I need to do some playing and asking > around to see what works and what doesn't. > > Robin > >> >> On Tue, Feb 9, 2010 at 6:22 PM, Robin Wood <[email protected]> wrote: >>> >>> So, Shmoocon is over and the swapshop happened. Unfortunately it >>> wasn't as successful as it could have been but for the first >>> attempt I >>> think it went OK. >>> >>> A number of people donated tables and others leached sets, the >>> problem >>> turned out to be that the initial donation was about 300GB and to >>> copy >>> that from one external disk to a second was taking up to 7 hours, >>> when >>> two people were copying that time shot up even more. >>> >>> When I get back home to the UK I'll do an inventory of what I >>> collected and find some way to share it out to those who want >>> copies. >>> I'll also have a play with different methods of running the system >>> to >>> try to drop that time down to something more manageable. >>> >>> Thanks to Darren and Larry for donating hardware and the PaulDotCom >>> crew in general for keeping an eye on the setup while I was running >>> round doing other con activities. >>> >>> Robin >>> _______________________________________________ >>> Pauldotcom mailing list >>> [email protected] >>> http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom >>> Main Web Site: http://pauldotcom.com >> >> >> _______________________________________________ >> Pauldotcom mailing list >> [email protected] >> http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom >> Main Web Site: http://pauldotcom.com >> > _______________________________________________ > Pauldotcom mailing list > [email protected] > http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom > Main Web Site: http://pauldotcom.com _______________________________________________ Pauldotcom mailing list [email protected] http://mail.pauldotcom.com/cgi-bin/mailman/listinfo/pauldotcom Main Web Site: http://pauldotcom.com
