Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
heasley h...@shrubbery.net writes: This sure smells to me like a solution in need of multicast (or, more accurately, deployed multicast) with a fall back to something like bittorrent for bootstrapping or when you've missed session data and can't recover. Multicast as a data stream would be much more efficient is collecting updates, although there are still issues that would need to be worked through like how do you know you heard everything. Oh, and it only works if you multicast deployed to your site of course. please, we do not want multicast as the only solution. does your noc know how to debug multicast forwarding problems? I didn't say only :-) . In fact, I pointed at bittorrent as a backup. I simply said it would be more efficient when sending out *new* changes. -- Wes Hardaker SPARTA, Inc. ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
Thu, May 24, 2012 at 06:36:51AM -0700, Wes Hardaker: I didn't say only :-) . In fact, I pointed at bittorrent as a backup. I simply said it would be more efficient when sending out *new* changes. understood; i'm registering/re-enforcing that if that is developed it must not be the only mathod. ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
heasley h...@shrubbery.net writes: Thu, May 24, 2012 at 06:36:51AM -0700, Wes Hardaker: I didn't say only :-) . In fact, I pointed at bittorrent as a backup. I simply said it would be more efficient when sending out *new* changes. understood; i'm registering/re-enforcing that if that is developed it must not be the only mathod. Agreed. If we can't do it over carrier pigeon, we have no real backup. -- Wes Hardaker SPARTA, Inc. ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
Wes, On May 23, 2012, at 9:22 PM, Wes Hardaker wrote: Bittorrent works well for sharing the load, but either requires a lot of bittorrent start files (whatever they're called), which then becomes hard to syncronize; or a small number of bittorrent files (one per aggregation point) that indicate you need to refetch and entire .zip file even if you already have most of it anyway. I'm far from an expert on Bittorrent but I believe this is not right. A .torrent file (what you called a bittorrent start file) can contain contain a list of files to be transferred, there's no mandate to zip them into a single file first. Presumably the BT protocol is smart about not transferring files you already have locally -- that's kind of the whole point. Rob's operational overview slide makes it clear he's imagining operating in the collection-of-files mode. Probably there are some issues with this mode of operation too -- if you list every single RPKI object individually then the size of the .torrent file scales linearly in the size of the unauthenticated/ tree. This seems likely to suck since I am guessing BT's normal operating regime is to transfer small numbers of large files rather than potentially vast numbers of relatively tiny files. The zip files Rob's slides talk about just contain the .torrent file and manifest (which is also O(N) in the size of the tree, sigh). Speaking purely as a Bittorrent amateur + not the author of the slides! --John ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
On Thu, May 24, 2012 at 4:13 PM, John G. Scudder j...@juniper.net wrote: Wes, On May 23, 2012, at 9:22 PM, Wes Hardaker wrote: Bittorrent works well for sharing the load, but either requires a lot of bittorrent start files (whatever they're called), which then becomes hard to syncronize; or a small number of bittorrent files (one per aggregation point) that indicate you need to refetch and entire .zip file even if you already have most of it anyway. I'm far from an expert on Bittorrent but I believe this is not right. A .torrent file (what you called a bittorrent start file) can contain contain a list of files to be transferred, there's no mandate to zip them into a single file first. Presumably the BT protocol is smart about not transferring files you already have locally -- that's kind of the whole point. Rob's operational overview slide makes it clear he's imagining operating in the collection-of-files mode. per publication-point I think? So you have to collect N-thousand of these torrent files, and keep N-thousand torrent jobs running and keep state on N-thousand remote torrent files changing over time... some of this could be alleviated in many ways, worst case though is that. Probably there are some issues with this mode of operation too -- if you list every single RPKI object individually then the size of the .torrent file scales linearly in the size of the unauthenticated/ tree. This seems likely to suck since I am guessing BT's normal operating regime is to transfer small numbers of large files rather than potentially vast numbers of relatively tiny files. well, folk like fedora/ubuntu/etc offer full distributions over bittorrent ... I've not done a find . | wc -l on a system like this in a long time, but 4-8 gb over bittorrent for a few thousands of files seems to work fairly well for them. The zip files Rob's slides talk about just contain the .torrent file and manifest (which is also O(N) in the size of the tree, sigh). maybe looking at a bittorrent-like solution for intra-as distribution is interesting though? though that also seems (to me) to fall into 'local implementation' land, from the vendor/arms-dealer that sold me the solution. Speaking purely as a Bittorrent amateur + not the author of the slides! me too, as an amateur of bittorrentz I mean. -chris --John ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
Re: [sidr] sidrSlides for RPKI Over BitTorrent presentation
Rob Austein s...@hactrn.net writes: For those who didn't get to see the end of the RPKI Over BitTorrent presentation in today's SIDR meeting, the full slide deck is available at http://www.ietf.org/proceedings/83/slides/slides-83-sidr-9.pdf and should be relatively self-explanatory. I've been thinking about this a lot lately, as it'll certainly be a challenge to ensure everyone is up to date. It doesn't matter whether you should have pulled or whether you pull right when you get a request. Either way, there is an underlying data distribution problem. Most importantly, every bit of the repository needs to pretty much get everywhere. Rdist works well for trying not to duplicate data. Bittorrent works well for sharing the load, but either requires a lot of bittorrent start files (whatever they're called), which then becomes hard to syncronize; or a small number of bittorrent files (one per aggregation point) that indicate you need to refetch and entire .zip file even if you already have most of it anyway. But the real underlying problem is what I said above: every bit needs to get efficiently to every place, ideally at time of initial publication. This sure smells to me like a solution in need of multicast (or, more accurately, deployed multicast) with a fall back to something like bittorrent for bootstrapping or when you've missed session data and can't recover. Multicast as a data stream would be much more efficient is collecting updates, although there are still issues that would need to be worked through like how do you know you heard everything. Oh, and it only works if you multicast deployed to your site of course. -- Wes Hardaker SPARTA, Inc. ___ sidr mailing list sidr@ietf.org https://www.ietf.org/mailman/listinfo/sidr
