On Thu, Oct 16, 2003 at 10:31:47AM +0200, Niklas Bergh wrote: > >On Wed, Oct 15, 2003 at 02:17:57PM -0500, Brandon Low wrote: > >> How long do you think connection multiplexing will take to impliment? > > >I don't think it will take long to implement. However I haven't fully > >thought through the receiving side. Debugging will be a problem of > >course, but with a local test network it should be feasible. > >> > >> Current unstable -> stable merge first, then multiplexing on unstable > >> branch with a split network between unstable and stable might be one way > >> to handle this. > > Merging first would be good if we can get unstable to work reasonably well, > see my next comment.
Absolutely, if we can. In which case I'd probably implement muxing with backwards compatibility, after we merge. > > >If the unstable code is likely to become good enough to merge before > >multiplexing is in... > > Something happened in 6251 (or another of the releases made during the last > 24 hours) that took us one large step closer to that. For the first time in > 20+ builds my node isn't using 100% CPU all the time, it is still consuming > about double as much CPU as the builds of the past but it is a huge > enhancement still. Excellent. > > Neither have I yet seen a thread explosion, active pooled jobs is staying > healthily low and the size of the fast jobs queue is staying low (=empty > almost all the time). Neither have I yet seen any of the > now-ever-so-frequent-but-not-previously-existing OOM.s. Doubleplusgood. > > 6251 is QR:ing *much* less than previous builds and is now limited by > available bandwidth instead of by routingTime.. as it should be. Check. So apparently we've gotten somewhere with the opening too many outbound connections issue, which was the main purpose of 6250 (as well as some importantish bugfixes). > > > >> Things I can think of that make multiplexing the way to go right now: > >> -Reduce CPU overhead by massively reducing connections > >Yup. > >> -Reduce network overhead for new connections > >Yup. > >> -Allow firewalled nodes to be fully functional parts of the network > >> just without ARK insertion. This will give us some of the scalability > >> advantages of Gnutella's Ultrapeer technique > >You mean shadow nodes? Maybe. I'd rather just set a flag on the > >reference and have the node it connects to not forward the reference. > >> -Allow us to go back to queueing messages on connections rather than on > >> peers, this is probably a good thing as it should help get rid of the > >> HUGE message send times that have become a problem on latest unstable. > >Not necessarily. The hugeness is merely a diagnostic issue. And I'm not > >about to back out PeerHandler. We might have two connections to a node > >for some reason or other. > > Will it also be one step closer to queueing trailers? Something like most > other P2P networks 'Allow a maximum of X sends (to each peer possibly) at > any given time'. A long time ago someone said something intelligent on this > issue on this list: Queueing trailers is bad. Because we do not control the input speed. > If we are going to send 100 items to another node, what is the best way? > Sending them all in parallell or sending them one or a few at a time? > The answer was that it is suprisingly much better to send them somewhat > serialized since at any given time, between when we start transfering and > when we are done transfering all of them, more files will have reached their > destination when compared to sending them all in parallell. Maybe, if they are all in the datastore. But most of the time, most of them are not, and I'm not sure it's worth the special case optimization. > > Regards > /N -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
