"Harik A'ttar" writes:
> very quickly (and start timing out, thread loss, etc). A number of > people are hoping it's blocking IO vs Non-Blocking, since that's > fixable. Sorry to reply to my own message, but I've been thinking about it and the more I think about it, the more sense it makes. Realistically, we get thread-pinch situations and the node spends all it's time burning CPU sending encrypted QR messages, and ticker'd events start failing. This is a Bad Thing(tm) assuming we have NBIO, we only need one thread to handle network traffic. This handles the basics like encryption, accepts, and message passing. Another thread manages the outstanding queries. Basically, they sit on a queue waiting to be timed-out unless a message comes in that changes them. (a Keep Waiting or End-Of-Query (RNF, DNF, DF) message.) The obvious advantage is we don't have to do any work on them unless there's network activity or they timeout, so they sit in a thread that's sleeping waiting for something to change. This lets a node have potentially thousands of outstanding queries with minimal overhead. If a DS search takes a reasonable amount of time, we're not going to kill ourselves by doing this. It dosn't take very long at all to do a binary search of 20k records. (nor does a tree walk, adding the first nibble to an int[16] array. Something is very wrong with the DS implementation if that takes as long as it currently does) So, now we lots of outstanding queries, what does that buy us? A lot less CPU burned rejecting queries. It also means that they get through, and DNF or DF eventually. As various people keep saying, DFs = Specalization = Good. All the large query queue does is allow us to keep transient load surges from despecing the network and throwing us into global overload. Incidentaly, there's no reason NBIO is required for this. As small as FN is right now, you can put two blocking threads per connection and limit yourself to one connection per remote node... one reader, one writer. I've got to re-read the FNP doc to verify there's nothing sticky with multiplexing queries on one channel, but it looked fine to me. Comments? I realize this is a major infrastructure change. I've had to do reflows like this on other projects and it's always a bitch. The question is, do we think it's a more efficent method of handling the traffic then one-thread-per-query. --Dan _______________________________________________ devl mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/devl
