Hrrm. After further investigation... In Fred, things can and do randomly take ages. We have known about this for a while. It is due to the multiplicity of threads, the JVM, etc etc. Some time back I spent a week timing various events to try to find what was making routingTime's so slow - it was a horrible mess. Having added something to report when an event run on the scheduling thread took more than 100ms, I got, in the period from startup at 3:47:43 PM, to 3:53:29 PM, a few minutes later, 26 events. They were all of this pattern:
SEC Oct 14, 2003 3:48:40 PM (freenet.Ticker, write interface thread, ERROR): Handling [EMAIL PROTECTED]@31aa808edfad82cb,true@ 1066142920403:1066142914910:true:null:freenet.Message: DataRequest @null @ 31aa808edfad82cb took more than 100ms! Always a DataRequest, always :true: - meaning they succeeded. Apparently cancelling a SendFinished and scheduling a restart takes a long time?! I will investigate. Anyway, it would probably be worthwhile to implement a hybrid scheme: 3 classes of events: Group A - guaranteed to run REALLY fast. If execution takes too long we log an error. Run them on the selector threads, for minimum overhead. Group B - expected to run pretty quickly. Won't do network I/O, for example, or asymmetric crypto. Run them on a single thread reading from a queue. Time them and feed the timings to a diagnostic var. If the queue exceeds a certain length, move the overflow onto the ticker in the usual way - and produce a moderately loud log message. Group C - everything else. Runs through the Ticker in the usual way. So please, commit your code. It may not be the final form but it will serve as a useful basis. On Tue, Oct 14, 2003 at 03:40:03PM +0100, Toad wrote: > On Tue, Oct 14, 2003 at 09:29:11AM +0200, Niklas Bergh wrote: > > > > So, last night toad added code that enables fred to treat some TickerEvents > > as "fast events".. "Fast events" are executed on the thread that tries to > > schedule them instead of on a Thread from the Pool, effectively throttling > > unnecesary thread creation and removing some unnecessary scheduling > > overhead. An obvious side effect of this mechanism is that this the thread > > that wants to schedule an event might instead end up executing it, beeing > > blocked during the time the execution consumes. Hence the name "fast > > events"... we should avoid executing anything that isn't "fast" on the > > scheduling thread. Unfortunately, today, it has proved that bucketloads of > > "fast events" still consumes significant amounts of execution time and > > since, in the current architecture, the threads ending up executing the > > bucketloads of "fast events" are the *SL threads.. the network communication > > threads.. thereby blocking network communication more than desired. > > Well, the "fast events" in question evidently weren't that fast. That is > a bug. Can we try to fix the bug before adding bucketloads of overhead > back in for them? > > > > I am about the commit a change that enables the Ticker to take a middle > > route between the old behaviour and the new one. I am adding a thread, > > internal to the Ticker, that is dedicated to handle all of these "fast > > events", thereby relieving the scheduling thread of the execution but still > > not causing the ThreadPool to spawn large amounts of threads to execute > > these "fast events". This thread will have a maximum allowed backlog, when > > the backlog is full the Ticker will revert to letting the scheduling thread > > execute the event (the current queue size and the actual events can be seen > > in a separate table in the 'Pending Tasks' infolet). > > See above. > > > > This graph indicates the effect of this change on network bw usage (before > > change is to the left, after is to the right): > > http://194.236.28.174/freenetstuff/6237_and%206237modified_bwusage.jpg > > Cool. Does not invalidate what I said above though. > > > > /N > > -- > Matthew J Toseland - [EMAIL PROTECTED] > Freenet Project Official Codemonkey - http://freenetproject.org/ > ICTHUS - Nothing is impossible. Our Boss says so. > _______________________________________________ > Devl mailing list > [EMAIL PROTECTED] > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
