On Saturday 10 January 2009 13:26, Daniel Cheng wrote: > On Sat, Jan 10, 2009 at 8:19 PM, Matthew Toseland > <[email protected]> wrote: > > On Saturday 10 January 2009 11:44, Daniel Cheng wrote: > >> On Sat, Jan 10, 2009 at 9:17 AM, Matthew Toseland > >> <[email protected]> wrote: > >> > On Monday 22 December 2008 16:04, [email protected] wrote: > >> >> Author: j16sdiz > >> >> Date: 2008-12-22 16:04:07 +0000 (Mon, 22 Dec 2008) > >> >> New Revision: 24747 > >> >> > >> >> Modified: > >> >> trunk/plugins/XMLSpider/XMLSpider.java > >> >> Log: > >> >> no retry, fail faster > >> > > >> > Why? BTW retries = 1 still means 1 retry i.e. 2 attempts iirc. It must be > > no > >> > more than 2 to avoid the cooldown queue. > >> > >> higher turnover rate. > > > > I thought you were having performance issues with XMLSpider core, in which > > case it makes more sense to try more? > > Performance on this is not a issue after migrating to perst. > > There is two remaining bottleneck - > (1) The content filter is slow. I have no idea why, but checking > for valid URI is really slow
Do you think this contributes to fproxy lack of performance? > (2) ClientGetter.start() try to check the data store before return. > We are in a low priority thread, it have problem getting the > share of synchronization locks. Doesn't happen on the db4o branch - datastore checking is serialised on a separate thead. > > >> >> Modified: trunk/plugins/XMLSpider/XMLSpider.java > >> >> =================================================================== > >> >> --- trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:03:41 UTC > > (rev > >> > 24746) > >> >> +++ trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:04:07 UTC > > (rev > >> > 24747) > >> >> @@ -526,8 +526,8 @@ > >> >> > >> >> /* Initialize Fetch Context */ > >> >> this.ctx = pr.getHLSimpleClient().getFetchContext(); > >> >> - ctx.maxSplitfileBlockRetries = 2; > >> >> - ctx.maxNonSplitfileRetries = 2; > >> >> + ctx.maxSplitfileBlockRetries = 1; > >> >> + ctx.maxNonSplitfileRetries = 1; > >> >> ctx.maxTempLength = 2 * 1024 * 1024; > >> >> ctx.maxOutputLength = 2 * 1024 * 1024; > >> >> allowedMIMETypes = new HashSet<String>(); > >> >> > >> >> _______________________________________________ > >> >> cvs mailing list > >> >> [email protected] > >> >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs > >> >> > >> >> > >> > > >> > _______________________________________________ > >> > Devl mailing list > >> > [email protected] > >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >> > > >> _______________________________________________ > >> Devl mailing list > >> [email protected] > >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >> > >> > > > > _______________________________________________ > > Devl mailing list > > [email protected] > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > _______________________________________________ > Devl mailing list > [email protected] > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >
pgpUCXf37tVEt.pgp
Description: PGP signature
_______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
