On Sat, Jan 10, 2009 at 8:19 PM, Matthew Toseland
<[email protected]> wrote:
> On Saturday 10 January 2009 11:44, Daniel Cheng wrote:
>> On Sat, Jan 10, 2009 at 9:17 AM, Matthew Toseland
>> <[email protected]> wrote:
>> > On Monday 22 December 2008 16:04, [email protected] wrote:
>> >> Author: j16sdiz
>> >> Date: 2008-12-22 16:04:07 +0000 (Mon, 22 Dec 2008)
>> >> New Revision: 24747
>> >>
>> >> Modified:
>> >> trunk/plugins/XMLSpider/XMLSpider.java
>> >> Log:
>> >> no retry, fail faster
>> >
>> > Why? BTW retries = 1 still means 1 retry i.e. 2 attempts iirc. It must be
> no
>> > more than 2 to avoid the cooldown queue.
>>
>> higher turnover rate.
>
> I thought you were having performance issues with XMLSpider core, in which
> case it makes more sense to try more?
Performance on this is not a issue after migrating to perst.
There is two remaining bottleneck -
(1) The content filter is slow. I have no idea why, but checking
for valid URI is really slow
(2) ClientGetter.start() try to check the data store before return.
We are in a low priority thread, it have problem getting the
share of synchronization locks.
>>
>> >>
>> >> Modified: trunk/plugins/XMLSpider/XMLSpider.java
>> >> ===================================================================
>> >> --- trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:03:41 UTC
> (rev
>> > 24746)
>> >> +++ trunk/plugins/XMLSpider/XMLSpider.java 2008-12-22 16:04:07 UTC
> (rev
>> > 24747)
>> >> @@ -526,8 +526,8 @@
>> >>
>> >> /* Initialize Fetch Context */
>> >> this.ctx = pr.getHLSimpleClient().getFetchContext();
>> >> - ctx.maxSplitfileBlockRetries = 2;
>> >> - ctx.maxNonSplitfileRetries = 2;
>> >> + ctx.maxSplitfileBlockRetries = 1;
>> >> + ctx.maxNonSplitfileRetries = 1;
>> >> ctx.maxTempLength = 2 * 1024 * 1024;
>> >> ctx.maxOutputLength = 2 * 1024 * 1024;
>> >> allowedMIMETypes = new HashSet<String>();
>> >>
>> >> _______________________________________________
>> >> cvs mailing list
>> >> [email protected]
>> >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs
>> >>
>> >>
>> >
>> > _______________________________________________
>> > Devl mailing list
>> > [email protected]
>> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
>> >
>> _______________________________________________
>> Devl mailing list
>> [email protected]
>> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
>>
>>
>
> _______________________________________________
> Devl mailing list
> [email protected]
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
>
_______________________________________________
Devl mailing list
[email protected]
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
