On Mon, Dec 20, 2004 at 03:40:44PM -0800, Doug Cutting wrote: > John X wrote: > >BasicUrlNormalizer.java should be made thread safe as > > > >< public String normalize(String urlString) > >--- > > > >> public synchronized String normalize(String urlString) > > > > > >If no objection, I will commit it late. > > Good catch. In general, we should be careful not to synchronize too > much.
I agree. >I think that only the substituteUnnecessaryRelativePaths() method > needs to be synchronized. > > If we specified the following flag: > > http://jakarta.apache.org/oro/api/org/apache/oro/text/regex/Perl5Compiler.html#READ_ONLY_MASK > > then only the matcher would need to be synchronized. > > We could avoid even that if we stored the matcher using a ThreadLocal. > This is easily done, as in: > > private ThreadLocal matchers = new ThreadLocal() { > protected synchronized Object initialValue() { > return new Perl5Matcher(); > } > }; > > private Perl5Matcher getMatcher() { > return (Perl5Matcher)matchers.get(); > } > > Then replace references to 'matcher' with 'getMatcher()'. > > That may be overkill if BasicUrlNormalizer isn't used in heavily > threaded code. Does it get invoked much under the fetcher or parser? I don't have number for how often BasicUrlNormalizer gets invoked in fetcher or parser. I do have a tool logically close to parser. When it runs over a segment with 100,000 fetches in 10 threads, there are about 600 or so NullPointerException problems. A rare event. I was in fact wondering if I have code problem elsewhere. John ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
