[
https://issues.apache.org/jira/browse/NUTCH-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13961211#comment-13961211
]
Sebastian Nagel commented on NUTCH-1747:
----------------------------------------
+1
Looks like inProgress was intended to hold more than the bare count of
FetchItems in progress. In doubt, we can get the in-progress FetchItems and
their associated queue from FetcherThreads (cf. NUTCH-1182).
> Use AtomicInteger as semaphore in Fetcher
> -----------------------------------------
>
> Key: NUTCH-1747
> URL: https://issues.apache.org/jira/browse/NUTCH-1747
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.8
> Reporter: Julien Nioche
> Priority: Minor
> Attachments: NUTCH-1747-trunk.patch
>
>
> In Fetcher we currently use
> Set<FetchItem> inProgress = Collections.synchronizedSet(new
> HashSet<FetchItem>());
> as semaphore within the FetchItemQueues to keep track of the URLs being
> fetched and prevent threads from pulling from them. It works fine but we
> could use AtomicIntegers instead as all we need is the counts, not the
> contents.
> This change would have little impact on the performance but would make the
> code a bit cleaner.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
