[
https://issues.apache.org/jira/browse/NUTCH-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17452112#comment-17452112
]
Lewis John McGibbney edited comment on NUTCH-2910 at 12/2/21, 1:49 AM:
-----------------------------------------------------------------------
Hi [~snagel] thanks for reviewing this and chiming in so quickly. I remembered
looking at this before and your explanation is where I eventually ended up...
it had just slipped my mind.
The good thing is that nothing is wrong here so ultimately I will resolve this
issue as not a problem however I think making use HADOOP-15005 tags would help
a lot. It's just time consuming to go through nutch-site.xml and tag
everything. I can maybe have some interns divide and conquer this work.
I made some trivial updates to the wiki page you referenced above. I totally
forgot that even existed as well... doh.
was (Author: lewismc):
Hi [~snagel] thanks for reviewing this and chiming in so quickly. I remembered
looking at this before and your explanation is where I eventually ended up...
it had just slipped my mind.
The good thing is that nothing is wrong here so ultimately I will resolve this
issue as not a problem however I will open a ticket to make use of HADOOP-15005
tags. i think this will help a lot. It's just time consuming to do though and
do that. I can maybe have some interns divide and conquer this work.
I made some trivial updates to the wiki page you referenced above. I totally
forgot that even existed as well... doh.
> FetchItemQueues overloaded constructor also interprets fetcher timeout as -1
> e.g. no-timeout.
> ---------------------------------------------------------------------------------------------
>
> Key: NUTCH-2910
> URL: https://issues.apache.org/jira/browse/NUTCH-2910
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.18
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.19
>
>
> The FetchItemQueues overloaded constructor [attempts to obtain the
> *NON-EXISTENT* _fetcher.timelimit_ configuration
> property|https://github.com/apache/nutch/blob/master/src/java/org/apache/nutch/fetcher/FetchItemQueues.java#L84].
> {code:java}
> this.timelimit = conf.getLong("fetcher.timelimit", -1);
> {code}
> As you can see a default value of -1 is provided. The first parameter is
> however wrong. It should instead reference the following configuration
> property.
> {code:xml}
> <property>
> <name>fetcher.timelimit.mins</name>
> <value>-1</value>
> <description>This is the number of minutes allocated to the fetching.
> Once this value is reached, any remaining entry from the input URL list is
> skipped
> and all active queues are emptied. The default value of -1 deactivates the
> time limit.
> </description>
> </property>
> {code}
> Note, *_fetcher.timelimit.mins_*
> I think that this essentially means the Fetcher has no time limit which is
> ofcourse not desired.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
