[
https://issues.apache.org/jira/browse/NUTCH-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081587#comment-13081587
]
Markus Jelsma commented on NUTCH-1028:
--------------------------------------
Distributed mode is recommended indeed and in production there is no problem
due to the counter. I, and likely others (with slower computers), will at least
see progress with `large` local test parses. I was also looking in trunk but
wasn't really sure where the parsing actually happens in ParserJob.
Is it here?
{code}
URLWebPage redirectedPage = parseUtil.process(key, page);
ParseStatus pstatus = page.getParseStatus();
{code}
> Log parser keys
> ---------------
>
> Key: NUTCH-1028
> URL: https://issues.apache.org/jira/browse/NUTCH-1028
> Project: Nutch
> Issue Type: Task
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Trivial
> Fix For: 1.4, 2.0
>
> Attachments: NUTCH-1028-1.4-1.patch
>
>
> The parser can take ages (many hours) to complete. During this time the only
> output is an error or warning if it's unable to parse something (which is
> very common). Sometimes the parser can run for several hours without any
> output: this is scary. I propose to add a LOG.info to the mapper and write
> the key when parsing, similar to the fetcher.
> Thoughts?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
