Patrick Mézard created NUTCH-2791:
-------------------------------------
Summary: domainstats, protocolstats and crawlcomplete do not
handle GCS URLs
Key: NUTCH-2791
URL: https://issues.apache.org/jira/browse/NUTCH-2791
Project: Nutch
Issue Type: Bug
Components: util
Affects Versions: 1.17
Reporter: Patrick Mézard
I am running Nutch in GCP Dataproc. The domainstats, protocolstats and
crawlcomplete commands do not resolve crawldb paths correctly from GCS URLs.
Also:
* protocolstats has an off-by-one error when resolving the numReducers argument
* crawlcomplete -inputDirs is inconsistent with other command: where
domainstats expect directories containing a "current" path, crawlcomplete looks
for directories containing a "crawldb/current" path.
I will send a PR after creating the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
