sebastian-nagel commented on code in PR #958:
URL: https://github.com/apache/nutch/pull/958#discussion_r3986597317
##########
src/bin/nutch:
##########
@@ -147,17 +147,18 @@ fi
local=true
-# NUTCH_JOB
-if [ -f "${NUTCH_HOME}"/*nutch*.job ]; then
- local=false
- for f in "$NUTCH_HOME"/*nutch*.job; do
+# NUTCH_JOB
+for f in "${NUTCH_HOME}"/*nutch*.job; do
Review Comment:
If there are two or more job files in NUTCH_HOME, that's already an issue
very likely. But ok for this fix. If we are lucky the last job file is the
right one.
##########
src/bin/nutch:
##########
@@ -336,7 +337,7 @@ else
# check that hadoop can be found on the path
if [ $(which hadoop | wc -l ) -eq 0 ]; then
echo "Can't find Hadoop executable. Add HADOOP_COMMON_HOME/bin to the path
or run in local mode."
- exit -1;
Review Comment:
Maybe we should change the behavior of Generator and exit with a more unique
code, in case no segment was created (nothing to fetch). But would be a
breaking change.
##########
src/bin/nutch:
##########
@@ -336,7 +337,7 @@ else
# check that hadoop can be found on the path
if [ $(which hadoop | wc -l ) -eq 0 ]; then
echo "Can't find Hadoop executable. Add HADOOP_COMMON_HOME/bin to the path
or run in local mode."
- exit -1;
Review Comment:
This is somewhat sensitive because the script bin/crawl expects that
`bin/nutch generate ...` exits with code 1 when no segment was created. All
errors should ideally use other error codes.
Of course, this is not 100% working, because the Generator uses Hadoop's
ToolRunner which maps exceptions (don't whether all) to 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
