[
https://issues.apache.org/jira/browse/HADOOP-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540721
]
Arun C Murthy commented on HADOOP-1952:
---------------------------------------
Lohit, I see your latest patch removes some combiner options... are you sure
about these changes? If so, are they related to this jira?
{noformat}
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java
===================================================================
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java
(revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingFailure.java
(working copy)
@@ -47,7 +47,6 @@
"-input", INVALID_INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
//"-verbose",
//"-jobconf", "stream.debug=set"
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java
===================================================================
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java
(revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestGzipInput.java
(working copy)
@@ -47,7 +47,6 @@
"-input", INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
"-jobconf", "stream.recordreader.compression=gzip"
};
Index:
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java
===================================================================
---
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java
(revision 588543)
+++
src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreaming.java
(working copy)
@@ -40,7 +40,6 @@
protected String map = StreamUtil.makeJavaCommand(TrApp.class, new
String[]{".", "\\n"});
// reduce behave like /usr/bin/uniq. But also prepend lines with R.
// command-line combiner does not have any effect any more.
- protected String combine = StreamUtil.makeJavaCommand(UniqApp.class, new
String[]{"C"});
protected String reduce = StreamUtil.makeJavaCommand(UniqApp.class, new
String[]{"R"});
protected String outputExpect =
"Rare\t\nRblue\t\nRbunnies\t\nRpink\t\nRred\t\nRroses\t\nRviolets\t\n";
@@ -66,7 +65,6 @@
"-input", INPUT_FILE.getAbsolutePath(),
"-output", OUTPUT_DIR.getAbsolutePath(),
"-mapper", map,
- "-combiner", combine,
"-reducer", reduce,
//"-verbose",
//"-jobconf", "stream.debug=set"
{noformat}
> Streaming does not handle invalid -inputformat (typo by users for example)
> ---------------------------------------------------------------------------
>
> Key: HADOOP-1952
> URL: https://issues.apache.org/jira/browse/HADOOP-1952
> Project: Hadoop
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.14.1
> Reporter: lohit vijayarenu
> Assignee: lohit vijayarenu
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: CatchInvalidInputFormat.patch, HADOOP-1952-1.patch,
> HADOOP-1952-2.patch
>
>
> Hadoop Streaming does not handle invalid inputformat class. For example
> -inputformat INVALID class would not be thrown as an error. Instead it
> defaults to StreamInputFormat. If an invalid inputformat is specified, it is
> good to fail.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
