[
https://issues.apache.org/jira/browse/PIG-5087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818643#comment-15818643
]
Koji Noguchi commented on PIG-5087:
-----------------------------------
e2e continue to surprise me with undocumented feature, this time with "pig23"
param.
{code:title=test/e2e/pig/tests/nightly.conf}
# test streaming
'num' => 3,
'pig' => q\
a = load ':INPATH:/singlefile/studenttab10k' using PigStorage() as (name, age,
gpa);
b = mapreduce ':MAPREDJARS:/hadoop-streaming.jar' Store a into
':OUTPATH:.intermediate.1' Load ':OUTPATH:.intermediate.2' as (name:chararray,
count: int) `-input :OUTPATH:.intermediate.1 -output :OUTPATH:.intermediate.2
-mapper cat -reducer wc`;
store b into ':OUTPATH:';\,
'pig23' => q\
a = load ':INPATH:/singlefile/studenttab10k' using PigStorage() as (name, age,
gpa);
b = mapreduce ':MAPREDJARS:/hadoop-0.23.0-streaming.jar' Store a into
':OUTPATH:.intermediate.1' Load ':OUTPATH:.intermediate.2' as (name:chararray,
count: int) `-input :OUTPATH:.intermediate.1 -output :OUTPATH:.intermediate.2
-mapper cat -reducer wc`;
store b into ':OUTPATH:';\,
'notmq' => 1,
},
{code}
After PIG-4923, there's no more hadoopversion=23 and started picking up old
streaming jar leading to the class not found error.
> e2e Native3 failing after PIG-4923 (dropping hadoop 1.x)
> --------------------------------------------------------
>
> Key: PIG-5087
> URL: https://issues.apache.org/jira/browse/PIG-5087
> Project: Pig
> Issue Type: Test
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Trivial
>
> After hadoop 1.x was dropped in PIG-4923, e2e Native 3 test started failing
> with
> {noformat}
> 2017-01-10 22:23:38,070 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.streaming.PipeMapRunner not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2248)
> at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:1127)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1850)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.streaming.PipeMapRunner not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2216)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2240)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.streaming.PipeMapRunner not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2122)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)
> ... 9 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
