[
https://issues.apache.org/jira/browse/BIGTOP-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13651548#comment-13651548
]
Mark Grover commented on BIGTOP-949:
------------------------------------
Thanks for looking into this, Jarcec! I appreciate your insight and thoughts.
What you are saying makes sense to me.
I also tested Sqoop against Bigtop 0.5 (which has Hadoop 2.0.2-alpha) and Sqoop
import job failed there as well with the following error:
{code}
Error: java.lang.NullPointerException
at java.lang.String.<init>(String.java:593)
at
org.apache.sqoop.job.mr.ConfigurationUtils.loadConfiguration(ConfigurationUtils.java:77)
at
org.apache.sqoop.job.mr.ConfigurationUtils.getConnectorConnection(ConfigurationUtils.java:38)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:69)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
{code}
Note that this error even though looks similar (related to retrieving secrets
in credentials) at the first glance is a little different. The error on Bigtop
0.6 (hence Hadoop 2.0.4-alpha) happens in Job Commit phase (aka output
committer) while the error on Bigtop 0.5 (hence Hadoop 2.0.2-alpha) happens in
the map task.
My hunch is this related to HADOOP-8726. If I were to *guess*, I would say
while HADOOP-8726 made the credentials available to MR jobs, the
OutputCommitter committer was missed, like Jarcec mentioned.
> Add Sqoop tests
> ---------------
>
> Key: BIGTOP-949
> URL: https://issues.apache.org/jira/browse/BIGTOP-949
> Project: Bigtop
> Issue Type: Bug
> Components: Tests
> Affects Versions: 0.6.0
> Reporter: Mark Grover
> Assignee: Mark Grover
> Priority: Blocker
> Fix For: 0.6.0
>
>
> We added Sqoop2 (1.99 something) to Bigtop recently. We should add some
> integration tests for it as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
