[
https://issues.apache.org/jira/browse/IGNITE-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904968#comment-15904968
]
Ivan Veselovsky commented on IGNITE-4813:
-----------------------------------------
Pull request: https://github.com/apache/ignite/pull/1610
> Ignite map-reduce engine should set MRJobConfig.TASK_ATTEMPT_ID
> ---------------------------------------------------------------
>
> Key: IGNITE-4813
> URL: https://issues.apache.org/jira/browse/IGNITE-4813
> Project: Ignite
> Issue Type: Bug
> Components: hadoop
> Affects Versions: 1.8
> Reporter: Ivan Veselovsky
> Assignee: Ivan Veselovsky
> Fix For: 2.0
>
>
> Hadoop "join" example fails on Ignite with the error like this:
> {code}
> Out: class org.apache.ignite.IgniteCheckedException: class
> org.apache.ignite.IgniteCheckedException: null
> [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:102)
> [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Task.run(HadoopV2Task.java:55)
> [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.run(HadoopV2TaskContext.java:266)
> [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.runTask(HadoopRunnableTask.java:209)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call0(HadoopRunnableTask.java:144)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:116)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:114)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.runAsJobOwner(HadoopV2TaskContext.java:573)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:114)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:46)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService$2.body(HadoopExecutorService.java:186)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> java.lang.Thread.run(Thread.java:745)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: Caused by:
> java.lang.NullPointerException
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.<init>(TaskAttemptContextImpl.java:49)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.hadoop.mapreduce.lib.join.Parser$WNode.createRecordReader(Parser.java:348)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.hadoop.mapreduce.lib.join.Parser$CNode.createRecordReader(Parser.java:486)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.hadoop.mapreduce.lib.join.CompositeInputFormat.createRecordReader(CompositeInputFormat.java:143)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:69)
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: ... 12
> more
> [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out:
> {code}
> This is because
> org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Context sets the
> job id and task id, but does not set task attempt id. In Hadoop this is done
> in method org.apache.hadoop.mapred.Task#localizeConfiguration .
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
