Ivan Veselovsky created IGNITE-1176:
---------------------------------------
Summary: Ignite Hadoop fails to unpack a DistributedCache archive
with fragment identifier
Key: IGNITE-1176
URL: https://issues.apache.org/jira/browse/IGNITE-1176
Project: Ignite
Issue Type: Bug
Reporter: Ivan Veselovsky
Assignee: Ivan Veselovsky
Encountered this issue trying to set up TPC-DS benchmarks for Ignite.
When running the preperation hadoop job, it loads a jar archive into the
DistributedCache . But the jar URL has fragment identifier meaning the symlink
name to be created from job local dir. Some trouble seems to be there, because
on Ignite node I observe the following exception.
It looks like
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager#processFiles
method does not cut off the fragment identifier when trying to unpack the
archive.
[22:08:20,400][ERROR][ignite-#9%pub-null%][HadoopJobTracker] Failed to submit
job: 4e23a716-f22d-44a5-986b-7d79c941ec7e_1
class org.apache.ignite.IgniteCheckedException: Cannot unpack archive
[path=/tmp/-2be2732670ff4dfb167ff6671307bd82.jar#dsdgen,
jobId=4e23a716-f22d-44a5-986b-7d79c941ec7e_1]
at
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager.prepareJobEnvironment(HadoopV2JobResourceManager.java:163)
at
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2Job.initialize(HadoopV2Job.java:264)
at
org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.job(HadoopJobTracker.java:1010)
at
org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker.submit(HadoopJobTracker.java:268)
at
org.apache.ignite.internal.processors.hadoop.HadoopProcessor.submit(HadoopProcessor.java:185)
at
org.apache.ignite.internal.processors.hadoop.HadoopImpl.submit(HadoopImpl.java:69)
at
org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:48)
at
org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolSubmitJobTask.run(HadoopProtocolSubmitJobTask.java:31)
at
org.apache.ignite.internal.processors.hadoop.proto.HadoopProtocolTaskAdapter$Job.execute(HadoopProtocolTaskAdapter.java:94)
at
org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:482)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6147)
at
org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:476)
at
org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:429)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot unpack archive
[path=/tmp/-2be2732670ff4dfb167ff6671307bd82.jar#dsdgen,
jobId=4e23a716-f22d-44a5-986b-7d79c941ec7e_1]
at
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager.processFiles(HadoopV2JobResourceManager.java:242)
at
org.apache.ignite.internal.processors.hadoop.v2.HadoopV2JobResourceManager.prepareJobEnvironment(HadoopV2JobResourceManager.java:146)
... 16 more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
