[
https://issues.apache.org/jira/browse/MAPREDUCE-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897002#comment-16897002
]
Adam Antal commented on MAPREDUCE-7225:
---------------------------------------
Thanks for the patch [~pbacsko],
It looks that the code is correct and it does not throw URISyntaxException, but
I think the valid URI should be "hdfs://home/hadoop/". Could you please add
extra slash in line 400 and 403?
Just a question: since partly this behaviour is connected to the Path object,
maybe we can use a spy instead of a mock {{Path}} object? (This would couple
the test to the {{Path}}'s behaviour, which would rely upon anyways)
Another question: what happens if we give the root folder ("/") to the
{{JobSubmitter}}? Maybe we could include another testcase for that. This is
something that users should be discouraged, maybe even prohibited to do, but
obviously not the responsibility of this issue to incorporate that change.
Also I see 2 checkstyle problems - not sure about the second one is valid
though. Could you please take care of those?
> Fix broken current folder expansion during MR job start
> -------------------------------------------------------
>
> Key: MAPREDUCE-7225
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7225
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.9.0, 3.0.3
> Reporter: Adam Antal
> Assignee: Peter Bacsko
> Priority: Major
> Attachments: MAPREDUCE-7225-001.patch, MAPREDUCE-7225-002.patch,
> MAPREDUCE-7225-002.patch
>
>
> Starting a sleep job giving "." as files that should be localized is working
> fine up until 2.9.0, but after that the user is given an
> IllegalArgumentException. This change is a side-effect of HADOOP-12747 where
> {{GenericOptionsParser#validateFiles}} function got modified.
> Can be reproduced by starting a sleep job with "-files ." given as extra
> parameter. Log:
> {noformat}
> sudo -u hdfs hadoop jar hadoop-mapreduce-client-jobclient-3.0.0.jar sleep
> -files . -m 1 -r 1 -rt 2000 -mt 2000
> WARNING: Use "yarn jar" to launch YARN applications.
> 19/07/17 08:13:26 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm21
> 19/07/17 08:13:26 INFO mapreduce.JobResourceUploader: Disabling Erasure
> Coding for path: /user/hdfs/.staging/job_1563349475208_0017
> 19/07/17 08:13:26 INFO mapreduce.JobSubmitter: Cleaning up the staging area
> /user/hdfs/.staging/job_1563349475208_0017
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:168)
> at org.apache.hadoop.fs.Path.<init>(Path.java:180)
> at org.apache.hadoop.fs.Path.<init>(Path.java:125)
> at
> org.apache.hadoop.mapreduce.JobResourceUploader.copyRemoteFiles(JobResourceUploader.java:686)
> at
> org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:262)
> at
> org.apache.hadoop.mapreduce.JobResourceUploader.uploadResourcesInternal(JobResourceUploader.java:203)
> at
> org.apache.hadoop.mapreduce.JobResourceUploader.uploadResources(JobResourceUploader.java:131)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:99)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:194)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1726)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
> at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:273)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at
> org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:139)
> at
> org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:147)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
