[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623648#comment-15623648
]
Sangjin Lee commented on HADOOP-13410:
--
Opened HADOOP-13776.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha1
>
> Attachments: HADOOP-13410.001.patch, HADOOP-13410.002.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623567#comment-15623567
]
Andrew Wang commented on HADOOP-13410:
--
Thanks for the clarification Sangjin. Yea, I think we need a new JIRA. Even
though we missed setting the fix version for the 3.0.0-alpha1 changelog, we can
still set it now so it's correct in JIRA.
If you want, we could reuse the revert JIRA (HADOOP-13620) for the second patch
since alpha2 hasn't gone out yet, but it's probably more clear to just use a
new JIRA.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha1
>
> Attachments: HADOOP-13410.001.patch, HADOOP-13410.002.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623442#comment-15623442
]
Sangjin Lee commented on HADOOP-13410:
--
Sorry it is bit confusing.
The original commit for this JIRA did make alpha 1 (although the fix version
was not marked as such because I thought mistakenly it was committed too late
for alpha 1).
Then [~bibinchundatt] found it breaks MR job submission, and it was reverted in
HADOOP-13620 which is on alpha 2 (the issue was discovered after alpha 1 was
released).
I then reopened this JIRA to fix the original issue correctly. That's what the
2nd patch is about.
[~andrew.wang], I take that your suggestion is not to reuse this JIRA but open
a new one to fix this issue correctly?
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha1
>
> Attachments: HADOOP-13410.001.patch, HADOOP-13410.002.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622683#comment-15622683
]
Sangjin Lee commented on HADOOP-13410:
--
The unit test failure is unrelated. I would greatly appreciate feedback. Thanks!
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch, HADOOP-13410.002.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15617053#comment-15617053
]
Hadoop QA commented on HADOOP-13410:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 51s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.net.TestClusterTopology |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13410 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835921/HADOOP-13410.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux c276506dd745 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 112f04e |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10925/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10925/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10925/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
>
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616684#comment-15616684
]
Matt Motter commented on HADOOP-13410:
--
[~sjlee0],
{quote}
That should address your issue too, correct?
{quote}
Yes, removing the unpacked root ("unpacked/") and adding the jar back in, while
still retaining the unpacked classes ("unpacked/classes") and libs
("unpacked/libs") would fix the issue I described above.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616572#comment-15616572
]
Sangjin Lee commented on HADOOP-13410:
--
Thanks for the input [~matt.mot...@cerner.com]. I am now thinking that we need
to keep the jar over the unpacked root ("unpacked/"), which is the opposite of
the previous attempt to solve this (we would still need to retain
"unpacked/classes" and "unpacked/libs").
That should address your issue too, correct?
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616434#comment-15616434
]
Matt Motter commented on HADOOP-13410:
--
I realize this issue has already been resolved, but this muddies the waters a
bit for users who leverage an uber jar with AOT compilation. This is because
the unpacked jar modifies time stamps to when the command is running. Doing so
causes timestamps for .clj files and there respective class files to have the
same timestamp, which causes Clojure to always dynamically recompile the source.
In previous releases the original jar was being added to the class path after
the unpacked jar. It would have been more benefical for users if the original
jar was added before the unpacked jar so its class files would have been
leveraged over the now updated class files within the unpacked jar. The only
work around to this is to specify the original jar on the HADOOP_CLASSPATH and
set HADOOP_USER_CLASSPATH_FIRST true, which most users tend to forget to do and
just becomes added work to run commands.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502054#comment-15502054
]
Yuanbo Liu commented on HADOOP-13410:
-
Sorry about breaking the trunk, I will review it soon.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494049#comment-15494049
]
Hudson commented on HADOOP-13410:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10445 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10445/])
Revert "HADOOP-13410. RunJar adds the content of the jar twice to the (sjlee:
rev fcbac00212e99c91bd1368e002823d1260b3879f)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494023#comment-15494023
]
Bibin A Chundatt commented on HADOOP-13410:
---
[~sjlee0]
HADOOP-13410 .Patch is available in trunk and 3.0.0-alpha1 also.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493969#comment-15493969
]
Bibin A Chundatt commented on HADOOP-13410:
---
[~sjlee0]
{quote}
Could you revert it (it's only on trunk)? Let me know if you need my help
{quote}
i don't have rights for the same.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493948#comment-15493948
]
Sangjin Lee commented on HADOOP-13410:
--
Yes, in light of that, we should revert this and rework it. Could you revert it
(it's only on trunk)? Let me know if you need my help. Thanks for reporting the
issue!
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493933#comment-15493933
]
Bibin A Chundatt commented on HADOOP-13410:
---
IMHO we should revert this patch. [~sjlee0] thoughts?
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493238#comment-15493238
]
Bibin A Chundatt commented on HADOOP-13410:
---
[~sjlee0]/[~yuanbo]
Could you please look into MAPREDUCE-6779 IIUC this jira is Job submission
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423366#comment-15423366
]
Sangjin Lee commented on HADOOP-13410:
--
This should not have any bearing (one way or the other) on HADOOP-12728. If I
understood correctly, HADOOP-12728 seems to be an issue of the order between
the jar in the argument and what's in the underlying CLASSPATH or
HADOOP_CLASSPATH. This JIRA concerns removing the redundant entries for the
jar, and does not affect the above ordering problem.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422952#comment-15422952
]
Allen Wittenauer commented on HADOOP-13410:
---
I wonder how much this makes HADOOP-12728 worse. In other words, if the jar
pointed to by the user is trying to override a class in the classpath, does
this make it worse?
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418300#comment-15418300
]
Hudson commented on HADOOP-13410:
-
SUCCESS: Integrated in Hadoop-trunk-Commit #10266 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10266/])
HADOOP-13410. RunJar adds the content of the jar twice to the classpath (sjlee:
rev 4d3ea92f4fe4be2a0ee9849c65cb1c91b0c5711b)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418284#comment-15418284
]
Yuanbo Liu commented on HADOOP-13410:
-
[~sjlee0] Thanks for your committing and [~raviprak] thanks for your review.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Fix For: 3.0.0-alpha2
>
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418147#comment-15418147
]
Ravi Prakash commented on HADOOP-13410:
---
Sure! Here's my +1 too for trunk
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418128#comment-15418128
]
Sangjin Lee commented on HADOOP-13410:
--
I'm +1 with the change. Do let me know if there is feedback or objections to
the change. I am thinking of committing this only to 3.0.0 (unless anyone needs
this on 2.x).
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414547#comment-15414547
]
Sangjin Lee commented on HADOOP-13410:
--
There is more than "/". We also add "/classes/" and "/libs/*" to the classpath.
Since we unpack it to add different elements to the classpath, it would make
more sense to remove the jar from the classpath IMO.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414531#comment-15414531
]
Ravi Prakash commented on HADOOP-13410:
---
Hi Sangjin! Thanks for your contribution. Is there a reason you are choosing to
remove the jar instead of the classes which have been expanded? I'm obviously
+1 for keeping just one of those around. Am wondering which one we should keep,
and which one we should let go.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414527#comment-15414527
]
Sangjin Lee commented on HADOOP-13410:
--
[~qwertymaniac], you're right. HADOOP-9424 is somewhat similar but can be
considered separately. I'll comment there.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413827#comment-15413827
]
Harsh J commented on HADOOP-13410:
--
There is a use-case that requires something similar, see HADOOP-9424. Its not
directly related to what is being pointed out here though.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413770#comment-15413770
]
Sangjin Lee commented on HADOOP-13410:
--
Thanks [~yuanbo]! The patch does what the JIRA calls for, and I tested it
locally.
That said, I'd like to find out from the community if there is any reason that
the jar itself needs to remain in the classpath after the unjarred content is
added to the classpath. I'll ask the community.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391579#comment-15391579
]
Hadoop QA commented on HADOOP-13410:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
3s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819873/HADOOP-13410.001.patch
|
| JIRA Issue | HADOOP-13410 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux eff7a9286295 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 7052ca8 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10073/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10073/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
> Attachments: HADOOP-13410.001.patch
>
>
> Today when you run a "hadoop jar" command,
[jira] [Commented] (HADOOP-13410) RunJar adds the content of the jar twice to the classpath
[
https://issues.apache.org/jira/browse/HADOOP-13410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391483#comment-15391483
]
Yuanbo Liu commented on HADOOP-13410:
-
[~sjlee0] Agree with you that there is no need to add the jar path to
classpath. I think {{TestRunJar#testClientClassLoader}} has covered the test
case, so I did not write a test case for this patch. Hope to get your thoughts.
> RunJar adds the content of the jar twice to the classpath
> -
>
> Key: HADOOP-13410
> URL: https://issues.apache.org/jira/browse/HADOOP-13410
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Reporter: Sangjin Lee
>Assignee: Yuanbo Liu
>
> Today when you run a "hadoop jar" command, the jar is unzipped to a temporary
> location and gets added to the classloader.
> However, the original jar itself is still added to the classpath.
> {code}
> List classPath = new ArrayList<>();
> classPath.add(new File(workDir + "/").toURI().toURL());
> classPath.add(file.toURI().toURL());
> classPath.add(new File(workDir, "classes/").toURI().toURL());
> File[] libs = new File(workDir, "lib").listFiles();
> if (libs != null) {
> for (File lib : libs) {
> classPath.add(lib.toURI().toURL());
> }
> }
> {code}
> As a result, the contents of the jar are present in the classpath *twice* and
> are completely redundant. Although this does not necessarily cause
> correctness issues, some stricter code written to require a single presence
> of files may fail.
> I cannot think of a good reason why the jar should be added to the classpath
> if the unjarred content was added to it. I think we should remove the jar
> from the classpath.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
