[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080933#comment-17080933
]
Hadoop QA commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
36s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
33s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
37s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 6s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed TAP tests | hadoop_add_common_to_classpath.bats.tap |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.8 Server=19.03.8 Image:yetus/hadoop:e6455cc864d |
| JIRA Issue | HADOOP-13344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12825353/HADOOP-13344.01.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml shellcheck shelldocs |
| uname | Linux 476565e80f7b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 275c478 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| shellcheck | v0.3.7 |
| TAP logs |
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406910#comment-16406910
]
genericqa commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 30s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
11s{color} | {color:red} The patch generated 1 new + 20 unchanged - 0 fixed =
21 total (was 20) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
19s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 1m
38s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 23s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed TAP tests | hadoop_add_common_to_classpath.bats.tap |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HADOOP-13344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12825353/HADOOP-13344.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml shellcheck shelldocs |
| uname | Linux ecc6cfdffefa 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / fe224ff |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| shellcheck | v0.4.6 |
|
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185030#comment-16185030
]
Hadoop QA commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 new + 20 unchanged - 0 fixed =
21 total (was 20) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
30s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m
17s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
25s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 0s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 77m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.metrics2.sink.TestFileSink |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HADOOP-13344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12825353/HADOOP-13344.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml shellcheck shelldocs |
| uname | Linux 6c93c4f49a50 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ca669f9 |
| Default Java | 1.8.0_144 |
| shellcheck | v0.4.6 |
| shellcheck |
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069349#comment-16069349
]
Hadoop QA commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 new + 21 unchanged - 0 fixed =
22 total (was 21) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
24s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 16s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed TAP tests | hadoop_add_common_to_classpath.bats.tap |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-13344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12825353/HADOOP-13344.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml shellcheck shelldocs |
| uname | Linux 57971ac18f5f 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / af2773f |
| Default Java | 1.8.0_131 |
| shellcheck | v0.4.6 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12667/artifact/patchprocess/diff-patch-shellcheck.txt
|
| TAP logs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12667/artifact/patchprocess/patch-hadoop-common-project_hadoop-common.tap
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12667/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12667/testReport/ |
| modules | C: hadoop-assemblies hadoop-common-project/hadoop-common U: . |
| Console output |
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556662#comment-15556662
]
Thomas Poepping commented on HADOOP-13344:
--
Hey Allen, re #1:
I wrote it wrongly. The intention was for
"${HADOOP_USE_BUILTIN_SLF4J_BINDING:-true}", which will set the variable to
"true" if it is currently unset [1]
I'll look into resolving your comments for #2.
[1]
http://stackoverflow.com/questions/27445455/what-does-the-colon-dash-mean-in-bash
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.01.patch, HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491220#comment-15491220
]
Allen Wittenauer commented on HADOOP-13344:
---
Two problems:
#1:
{code}
if [[ -n "${HADOOP_USE_BUILTIN_SLF4J_BINDING}:-true" ]];
{code}
This will always be true because there will always be a value for -n to
evaluate. You can test this on the command line:
{code}
$ echo "${HADOOP_USE_BUILTIN_SLF4J_BINDING}:-true"
:-true
{code}
-n is probably the wrong thing to use here. I'd recommend reconstructing so
that you compare HADOOP_USE_BUILTIN_SLF4J_BINDING against an explicit true or
false setting.
#2:
There are unit tests for the shell scripts. In this case,
hadoop-common-project/hadoop-common/src/test/scripts/hadoop_add_common_to_classpath.bats
failed because the value generated by hadoop_add_common_to_classpath is now
generating a different value than what was previously expected due to the extra
classpath being present. The test case should be updated to test both settings
of HADOOP_USE_BUILTIN_SLF4J_BINDING.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.01.patch, HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491179#comment-15491179 ] Thomas Poepping commented on HADOOP-13344: -- Can someone point to what failed in the previous build? I'm not sure what went wrong. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.01.patch, HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491126#comment-15491126 ] Allen Wittenauer commented on HADOOP-13344: --- bq. Hm, didn't get a test run. I'll resubmit the patch? It's becoming more and more common that the precommit-admin job that fires off the other job that actually does the work fails because builds.apache.org times out. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.01.patch, HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491106#comment-15491106
]
Hadoop QA commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 new + 75 unchanged - 0 fixed =
76 total (was 75) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
9s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
12s{color} | {color:green} hadoop-assemblies in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 4s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed TAP tests | hadoop_add_common_to_classpath.bats.tap |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HADOOP-13344 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12825353/HADOOP-13344.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml shellcheck shelldocs |
| uname | Linux b0d429e82bd8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2a8f55a |
| Default Java | 1.8.0_101 |
| shellcheck | v0.4.4 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10515/artifact/patchprocess/diff-patch-shellcheck.txt
|
| TAP logs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/10515/artifact/patchprocess/patch-hadoop-common-project_hadoop-common.tap
|
| unit |
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15490991#comment-15490991 ] Thomas Poepping commented on HADOOP-13344: -- Hm, didn't get a test run. I'll resubmit the patch? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.01.patch, HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435167#comment-15435167 ] Thomas Poepping commented on HADOOP-13344: -- Terribly sorry guys, I have a different diff now that should clear a few things up. I'll get that uploaded today and we can continue the conversation from there. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434542#comment-15434542 ] Steve Loughran commented on HADOOP-13344: - grepping inside the JARs scares me. I'm particularly worried about what happens if someone sticks an uber JAR on the CP, such as groovy-all or spark-assembly: excluding the JAR as it has SL4FJ in runs the risk of removing things. It's also going to force a grep of every line of every JAR, if I'm not mistaken, which will be expensive. Can't you just have a variable listing wildcards of JARs to blacklist? Easy to test, easy to understand, less hidden intelligence/obscure support calls > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15433256#comment-15433256 ] Thomas Poepping commented on HADOOP-13344: -- We cannot, because we may not have access to the other classpath. The problem here is in applications with their own classpath that consume the hadoop classpath, rather than something like a hadoop jar that wants to use its own SLF4J binding. Does that make sense? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419727#comment-15419727 ] Allen Wittenauer commented on HADOOP-13344: --- Maybe we are thinking about the problem wrong. Could we check to see if a binding already exists and if not add one to the classpath? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419433#comment-15419433 ] Thomas Poepping commented on HADOOP-13344: -- I would say we can't unconditionally remove the binding, because hadoop code itself will still need an slf4j binding to accurately log items it tries to log. I'll work to getting my updated patch posted here > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419313#comment-15419313
]
Sean Busbey commented on HADOOP-13344:
--
For trunk, are we better off just unconditionally removing the binding?
Presuming [~aw] still feels this way:
{quote}
* I think moving it to a place that may be optionally triggered off is a really
the only realistic fix here. Munging the classpath to remove it (especially
when we tend to use wildcards to speed things up), is just painful.
* Since that path is default on, this shouldn't be an incompatible break. Users
who turn it off are making the decision to disable it the same way they would a
plug-in. This is nearly the same sorts of decisions we give users to turn
things on, such as special options to fsck.
{quote}
then sure, I'd be fine with the pom fixes as a follow-up so long as the
solution here doesn't obviously rely on something that would change as a result.
If we're going to unconditionally remove the binding then I'd consider the
maven fixes blocking, wether they happen here or in a different JIRA.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419298#comment-15419298 ] Thomas Poepping commented on HADOOP-13344: -- this seems like a different issue, but I understand what you're saying. as it stands now, it looks like hadoop-mapreduce-project, hadoop-mapreduce-client, and hadoop-minikdc all have the slf4j binding as a compile dependency, rather than a runtime dependency. changing this would exclude it for dependent projects. I would recommend a different JIRA for this issue, however, because this is not the issue I'm trying to solve here. The issue I'm trying to solve is that hadoop-config.sh will place the slf4j binding on the classpath no matter what. Applications that must be installed with hadoop to function (e.g. hive) will end up having Hadoop's slf4j-binding on their classpath, even if they don't want it there. This option would give the opportunity, in this instance, to remove that dependency from the classpath. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419277#comment-15419277 ] Sean Busbey commented on HADOOP-13344: -- Right now, sure, you'd use an exclusion to remove it. But it being there at all is a bug. We already exclude a ton of other transitive dependencies that aren't needed for the downstream facing client. Why would we not also exclude this one? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419263#comment-15419263 ] Thomas Poepping commented on HADOOP-13344: -- so let's say I'm a maven based consumer of hadoop client, i.e. I have a dependency on some hadoop-client artifact in my pomfile. What would you suggest? How would I include or exclude this binding in my dependency tree? There's already a builtin way to exclude it -- using . Why go around that? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419243#comment-15419243
]
Sean Busbey commented on HADOOP-13344:
--
{quote}
If there is a downstream project including hadoop-common (or any other
component), I say it's the responsibility of that project to exclude the slf4j
binding if they don't want it. I don't believe any changes to pom.xml are
necessary.
{quote}
disagree. Specifically, if folks are using the hadoop-client artifact(s) those
are expressly supposed to be a library for downstream users. SLF4j is pretty
clear on their guidance for libraries wrt including bindings. Just have the API
and let an end user decide if they need logging to go somewhere.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419235#comment-15419235 ] Thomas Poepping commented on HADOOP-13344: -- Coming back to this, what downstream users would we expect? The problem we're trying to solve is that the slf4j binding is in the HADOOP_CLASSPATH variable, setup by hadoop-config.sh. That is not controllable by a downstream project, which is why I'm changing it here. If there is a downstream project including hadoop-common (or any other component), I say it's the responsibility of that project to exclude the slf4j binding if they don't want it. I don't believe any changes to pom.xml are necessary. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403204#comment-15403204 ] Sean Busbey commented on HADOOP-13344: -- it would be easier (for me at least) to comment on a specific patch rather than a description. at first blush, it doesn't sound like your proposal changes any of the declared dependencies within maven. If that's correct, then downstream users will still pull in the dependency when they shouldn't. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402638#comment-15402638 ] Thomas Poepping commented on HADOOP-13344: -- bump? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385074#comment-15385074
]
Thomas Poepping commented on HADOOP-13344:
--
Sorry for the late response guys, had some other stuff come up.
I've experimented some with the patch for trunk, and it looks like I should be
able to change the directory structure in
hadoop-assemblies/src/main/resources/assemblies/hadoop-dist.xml. I've added a
new directory: slf4j-lib for each component under
/share/hadoop/${hadoop.component}/lib, and verified that the slf4j lib is not
added for common.
Does this seem like it should work to you guys? The change "for each classpath"
only affects common and the webserver modules. I wanted to run the
implementation by you guys -- it doesn't seem as complicated as you made it
sound, so I'm sure I'm missing something.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
>Assignee: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373711#comment-15373711 ] Sean Busbey commented on HADOOP-13344: -- downstream folks ought to be using hadoop-client, so we'd have to change that. given recent developments I guess we'd need to change hdfs-client as well. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373399#comment-15373399 ] Thomas Poepping commented on HADOOP-13344: -- I don't think we would want (or need to) change dependency information for anything but common, because that's the classpath most often used by other applications. I'm taking a deeper look into it. It may be that the best solution is to remove the slf4j binding from every classpath, and I agree, that would be a big change. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373028#comment-15373028 ] Sean Busbey commented on HADOOP-13344: -- I think Allen means that you might have to alter poms to make any slf4j binding dependencies test scope (presuming the tests use biding-specific configuration files currently). I think Steve is pointing to the same issue wrt applications that create assemblies, if our existing poms include an slf4j binding as a runtime or compile dependency. If we remove the bindings from all those poms, we might then have to add it back in somewhere to ensure one is available in those places where we want a binding (namely when launching our daemons). If we have to change the root pom then the patch will probably be too big for jenkins, since it will trigger a full build across all modules. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15372708#comment-15372708 ] Steve Loughran commented on HADOOP-13344: - one more piece of fun here: any app which creates an assembly JAR (example: spark, maybe hive) and which does not then shadow SLF4J ends up including the slf4j classes, so making it near-impossible to guarantee that all duplicate copies of SLF4J have been removed from the CP. Painful > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371819#comment-15371819 ] Thomas Poepping commented on HADOOP-13344: -- I'm sorry Allen, I don't think I understand how that makes this a script and pom patch for any poms but the root? Wouldn't we just change the directory structure and where we put the slf4j binding, then change the script to optionally *not* add that directory to the classpath based on a variable ($HADOOP_REMOVE_SLF4J_BINDING maybe) ? Thanks for the help. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371790#comment-15371790 ] Allen Wittenauer commented on HADOOP-13344: --- I've been thinking about this off and on all day. IMHO: * I get why slf4j doesn't want multiple bindings in place as a CYA maneuver but it's still annoying. (officially, believe it or not, there is no set order for a JVM to process the classpath. I actually did the research when cleaning up the shell code a few years ago. So the docs are correct in that it might be random, but unofficially... changing that would break the world.) * I think moving it to a place that may be optionally triggered off is a really the only realistic fix here. Munging the classpath to remove it (especially when we tend to use wildcards to speed things up), is just painful. * Since that path is default on, this shouldn't be an incompatible break. Users who turn it off are making the decision to disable it the same way they would a plug-in. This is nearly the same sorts of decisions we give users to turn things on, such as special options to fsck. * Unofficially, this will break things like Apache Big Top which does a lot of strange and weirdo directory munging since it literally re-arranges the distribution as built by Apache Hadoop. But live by the sword, die by the sword. This will effectively turn this into a script + pom patch. As a result, it *might* end up being too big for Jenkins if all of the poms have slf4j listed as a dependency. So we need to keep an eye on that. One thing I forgot to mention: bq. I assume that any documentation that already exists regarding hadoop-config.sh should have this feature documented. branch-2's scripts are completely and totally undocumented. That's not true for trunk. Documentation for the var to turn this on/off should get added to hadoop-env.sh. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371374#comment-15371374 ] Sean Busbey commented on HADOOP-13344: -- removing things from the classpath breaks compatibility because downstream folks probably rely on things being there. (those this type of compatibility is not one that hadoop as a project makes promises about ([ref compatibility as of 2.7.2|hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Classpath]) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371160#comment-15371160 ] Thomas Poepping commented on HADOOP-13344: -- I think that's a good solution for trunk. There's less messing about with grep and find and all that. I'm not sure how you guys want to proceed with branch-2. How do you think it might break compatibility? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping >Assignee: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371148#comment-15371148 ] Sean Busbey commented on HADOOP-13344: -- yes, for client classpath we have been doing things wrong so far as SLF4J is concerned. (presuming you think of the client classpath as being for using our code as a library to interact with the cluster) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371141#comment-15371141 ] Allen Wittenauer commented on HADOOP-13344: --- Reading http://www.slf4j.org/codes.html, I'm inclined to think we're doing this wrong in Hadoop. (Like that never happens...) Maybe we should pull the binding out, put it in a separate dir, then use a var to pull that dir in. I suspect that will break compatibility in 2.x though. (Although if hdfs-client can go in with it's insanity, this isn't much worse.) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371137#comment-15371137 ] Sean Busbey commented on HADOOP-13344: -- we should do the work for trunk on this jira. once we have a solution that works for trunk we can figure out how much is applicable to branch-2. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371117#comment-15371117 ] Thomas Poepping commented on HADOOP-13344: -- Would you suggest we keep this open, and wait until the patch is accepted to trunk before coming back to it? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371115#comment-15371115 ] Thomas Poepping commented on HADOOP-13344: -- The problem is not that the user slf4j binding isn't being used, it's that slf4j will report an error to stderr : "SLF4J: Class path contains multiple SLF4J bindings ... etc" https://github.com/qos-ch/slf4j/blob/c2463021fe0944a03d3fdcc14c3914c86d4a7a4f/slf4j-api/src/main/java/org/slf4j/LoggerFactory.java#L319 This change is to provide users a way around that. We default to *not* using this option (a bit of iffy not logic, I know), so that we don't unintentionally hide this error for people it might negatively affect. To your other points, understood. I'll do some looking into how we can find the offending slf4j binding in the most platform independent way. > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371084#comment-15371084 ] Allen Wittenauer commented on HADOOP-13344: --- Maybe I'm being dense (won't be the first time, won't be the last), but why doesn't just having custom bindings in the classpath first not work? > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15370973#comment-15370973 ] Steve Loughran commented on HADOOP-13344: - ...I'm with Sean here: let's get the feature into trunk, then do a version which works in branch-2. Even if the implementations are different, we don't want a feature in branch-2 which isn't in trunk. Expect [~aw] to have opinions on trunk patches, as he is the 1ary reviewer of changes to the scripts there > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365228#comment-15365228
]
Thomas Poepping commented on HADOOP-13344:
--
That was my mistake, I didn't mean to include 2.7.2 as a target version, only
2.8.0.
I found while writing this that we could use 'jar tf ${jarname}', but that was
always much slower than grepping the jarfile directly.
As far as documentation, I don't have any plans. I assume that any
documentation that already exists regarding hadoop-config.sh should have this
feature documented. As far as client applications have documentation, I don't
see any good way to do it. We can't very well change the documentation of each
application depending on Hadoop. It could also be added to a hadoop-config help
method, if one exists. I'll look into it.
First and foremost though -- I'll be targeting trunk. The reason I targeted
this directly is because the structure of the hadoop-config file is very
different in trunk.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: New Feature
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365211#comment-15365211 ] Sean Busbey commented on HADOOP-13344: -- thanks for working on this. I was just complaining about the inclusion of slf4j bindings in this classpath earlier today. :) > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: New Feature > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365206#comment-15365206 ] Thomas Poepping commented on HADOOP-13344: -- Thanks for the quick response -- I'm working on a patch that targets trunk as well > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: Improvement > Components: bin, scripts >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365208#comment-15365208
]
Sean Busbey commented on HADOOP-13344:
--
-1 (non-binding) on this going into 2.7.z. Maintenance releases aren't the
place to change the dependency set, even if it's opt-in. It invites folks to
rely on behavior that they won't be able to use if they have to revert to an
earlier maintenance release.
How are we planning to document this as an option for the client classpath?
Will operators have to have looked at the source of our scripts to know that
this feature exists?
{code}
+ find $parent_directory -maxdepth 1 -name "*.jar" -exec grep -HLs \
+ "org/slf4j/impl/StaticLoggerBinder.class" {} \; \
+ | paste -sd ':' -
{code}
At this point in bootstrapping, do we have access to a JRE that we could use to
enumerate jar contents (or some zip accessor)? that'd be more reliable than
greping the jar file directly.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: Improvement
> Components: bin, scripts
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365193#comment-15365193 ] Sean Busbey commented on HADOOP-13344: -- But you should make a patch that targets master first, since changes need to land there before going back to 2.y > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: Improvement > Components: bin >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[ https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365192#comment-15365192 ] Sean Busbey commented on HADOOP-13344: -- see this section on the wiki page linked above: https://wiki.apache.org/hadoop/HowToContribute#Naming_your_patch > Add option to exclude Hadoop's SLF4J binding > > > Key: HADOOP-13344 > URL: https://issues.apache.org/jira/browse/HADOOP-13344 > Project: Hadoop Common > Issue Type: Improvement > Components: bin >Affects Versions: 2.8.0, 2.7.2 >Reporter: Thomas Poepping > Labels: patch > Attachments: HADOOP-13344.patch > > > If another application that uses the Hadoop classpath brings in its own SLF4J > binding for logging, and that jar is not the exact same as the one brought in > by Hadoop, then there will be a conflict between logging jars between the two > classpaths. This patch introduces an optional setting to remove Hadoop's > SLF4J binding from the classpath, to get rid of this problem. > This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure > has been changed in 3.0.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13344) Add option to exclude Hadoop's SLF4J binding
[
https://issues.apache.org/jira/browse/HADOOP-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365040#comment-15365040
]
Hadoop QA commented on HADOOP-13344:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HADOOP-13344 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12816498/HADOOP-13344.patch |
| JIRA Issue | HADOOP-13344 |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9931/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add option to exclude Hadoop's SLF4J binding
>
>
> Key: HADOOP-13344
> URL: https://issues.apache.org/jira/browse/HADOOP-13344
> Project: Hadoop Common
> Issue Type: Improvement
> Components: bin
>Affects Versions: 2.8.0, 2.7.2
>Reporter: Thomas Poepping
> Labels: patch
> Attachments: HADOOP-13344.patch
>
>
> If another application that uses the Hadoop classpath brings in its own SLF4J
> binding for logging, and that jar is not the exact same as the one brought in
> by Hadoop, then there will be a conflict between logging jars between the two
> classpaths. This patch introduces an optional setting to remove Hadoop's
> SLF4J binding from the classpath, to get rid of this problem.
> This patch should be applied to 2.8.0, as bin/ and hadoop-config.sh structure
> has been changed in 3.0.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
