[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584532#comment-15584532 ] Hudson commented on HBASE-16712: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #1806 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1806/]) HBASE-16712 fix mvn hadoop-3.0 profile for mvn install (jmhsieh: rev 8e100628a45028365486e47858dad786cebb49b9) * (edit) pom.xml * (edit) hbase-resource-bundle/src/main/resources/META-INF/NOTICE.vm * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml * (edit) hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm HBASE-16712 ADDENDUM fix whitespace (jmhsieh: rev e1a6c94033bf4d986e72cd6109aac003b47e43d9) * (edit) hbase-resource-bundle/src/main/resources/supplemental-models.xml > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch, hbase-16712.v3.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583213#comment-15583213 ] Sean Busbey commented on HBASE-16712: - +1. Once this goes in, I'd like to make a follow on to switch from including a supplemental entry for Xerces to skipping it in LICENSE/NOTICE since we know it's not actually bundled. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch, hbase-16712.v3.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583067#comment-15583067 ] Jonathan Hsieh commented on HBASE-16712: Will fix whitespaces on commit. Tests shouldn't be affected by this patch. TestAdmin2, TestSnapshotCloneIndependence are on the flaky page. TestHCM was flakey for me locally, but I don't see how this would affect it. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch, hbase-16712.v2.patch, hbase-16712.v3.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576887#comment-15576887
]
Jonathan Hsieh commented on HBASE-16712:
In the next version I'll update the hadoop 3 version to 3.0.0-alpha1.
[~busbey] and I did some experiments. The xercesImpl being brought in seems to
be a bug in mvn (I'm using 3.3.9) not propagaing the excludes properly when
using -Dprofile.hadoop=3.0 . If I force the hadoop.profile to use 3.0 by
chaning the pom like below things work.
{code}
diff --git a/pom.xml b/pom.xml
index 3a24622..68600b9 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2046,7 +2046,8 @@
- !hadoop.profile
+ hadoop.profile
+ 2.0
@@ -2237,8 +2238,8 @@
hadoop-3.0
- hadoop.profile
- 3.0
+ !hadoop.profile
+
@@ -2316,7 +2317,7 @@
hadoop-hdfs
${hadoop-three.version}
test-jar
- test
+
javax.servlet.jsp
{code}
the xerces exclude is not needed in the hadoop3.0 profile.
I plan on documenting this in the example.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: h2-deps, h3-deps, hbase-16712.v0.patch,
> hbase-16712.v1.patch, hbase-16712.v2.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552653#comment-15552653
]
Hadoop QA commented on HBASE-16712:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 6s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 50s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 6s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 28s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12831994/hbase-16712.v2.patch |
| JIRA Issue | HBASE-16712 |
| Optional Tests | asflicense javac javadoc unit xml |
| uname | Linux 4aaf4d7d9d36 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7a38509 |
| Default Java | 1.8.0_101 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3858/testReport/ |
| modules | C: hbase-resource-bundle U: hbase-resource-bundle |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3858/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: h2-deps, h3-deps, hbase-16712.v0.patch,
> hbase-16712.v1.patch, hbase-16712.v2.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534079#comment-15534079 ] Sean Busbey commented on HBASE-16712: - dependency:tree uses the old maven 2 dependency walking, so that might explain a discrepancy in how it see the tree and how the remote resources plugin sees it. from the log, looks like xerces is coming from hadoop-hdfs test-jar, which is coming from hadoop-minicluster. but we exclude it from those in both the hadoop2 and hadoop3 profiles. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534064#comment-15534064 ] Jonathan Hsieh commented on HBASE-16712: No reason we shouldn't be able to that I'm aware of. I'll create a new jira to point there and send an email to dev@hbase to give a heads up. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15534053#comment-15534053 ] Sean Busbey commented on HBASE-16712: - hurm. looking at the logs, some stuff looks messed up because of the SNAPSHOT version. Any reason we can't switch from 3.0.0-SNAPSHOT to 3.0.0-alpha1? > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533939#comment-15533939 ] Jonathan Hsieh commented on HBASE-16712: [~busbey] check out the last commit here -- I've included the h?-deps and a log file with a -X of a 'mvn install -Dhadoop.profile=3.0. https://github.com/jmhsieh/hbase/commits/hbase-16712 > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533256#comment-15533256 ] Sean Busbey commented on HBASE-16712: - or if you're unsure about what kind of information gets shred about your build environment with maven in debug mode, you could send me the log out of band. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533253#comment-15533253
]
Sean Busbey commented on HBASE-16712:
-
{quote}The strange thing is that there is no xerces in the tree! If you comment
out the xerces bit in the supplemental, you end up with the error I posted
earlier. Where else would this dependency get pulled in from?
{quote}
yeah, this is very strange. Can you run the build that fails with a -X and then
post the log?
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: h2-deps, h3-deps, hbase-16712.v0.patch,
> hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533241#comment-15533241 ] Jonathan Hsieh commented on HBASE-16712: updated h3-deps to a dependency:tree that acutally has h3 in it. :) The strange thing is that there is no xerces in the tree! If you comment out the xerces bit in the supplemental, you end up with the error I posted earlier. Where else would this dependency get pulled in from? > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533222#comment-15533222 ] Jonathan Hsieh commented on HBASE-16712: Oops, too quick - something is not right with the deps files -- they are nearly identical, let me try again. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: h2-deps, h3-deps, hbase-16712.v0.patch, > hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533180#comment-15533180
]
Sean Busbey commented on HBASE-16712:
-
{quote}
On the new xerces dep – here's my evidence from the build:
{code}
--
This product includes Xerces2 Java Parser licensed under the The Apache
Software License, Version 2.0.
${dep.licenses[0].comments}
Please check this License for acceptability here:
https://www.apache.org/legal/resolved
If it is okay, then update the list named 'non_aggregate_fine' in the
LICENSE.vm file.
If it isn't okay, then revert the change that added the dependency.
More info on the dependency:
xerces
xercesImpl
2.9.1
maven central search
g:xerces AND a:xercesImpl AND v:2.9.1
project website
http://xerces.apache.org/xerces2-j
project source
http://svn.apache.org/viewvc/maven/pom/tags/apache-4/xercesImpl
{code}
Let me do a 'mvn clean install -DskipTests' with the different profiles, and
then I'll attach the deps for h2 and h3. I recently learned that 'mvn install'
is required for dependency:tree to give the right information.
{quote}
Thanks, that sounds good. Hadoop has been publishing an unneeded dependency on
Xerces since 2.6 or 2.7. This might mean they're just leaking it from a
different place, or it might mean that there is actually a need for downstream
clients now. I'd like us to know which it is before considering starting to
ship Xerces.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533176#comment-15533176
]
Sean Busbey commented on HBASE-16712:
-
{quote}
The Go license is brought in by re2j
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.0.0-SNAPSHOT:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-SNAPSHOT:compile
..
[INFO] | +- com.google.re2j:re2j:jar:1.0:compile
For the go license can you be more specific about what you suggest we do?
{quote}
BSD-3 clause licensed dependencies are only supposed to be mentioned in the
LICENSE file, not hte NOTICE file. So either we can include a supplemental xml
update that says re2j is BSD-3 clause instead of 'the go license', or we should
update the processing in the NOTICE file to ensure we don't include any
information for dependencies that use the go license.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533167#comment-15533167
]
Sean Busbey commented on HBASE-16712:
-
{quote}
We should have that (for LICENSE.vm and NOTICE.vm) error message googlable in
our docs so that others who run into it don't have to struggle to figure this
out as much.
{quote}
agreed. I thought we had an open JIRA for this, but it turns out I was thinking
of the one to redo things to rely on enforce (HBASE-16351).
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533161#comment-15533161
]
Sean Busbey commented on HBASE-16712:
-
{quote}
actually fail until the next line which you didn't include which does a
${dep.license[0]...) and gets a very cryptic array out of bounds error. I
actually got the new list by repeated going through looking at this output,
"fixing it", and then trying again until all were addressed. I'll leave more
explicit comments in the next rev.
{quote}
Right, but those lines could change and all a person gets is a line number in
the NOTICE.vm file to look at. That's why the LICENSE.vm file has an explicit
failure condition as a part of the "something is wrong" check, with a comment
that tells folks they need to look at the partially generated file for the
cause.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533150#comment-15533150
]
Jonathan Hsieh commented on HBASE-16712:
On the new xerces dep -- here's my evidence from the build:
{code}
--
This product includes Xerces2 Java Parser licensed under the The Apache
Software License, Version 2.0.
${dep.licenses[0].comments}
Please check this License for acceptability here:
https://www.apache.org/legal/resolved
If it is okay, then update the list named 'non_aggregate_fine' in the
LICENSE.vm file.
If it isn't okay, then revert the change that added the dependency.
More info on the dependency:
xerces
xercesImpl
2.9.1
maven central search
g:xerces AND a:xercesImpl AND v:2.9.1
project website
http://xerces.apache.org/xerces2-j
project source
http://svn.apache.org/viewvc/maven/pom/tags/apache-4/xercesImpl
{code}
Let me do a 'mvn clean install -DskipTests' with the different profiles, and
then I'll attach the deps for h2 and h3. I recently learned that 'mvn install'
is required for dependency:tree to give the right information.
The Go license is brought in by re2j
{code}
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.0.0-SNAPSHOT:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.0.0-SNAPSHOT:compile
..
[INFO] | +- com.google.re2j:re2j:jar:1.0:compile
{code}
For the go license can you be more specific about what you suggest we do?
Some of the failures wiht ${dep.licenses.isEmpty()} actually fail until the
next line which you didn't include which does a ${dep.license[0]...) and gets a
very cryptic array out of bounds error. I actually got the new list by
repeated going through looking at this output, "fixing it", and then trying
again until all were addressed. I'll leave more explicit comments in the next
rev.
We should have that (for LICENSE.vm and NOTICE.vm) error message googlable in
our docs so that others who run into it don't have to struggle to figure this
out as much.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530647#comment-15530647
]
Sean Busbey commented on HBASE-16712:
-
{code}
+#if( ${dep.licenses.isEmpty()} )
+ERROR: This product includes ${dep.name} which has no licenses!
+Revert the change if invalid or if intentional add license info to
sumpplemental-models.xml
+groupId: ${dep.groupId}
+articfactId: ${dep.artifactId}
+packaging: ${dep.packaging}
+version: ${dep.version}
+url: ${dep.url}
+#end
{code}
I don't think this will actually error out the build. Could you include a
failure inducing error with a pointer for fixing, [like the out of bounds
access in
LICENSE.vm|https://github.com/apache/hbase/blob/e51fcdd7788271d33cf02240155429358d71627b/hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm#L1675]?
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530644#comment-15530644
]
Sean Busbey commented on HBASE-16712:
-
{code}
+## See this FAQ link for justifications:
https://www.apache.org/legal/resolved.html
+## Note 'The Go license' is BSD 3 clause verbatim.
+#set($non_aggregate_fine = [ 'Public Domain', 'New BSD license', 'BSD
license', 'Mozilla Public License Version 2.0', 'Creative Commons Attribution
License, Version 2.5', 'The Go license', 'MPL 1.1'])
{code}
what's bringing in the go license, and can we update the uspplementation
information for it to properly name the BSD 3 clause instead? As-is, I think
this changeset will improperly include an entry in NOTICE for such a licensed
work, when it should only be in LICENSE.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530625#comment-15530625 ] Sean Busbey commented on HBASE-16712: - another bit on that Xerces dependency, there's supposed to be an enforcer rule that barfs on xercesImpl; did it not correctly yell about the inclusion? I don't see it getting removed in the patch. looking at the brought in dependencies, do you happen to have a tree list handy of how they get brought in? can we exclude more of them (curator-test, for example, is extremely suspicious as something needed to talk to HDFS)? Is the hadoop 3 profile making use of a different Hadoop dependency than Hadoop 2 profile? A quick skim shows that it includes a dependency on hadoop-annotations, is that needed? > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15530603#comment-15530603
]
Sean Busbey commented on HBASE-16712:
-
{quote}
+
+
+
+ xerces
+ xercesImpl
+
+
+
+ Apache License, Version 2.0
+ http://www.apache.org/licenses/LICENSE-2.0.txt
+ repo
+
+
+
+
{quote}
I've worked hard to keep Xerces out of our dependency set. Historically
Hadoop's use of the dependency has been limited to some tools that we don't
directly use (IIRC, the fsimage exporter). Is this dependency now required or
can we keep it out?
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15525057#comment-15525057
]
Hadoop QA commented on HBASE-16712:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
21s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 15s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
6s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 6s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
6s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 54s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12830440/hbase-16712.v1.patch |
| JIRA Issue | HBASE-16712 |
| Optional Tests | asflicense javac javadoc unit xml |
| uname | Linux e1ebdde433d9 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / db394f5 |
| Default Java | 1.8.0_101 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3725/testReport/ |
| modules | C: hbase-resource-bundle U: hbase-resource-bundle |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3725/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch, hbase-16712.v1.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[
https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524634#comment-15524634
]
Hadoop QA commented on HBASE-16712:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 7s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 7s
{color} | {color:green} hbase-resource-bundle in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 14s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12830421/hbase-16712.v0.patch |
| JIRA Issue | HBASE-16712 |
| Optional Tests | asflicense javac javadoc unit xml |
| uname | Linux f765ce717f96 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / b9ec59e |
| Default Java | 1.8.0_101 |
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/3721/artifact/patchprocess/whitespace-eol.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/3721/artifact/patchprocess/whitespace-tabs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3721/testReport/ |
| modules | C: hbase-resource-bundle U: hbase-resource-bundle |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3721/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> fix hadoop-3.0 profile mvn install
> --
>
> Key: HBASE-16712
> URL: https://issues.apache.org/jira/browse/HBASE-16712
> Project: HBase
> Issue Type: Bug
> Components: build, hadoop3
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-16712.v0.patch
>
>
> After the compile is fixed, mvn install fails due to transitive dependencies
> coming from hadoop3.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16712) fix hadoop-3.0 profile mvn install
[ https://issues.apache.org/jira/browse/HBASE-16712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524472#comment-15524472 ] Jonathan Hsieh commented on HBASE-16712: After including HBASE-16711, this command line completes successfully mvn clean test -DskipTests -Dhadoop.profile=3.0 install site [~busbey], you might want to take a look at this one. > fix hadoop-3.0 profile mvn install > -- > > Key: HBASE-16712 > URL: https://issues.apache.org/jira/browse/HBASE-16712 > Project: HBase > Issue Type: Bug > Components: build, hadoop3 >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-16712.v0.patch > > > After the compile is fixed, mvn install fails due to transitive dependencies > coming from hadoop3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
