[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908789#comment-16908789 ] Viraj Jasani edited comment on HBASE-22728 at 8/16/19 7:11 AM: --- If there is possibility of 1.3 release, let me work on backport to branch-1.3? was (Author: vjasani): If there is possibility of 1.3 release, let me work on backport to 1.3? > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5, 1.3.6 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch, > HBASE-22728.branch-1.18.patch, HBASE-22728.branch-1.19.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908609#comment-16908609
]
Andrew Purtell edited comment on HBASE-22728 at 8/16/19 1:46 AM:
-
Spot checks of {{mvn dependency:tree}} for hbase-annotations, hbase-common,
hbase-protocol, and hbase-client look good.
REST unit tests look good.
REST gateway test launched in tree looks good.
REST gateway test launched from binary tarball looks good.
Visual inspection of binary assembly looks good (except unrelated HBASE-22866).
Committing.
was (Author: apurtell):
Spot checks of {{mvn dependency:tree}} for hbase-annotations, hbase-common,
hbase-protocol, and hbase-client look good.
Unit tests look good.
REST gateway test launched in tree looks good.
REST gateway test launched from binary tarball looks good.
Visual inspection of binary assembly looks good (except unrelated HBASE-22866).
Committing.
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch,
> HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch,
> HBASE-22728.branch-1.18.patch, HBASE-22728.branch-1.19.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908146#comment-16908146
]
Viraj Jasani edited comment on HBASE-22728 at 8/15/19 3:02 PM:
---
{quote}this is surprising. Can we not just include the needed library as a
dependency of the hbase-assembly module?
{quote}
So far, this was not happening but after upgrading assembly plugin from 2.5 to
3.1.1, now we can include the needed lib as hbase-assembly dependency.
master branch has assembly plugin version 3.0.0
was (Author: vjasani):
{quote}this is surprising. Can we not just include the needed library as a
dependency of the hbase-assembly module?
{quote}
So far, this was not happening but after upgrading assembly plugin from 2.5 to
3.1.1, now we can include the needed lib as hbase-assembly dependency.
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch,
> HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch,
> HBASE-22728.branch-1.18.patch, HBASE-22728.branch-1.19.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907926#comment-16907926
]
Viraj Jasani edited comment on HBASE-22728 at 8/15/19 10:49 AM:
Just a small summary so far:
* Replaced all vulnerable mapper dependency(jackson-mapper-asl) with Jackson2
mapper(jackson-databind) in all modules.
* Included Jackson2 at compile scope in hbase-rest.
* hbase-shell requires dependency of jackson-core-asl. To tackle this, we
might need to upgrade JRuby eventually. For now, it's fine to include
jackson-core-asl(not vulnerable).
* Since HBase branch-1 no longer needs jackson-mapper-asl(as per #1), we can
live without it, but once we generate tar and extract it, we get these warnings
since Hadoop requires this dependency:
{code:java}
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper{code}
* Without including jackson-mapper-asl / Jackson2 dependencies as 'compile'
scope in hbase-common, we are not getting corresponding jars in lib folder of
extracted tarball. Need to resolve this issue since we should not include
jackson-mapper-asl with 'compile' scope in hbase-common/hbase-client/dependent
hbase-* of client.
was (Author: vjasani):
Just a small summary so far:
# Replaced all vulnerable mapper dependency(jackson-mapper-asl) with Jackson2
mapper(jackson-databind) in all modules.
# Included Jackson2 at compile scope in hbase-rest.
# hbase-shell requires dependency of jackson-core-asl. To tackle this, we
might need to upgrade JRuby eventually. For now, it's fine to include
jackson-core-asl(not vulnerable).
# Since HBase code no longer needs jackson-mapper-asl( #1), we can live
without it, but once we generate tar and extract it, we get these warnings
since Hadoop requires this dependency:
{code:java}
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper{code}
# Without including jackson-mapper-asl / Jackson2 dependencies as 'compile'
scope in hbase-common, we are not getting corresponding jars in lib folder of
extracted tarball. Need to resolve this issue since we should not include
jackson-mapper-asl with 'compile' scope in hbase-common/hbase-client/dependent
hbase-* of client.
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch,
> HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch,
> HBASE-22728.branch-1.18.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907889#comment-16907889 ] Viraj Jasani edited comment on HBASE-22728 at 8/15/19 8:03 AM: --- Oh yes, I just saw one project having hbase-common dependency. Hence, hbase-common should have provided scope for Jackson1. The only issue is without including dependencies at compile scope in hbase-common, they are not getting included as jar with assembly:single tar. Let me see what we can do here, may be some changes in hbase-assembly could help. Initially I tried including Jackson1 mapper as compile scope only in hbase-assembly(everywhere else had provided), but that didn't even include jar in lib of extracted tarball. was (Author: vjasani): Oh yes, I just saw one project having hbase-common dependency. Hence, hbase-common should have provided scope for Jackson1. The only issue is without including dependencies at compile scope in hbase-common, they are not getting included as jar with assembly:single tar. Let me see what we can do here, may be some changes in hbase-assembly could help. Initially I tried including Jackson1 mapper as compile scope only in hbase-assembly, but that didn't even have jackson*jar included in lib of extracted tarball. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch, > HBASE-22728.branch-1.18.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907025#comment-16907025 ] Viraj Jasani edited comment on HBASE-22728 at 8/14/19 8:42 AM: --- Thanks. Sure it should be good enough to call out in release note. With latest patch attached, we are including vulnerable mapper in hbase-common and hbase-assembly. was (Author: vjasani): Thanks. Sure it should be good enough to call out in release note. With latest patch attached, we are including vulnerable mapper only in hbase-common and hbase-assembly. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch, > HBASE-22728.branch-1.18.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906719#comment-16906719
]
Andrew Purtell edited comment on HBASE-22728 at 8/13/19 11:58 PM:
--
Not quite. Every process prints these warnings
{noformat}
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.ClassNotFoundException: org.codehaus.jackson.map.ObjectMapper
2019-08-13 16:32:34,148 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.SWebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,149 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org.apache.hadoop.hdfs.web.WebHdfsFileSystem
{noformat}
They aren't harmful but will result in bug reports.
Including the vulnerable mapper in our convenience binaries is fine albeit we
will want to call this out in a release note. It is Hadoop's requirement.
I think it's good enough to ensure hbase-client and its in project dependencies
(hbase-annotation, hbase-protocol, etc.) does not surprise by pulling in a
vulnerable version into a downstream project transitively.
Edit: And we have to make sure we don't use the vulnerable version in
hbase-rest too, of course.
was (Author: apurtell):
Not quite. Every process prints these warnings
{noformat}
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
2019-08-13 16:32:34,147 WARN [main] fs.FileSystem:
java.lang.ClassNotFoundException: org.codehaus.jackson.map.ObjectMapper
2019-08-13 16:32:34,148 WARN [main] fs.FileSystem: Cannot load filesystem:
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider
org.apache.hadoop.hdfs.web.SWebHdfsFileSystem could not be instantiated
2019-08-13 16:32:34,149 WARN [main] fs.FileSystem:
java.lang.NoClassDefFoundError: org.apache.hadoop.hdfs.web.WebHdfsFileSystem
{noformat}
They aren't harmful but will result in bug reports.
Including the vulnerable mapper in our convenience binaries is fine albeit we
will want to call this out in a release note. It is Hadoop's requirement.
I think it's good enough to ensure hbase-client and its in project dependencies
(hbase-annotation, hbase-protocol, etc.) does not surprise by pulling in a
vulnerable version into a downstream project transitively.
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch,
> HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906568#comment-16906568 ] Andrew Purtell edited comment on HBASE-22728 at 8/13/19 7:53 PM: - Let's step back and consider the basic motivation: We want to avoid putting vulnerable jackson dependencies on the classpath of unsuspecting user applications via transitive dependencies. An exception to this would be the shaded client, which of course must shade in those dependencies, but for this we can document a warning. So then we should try 'provided' or 'test' scope in client and then 'compile' scope anywhere else as needed, including or especially assembly, and that would meet the objective. We can say we have tried to do more, but it hasn't worked out. The hbase-rest changes are needed separately because we do use the object mapper there. was (Author: apurtell): Let's step back and consider the basic motivation: We want to avoid putting vulnerable jackson dependencies on the classpath of unsuspecting user applications via transitive dependencies. An exception to this would be the shaded client, which of course must shade in those dependencies, but for this we can document a warning. So then we should try 'provided' or 'test' scope in client and then 'compile' scope anywhere else as needed, including or especially assembly, and that would meet the objective. We can say we have tried to do more, but it hasn't worked out. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906568#comment-16906568 ] Andrew Purtell edited comment on HBASE-22728 at 8/13/19 7:52 PM: - Let's step back and consider the basic motivation: We want to avoid putting vulnerable jackson dependencies on the classpath of unsuspecting user applications via transitive dependencies. An exception to this would be the shaded client, which of course must shade in those dependencies, but for this we can document a warning. So then we should try 'provided' or 'test' scope in client and then 'compile' scope anywhere else as needed, including or especially assembly, and that would meet the objective. We can say we have tried to do more, but it hasn't worked out. was (Author: apurtell): Let's step back and consider the basic motivation: We want to avoid putting vulnerable jackson dependencies on the classpath of unsuspecting user applications via transitive dependencies. An exception to this would be the shaded client, which of course must shade in those dependencies, but for this we can document a warning. So then we should try 'provided' or 'test' scope in client and then 'compile' scope anywhere else as needed, including or especially assembly, would meet our objective. We can say we have tried to do more, but it hasn't worked out. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904381#comment-16904381 ] Viraj Jasani edited comment on HBASE-22728 at 8/13/19 7:00 PM: --- 'test' would not work for hbase-server since it has exposure of jackson dependency in source code. May be we can move to fasterxml.jackson for hbase-server too? Eventually we can backport HBASE-20587 to branch-1 but as part of this Jira, since we are moving to fasterxml.jackson for hbase-rest, may be we can stick to it for hbase-server too. Let me give it a shot and see if everything goes good including unpacking tarball and bringing up HMaster. was (Author: vjasani): 'test' would not work for hbase-server since it has exposure of jackson dependency in source code. May be we can move to fasterxml.jackson for hbase-server too and keep it at 'compile' scope(safer latest version)? Eventually we can backport HBASE-20587 to branch-1 but as part of this Jira, since we are moving to fasterxml.jackson for hbase-rest, may be we can stick to it for hbase-server too. Let me give it a shot and see if everything goes good including unpacking tarball and bringing up HMaster. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch, > HBASE-22728.branch-1.15.patch, HBASE-22728.branch-1.16.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904193#comment-16904193
]
Andrew Purtell edited comment on HBASE-22728 at 8/9/19 9:22 PM:
Still not done.
{{mvn clean install package assembly:single -DskipTests}} to create a tarball
in hbase-assembly/target/.
Unpack the tarball. Change directory into the tarball. Try to launch with
{{./bin/hbase master start}}
{noformat}
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.codehaus.jackson.map.ObjectMapper
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:227)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:162)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2844)
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
{noformat}
was (Author: apurtell):
Still not done.
{{mvn clean install package assembly:single -DskipTests}} to create a tarball.
Unpack the tarball. Change directory into the tarball. Try to launch with
{{./bin/hbase master start}}
{noformat}
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.codehaus.jackson.map.ObjectMapper
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:227)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:162)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2844)
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
{noformat}
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904193#comment-16904193
]
Andrew Purtell edited comment on HBASE-22728 at 8/9/19 9:19 PM:
Still not done.
{{mvn clean install package assembly:single -DskipTests}} to create a tarball.
Unpack the tarball. Change directory into the tarball. Try to launch with
{{./bin/hbase master start}}
{noformat}
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.codehaus.jackson.map.ObjectMapper
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:227)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:162)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2844)
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
{noformat}
was (Author: apurtell):
Still not done.
{{mvn clean install package assembly:single -DskipTests}} to create a tarball.
Unpack the tarball. Change directory into the tarball. Try to launch with
{{./bin/hbase master start}}
{{noformat}}
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMasterorg.codehaus.jackson.map.ObjectMapper
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:227)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:162)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:225)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2844)
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
{{noformat}}
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728-addendum.patch, HBASE-22728-addendum.patch,
> HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch,
> HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch,
> HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch,
> HBASE-22728.branch-1.12.patch, HBASE-22728.branch-1.14.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904164#comment-16904164
]
Andrew Purtell edited comment on HBASE-22728 at 8/9/19 8:03 PM:
So this is the result:
The compile scope appears only in hbase-rest.
Everything else is brought in at 'provided' or 'test' scopes.
Does this accomplish enough?
{noformat}
apurtell$ mvn dependency:tree|grep jackson
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +-
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.9.9:compile
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.9.9:compile
[INFO] | \-
com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.2:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.9.9:test
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.9.9:test
[INFO] | \-
com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.9.9:test
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.9:test
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.9.9:test
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.2:test
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
{noformat}
was (Author: apurtell):
So this is the result:
The compile scope is hbase-rest.
Everything else is 'provided' or 'test'.
Does this accomplish enough?
{noformat}
apurtell$ mvn dependency:tree|grep jackson
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +-
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.9.9:compile
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.9.9:compile
[INFO] | \-
com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.2:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:provided
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:provided
[INFO] | \-
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903273#comment-16903273 ] Andrew Purtell edited comment on HBASE-22728 at 8/8/19 7:49 PM: So this is annoying. I think we have to shade the org.codehaus.jackson dependencies and bundle them into our binary release, but not export them as a transitive dependency from Maven. Maybe a dependency on hbase-thirdparty is the way forward, because that kind of dependency shading is already done there. The minor release of 1.5.0, still pending, is an occasion where we can make this kind of change in our dependencies I think. was (Author: apurtell): So this is annoying. I think we have to shade the org.codehause.jackson dependencies and bundle them into our binary release, but not export them as a transitive dependency from Maven. Maybe a dependency on hbase-thirdparty is the way forward, because that kind of dependency shading is already done there. The minor release of 1.5.0, still pending, is an occasion where we can make this kind of change in our dependencies I think. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22728-addendum.patch, > HBASE-22728.branch-1.01.patch, HBASE-22728.branch-1.02.patch, > HBASE-22728.branch-1.04.patch, HBASE-22728.branch-1.06.patch, > HBASE-22728.branch-1.10.patch, HBASE-22728.branch-1.11.patch, > HBASE-22728.branch-1.12.patch > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902867#comment-16902867
]
Viraj Jasani edited comment on HBASE-22728 at 8/8/19 2:42 PM:
--
Thanks [~apurtell] and [~busbey] for all your inputs.
{quote}hbase-rest requires Jackson functionality; include at 'compile' scope
instead of 'provided' so we are functional out of the box
{quote}
[~apurtell] as part of this Jira, considering removal of RemoteHTable can be
done later, would you prefer to have 'compile' scope for hbase-rest and
exclusions for all com.fasterxml.jackson* for hbase-rest dependency inside
hbase-examples?
I just tried it locally with the above exclusion, the build seems fine and this
is the dependency tree:
{code:java}
[INFO] < org.apache.hbase:hbase-rest >-
[INFO] Building Apache HBase - Rest 1.5.0-SNAPSHOT [17/33]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-rest ---
[INFO] org.apache.hbase:hbase-rest:jar:1.5.0-SNAPSHOT
[INFO] +-
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.9.9:compile
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.9.9:compile
[INFO] | \-
com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.2:compile
[INFO] \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO]\- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO]
{code}
{code:java}
[INFO] --< org.apache.hbase:hbase-examples >---
[INFO] Building Apache HBase - Examples 1.5.0-SNAPSHOT [21/33]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-examples ---
[INFO] org.apache.hbase:hbase-examples:jar:1.5.0-SNAPSHOT
[INFO] \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO]\- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO]
{code}
was (Author: vjasani):
Thanks [~apurtell] and [~busbey] for all your inputs.
{quote}hbase-rest requires Jackson functionality; include at 'compile' scope
instead of 'provided' so we are functional out of the box
{quote}
[~apurtell] as part of this Jira, considering removal of RemoteHTable can be
done later, would you prefer to have 'compile' scope for hbase-rest and
exclusions for all com.fasterxml.jackson* for hbase-rest dependency inside
hbase-examples?
I just tried it locally with the above exclusion, the build seems fine and this
is the dependency tree:
{code:java}
[INFO] < org.apache.hbase:hbase-rest >-
[INFO] Building Apache HBase - Rest 1.5.0-SNAPSHOT [17/33]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-rest ---
[INFO] org.apache.hbase:hbase-rest:jar:1.5.0-SNAPSHOT
[INFO] +-
com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.9.9:compile
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.9.9:compile
[INFO] | \-
com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.2:compile
[INFO] \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO]\- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO]
{code}
{code:java}
[INFO] --< org.apache.hbase:hbase-examples >---
[INFO] Building Apache HBase - Examples 1.5.0-SNAPSHOT [21/33]
[INFO] [ jar ]-
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-examples ---
[INFO] org.apache.hbase:hbase-examples:jar:1.5.0-SNAPSHOT
[INFO] \- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
[INFO]\- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:test
[INFO]
{code}
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902323#comment-16902323
]
Andrew Purtell edited comment on HBASE-22728 at 8/7/19 5:24 PM:
[~busbey] I don't understand the shaded artifacts precommit failures
{noformat}
WARNING] Rule 4: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion
failed with message:
HBase has unsupported dependencies.
HBase requires that all dependencies be compiled with version 1.7 or earlier
of the JDK to properly build from source. You appear to be using a newer
dependency. You can use
either "mvn -version" or "mvn enforcer:display-info" to verify what version
is active.
Non-release builds can temporarily build with a newer JDK version by setting
the
'compileSource' property (eg. mvn -DcompileSource=1.8 clean package).
Found Banned Dependency: jdk.tools:jdk.tools:jar:1.8
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
{noformat}
If the build requires JDK 7, why are we using Java 8 to run the precommit job?
Can we make it JDK 7 only?
was (Author: apurtell):
[~busbey] I don't understand the shaded artifacts precommit failures
{noformat}
WARNING] Rule 4: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion
failed with message:
HBase has unsupported dependencies.
HBase requires that all dependencies be compiled with version 1.7 or earlier
of the JDK to properly build from source. You appear to be using a newer
dependency. You can use
either "mvn -version" or "mvn enforcer:display-info" to verify what version
is active.
Non-release builds can temporarily build with a newer JDK version by setting
the
'compileSource' property (eg. mvn -DcompileSource=1.8 clean package).
Found Banned Dependency: jdk.tools:jdk.tools:jar:1.8
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
{noformat}
If the build required JDK 7, why are we using Java 8 to run the precommit job?
Can we make it JDK 7 only?
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
> Attachments: HBASE-22728.branch-1.01.patch,
> HBASE-22728.branch-1.02.patch, HBASE-22728.branch-1.04.patch,
> HBASE-22728.branch-1.06.patch, HBASE-22728.branch-1.10.patch,
> HBASE-22728.branch-1.11.patch, HBASE-22728.branch-1.12.patch
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894541#comment-16894541 ] Viraj Jasani edited comment on HBASE-22728 at 7/27/19 8:17 PM: --- Even hbase-common doesn't need this dependency. was (Author: vjasani): Even hbase-common doesn't need this dependency > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894541#comment-16894541 ] Viraj Jasani edited comment on HBASE-22728 at 7/27/19 8:17 PM: --- Even hbase-common doesn't require this dependency. was (Author: vjasani): Even hbase-common doesn't need this dependency. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892893#comment-16892893
]
Andrew Purtell edited comment on HBASE-22728 at 7/25/19 3:47 PM:
-
Yes, we shade org.codehaus.**
{noformat}
org.codehaus
org.apache.hadoop.hbase.shaded.org.codehaus
{noformat}
Should we exclude it from hbase-shaded-client and also add exclusions to
whatever pulls it in to hbase-client?
was (Author: apurtell):
Yes, we shade org.codehaus.**
Should we exclude it from hbase-shaded-client and also add exclusions to
whatever pulls it in to hbase-client?
> Upgrade jackson dependencies in branch-1
>
>
> Key: HBASE-22728
> URL: https://issues.apache.org/jira/browse/HBASE-22728
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.4.10, 1.3.5
>Reporter: Andrew Purtell
>Priority: Major
> Fix For: 1.5.0, 1.3.6, 1.4.11
>
>
> Avoid Jackson versions and dependencies with known CVEs
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[
https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892281#comment-16892281
]
Andrew Purtell edited comment on HBASE-22728 at 7/25/19 12:13 AM:
--
This is what we have in branch-1
{noformat}
[INFO] org.apache.hbase:hbase-common:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.avro:avro:jar:1.7.7:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-procedure:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-client:jar:1.5.0-SNAPSHOT
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-metrics-api:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-common:jar:1.5.0-SNAPSHOT:compile
... (via avro)
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-hadoop-compat:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-common:jar:1.5.0-SNAPSHOT:compile
(via avro)
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-metrics:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-common:jar:1.5.0-SNAPSHOT:compile
... (via avro)
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-hadoop2-compat:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.8.5:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:2.8.5:compile
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] | | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-prefix-tree:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-server:jar:1.5.0-SNAPSHOT
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-testing-util:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-client:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- org.apache.hbase:hbase-server:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.8.5:compile
[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-thrift:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-client:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- org.apache.hbase:hbase-server:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.8.5:compile
[INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:2.8.5:compile
[INFO] | | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-rest:jar:1.5.0-SNAPSHOT
[INFO] +- com.sun.jersey:jersey-json:jar:1.9:compile
[INFO] | \- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:test
{noformat}
{noformat}
[INFO] org.apache.hbase:hbase-rsgroup:jar:1.5.0-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-client:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- org.apache.hbase:hbase-server:jar:1.5.0-SNAPSHOT:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile
[INFO] +-
[jira] [Comment Edited] (HBASE-22728) Upgrade jackson dependencies in branch-1
[ https://issues.apache.org/jira/browse/HBASE-22728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892292#comment-16892292 ] Andrew Purtell edited comment on HBASE-22728 at 7/25/19 12:09 AM: -- Although this is the Codehaus Jackson and not the Fasterxml Jackson and specifically the jackson-databind artifact is not pulled in, the CVE does implicate org.codehaus.jackson:jackson-mapper-asl 1.9.13. (See https://www.cvedetails.com/cve/CVE-2017-7525/). We are not going to see a new version of Codehaus Jackson and we are not going to be able to upgrade to Fasterxml Jackson without maybe breaking downstreamers by removing org.codehaus.jackson artifacts from the classpath (pulled in now by hbase-client, hbase-server, and others). I think we have to do a code analysis to determine if we call ObjectMapper#readValue directly or indirectly using untrusted user input to determine if RCE in an HBase process is possible. hbase-rest has the most exposure I'd imagine. An upgrade of Jersey and Jackson dependencies in hbase-rest alone would be a contained change. To do this we'd essentially backport the branch-2 version of hbase-rest. Otherwise I wonder if we can fix our POMs to not expose a dependency on jackson-mapper-asl so we wouldn't pull a vulnerable version into someone else's project unnecessarily. [~Apache9] [~busbey] [~elserj] [~stack] [~toffer] was (Author: apurtell): Although this is the Codehaus Jackson and not the Fasterxml Jackson and specifically the jackson-databind artifact is not pulled in, the CVE does implicate org.codehaus.jackson:jackson-mapper-asl 1.9.13. (See https://www.cvedetails.com/cve/CVE-2017-7525/). We are not going to see a new version of Codehaus Jackson and we are not going to be able to upgrade to Fasterxml Jackson without maybe breaking downstreamers by removing org.codehaus.jackson artifacts from the classpath (pulled in now by hbase-client, hbase-server, and others). I think we have to do a code analysis to determine if we call ObjectMapper#readValue directly or indirectly using untrusted user input to determine if RCE in an HBase process is possible. hbase-rest has the most exposure I'd imagine. An upgrade of Jersey and Jackson dependencies in hbase-rest alone would be a contained change. To do this we'd essentially backport the branch-2 version of hbase-rest. Otherwise I wonder if we can fix our POMs to not expose a dependency on jackson-mapper-asl so we wouldn't pull a vulnerable version into someone else's project unnecessarily. > Upgrade jackson dependencies in branch-1 > > > Key: HBASE-22728 > URL: https://issues.apache.org/jira/browse/HBASE-22728 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.4.10, 1.3.5 >Reporter: Andrew Purtell >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > > Avoid Jackson versions and dependencies with known CVEs -- This message was sent by Atlassian JIRA (v7.6.14#76016)
