[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821186#comment-17821186 ] PJ Fanning commented on HADOOP-18197: - I have https://github.com/apache/hadoop-thirdparty/pull/34 and have done some basic testing with Hadoop. I'm running https://github.com/apache/hadoop/pull/6593 as an experiment. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821183#comment-17821183 ] Steve Loughran commented on HADOOP-18197: - +1 for moving to 3.23; trying to maintain someone else's open source project, while a right, is not a duty. * we will have to get a new shaded lib out ASAP for RC3 * we should include in the release notes that people should upgrade their JDKs anyway, and that a future 3.4.x release will drop support for java 8. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820711#comment-17820711 ] Ayush Saxena commented on HADOOP-18197: --- If protobuf 3.23 has a fix & released, I think we should prefer that rather than patching protobuf ourselves in our code. I think we should attempt upgrading protobuf version to the one which has the fix, if that fails then we can try the pathing approach > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820707#comment-17820707 ] PJ Fanning commented on HADOOP-18197: - The fix only seems to be in protobuf-java 3.23 and above - https://github.com/protocolbuffers/protobuf/commit/d40aadf823cf7e1e62b65561656f689da8969463 Issue - https://github.com/protocolbuffers/protobuf/issues/11393 The options seem to be * sticking with the shaded protobuf 3_7 jar * upgrading the CI boxes to use JDKs where the issue doesn't happen and adding release notes * trying protobuf-java 3.23 instead of 3.21 * patching our shaded protobuf-java 3.21 jar - we could get the source of protobuf-java 3.21.12, apply the fix above and release a hadoop-shaded-protobuf_3_21 1.2.1 None of these are great but I favour the idea of patching our shaded protobuf-java 3.21.12. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement > Components: hadoop-thirdparty >Affects Versions: thirdparty-1.2.0 >Reporter: Ivan Viaznikov >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available, security > Fix For: thirdparty-1.2.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819229#comment-17819229
]
Steve Loughran commented on HADOOP-18197:
-
saw this in the context of surefire upgrade #6537.
java.lang.NoSuchMethodError:
java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer; is a recurrent nightmare;
it's a change in the ByteBuffer classic which added an overloaded position()
method in java9 *which was then backported to java8*
some java8 jvms are happy, some fail. and if you compile with a newer JVM with
the change (openjdk, corretto) you generate code which doesn't work everywhere.
fixable in code with a cast, but you need to edit the code
{code}
((Buffer)buffer).position()l
{code}
> Update protobuf 3.7.1 to a version without CVE-2021-22569
> -
>
> Key: HADOOP-18197
> URL: https://issues.apache.org/jira/browse/HADOOP-18197
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hadoop-thirdparty
>Affects Versions: thirdparty-1.2.0
>Reporter: Ivan Viaznikov
>Assignee: PJ Fanning
>Priority: Major
> Labels: pull-request-available, security
> Fix For: thirdparty-1.2.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The artifact `org.apache.hadoop:hadoop-common` brings in a dependency
> `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version
> released in 2013 and it contains a vulnerability
> [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569].
> Therefore, requesting you to clarify if this library version is going to be
> updated in the following releases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818651#comment-17818651
]
Ayush Saxena commented on HADOOP-18197:
---
I think this is causing some trouble due to some incompatibilities introduced
in Java
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1503/testReport/junit/org.apache.hadoop.hdfs.protocol/TestBlockListAsLongs/testFuzz/
{noformat}
java.lang.NoSuchMethodError:
java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at
org.apache.hadoop.thirdparty.protobuf.IterableByteBufferInputStream.read(IterableByteBufferInputStream.java:143)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.read(CodedInputStream.java:2080)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.tryRefillBuffer(CodedInputStream.java:2831)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.refillBuffer(CodedInputStream.java:2777)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawByte(CodedInputStream.java:2859)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64SlowPath(CodedInputStream.java:2648)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64(CodedInputStream.java:2641)
at
org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readSInt64(CodedInputStream.java:2497)
{noformat}
Rolling back the thirdparty upgrade from trunk, did solve this, so I doubt it
could be because of this change (didn't investigate further), there are lot of
other failures as well in the daily build, which I have just started to look
> Update protobuf 3.7.1 to a version without CVE-2021-22569
> -
>
> Key: HADOOP-18197
> URL: https://issues.apache.org/jira/browse/HADOOP-18197
> Project: Hadoop Common
> Issue Type: Improvement
> Components: hadoop-thirdparty
>Affects Versions: thirdparty-1.2.0
>Reporter: Ivan Viaznikov
>Assignee: PJ Fanning
>Priority: Major
> Labels: pull-request-available, security
> Fix For: thirdparty-1.2.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The artifact `org.apache.hadoop:hadoop-common` brings in a dependency
> `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version
> released in 2013 and it contains a vulnerability
> [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569].
> Therefore, requesting you to clarify if this library version is going to be
> updated in the following releases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756952#comment-17756952 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1686463509 says 3.21.x... we should take the latest one we can which doesn't include other surprises...pr and jira can be set to the final version which goes in as it is merged > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17756029#comment-17756029 ] ASF GitHub Bot commented on HADOOP-18197: - janjwerner-confluent commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1684101336 @steveloughran The title states upgrade protobuf to 3.21.7 while the version downloaded is 3.21.1. hope you can bump it up whenever you'll get back to this. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741520#comment-17741520 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1628605375 usual ongoing protobuf issues; AFAIK none of them lethal. YMMV I do want #4996 in so we can get protobuf 2.5 off the classpath. if you could take that up, it'd be good. that PR doesn't cut it, only make it optional. a followup would cut it. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741475#comment-17741475 ] ASF GitHub Bot commented on HADOOP-18197: - abhishekagarwal87 commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1628362069 @steveloughran - is this a genuine CVE in `hadoop-shaded-protobuf` or is it just to please the scanner gods? :) > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706870#comment-17706870 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1490261364 @xizhu-mstr @tooptoop4 i'm not actively working on this; too many pressing issues and after getting 3.3.5 out the door I'm catching up with the internal stuff. Either of you two want to take it on? I'd also like to get #4996 in; if anyone wants to run with that, I'd be very happy. We shouldn't need protobuf 2.5 on the CP given we aren't using it > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706867#comment-17706867 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1153223024 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: yeah > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706779#comment-17706779 ] ASF GitHub Bot commented on HADOOP-18197: - xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo. Should be protobuf-3.21 > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706777#comment-17706777 ] ASF GitHub Bot commented on HADOOP-18197: - xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694332#comment-17694332 ] ASF GitHub Bot commented on HADOOP-18197: - tooptoop4 commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1119615045 ## hadoop-project/pom.xml: ## @@ -87,10 +87,10 @@ 2.5.0 Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available, security > Time Spent: 1h 10m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694304#comment-17694304 ] t oo commented on HADOOP-18197: --- CVE-2022-3510 and CVE-2022-3509 h1. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available, security > Time Spent: 1h 10m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17623569#comment-17623569 ] t oo commented on HADOOP-18197: --- CVE-2022-3171 > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available, security > Time Spent: 1h 10m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17621198#comment-17621198
]
ASF GitHub Bot commented on HADOOP-18197:
-
hadoop-yetus commented on PR #4418:
URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1285780046
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 46m 14s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 45s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 29m 26s | | trunk passed |
| +1 :green_heart: | compile | 22m 17s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 7m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 30m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 1m 11s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| -1 :x: | compile | 0m 59s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| -1 :x: | javac | 0m 59s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -1 :x: | mvnsite | 0m 48s |
[/patch-mvnsite-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/patch-mvnsite-root.txt)
| root in the patch failed. |
| +1 :green_heart: | xmllint | 0m 0s | | No new issues. |
| -1 :x: | javadoc | 7m 33s |
[/results-javadoc-javadoc-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/results-javadoc-javadoc-root.txt)
| root generated 534 new + 2269 unchanged - 0 fixed = 2803 total (was 2269) |
| -1 :x: | shadedclient | 9m 53s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 7m 42s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/patch-unit-root.txt)
| root in the patch failed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 189m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4418 |
| Optional Tests | dupname asflicense codespell detsecrets shellcheck
shelldocs hadolint mvnsite unit compile javac javadoc mvninstall shadedclient
xmllint |
| uname | Linux eeeb6886f515 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4f05bf48185e1cb3edce862286a3fc01b41ea451 |
| Default Java | Red Hat, Inc.-1.8.0_345-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/testReport/ |
| Max. process+thread count | 530 (vs. ulimit of 5500) |
| modules | C: hadoop-project . U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/3/console |
| versions | git=2.9.5 maven=3.6.3 xmllint=20901 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615238#comment-17615238
]
ASF GitHub Bot commented on HADOOP-18197:
-
hadoop-yetus commented on PR #4418:
URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1273639382
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 37m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 37s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 5s | | trunk passed |
| +1 :green_heart: | compile | 21m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 19m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 7m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 0m 48s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| -1 :x: | mvninstall | 0m 20s |
[/patch-mvninstall-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | mvninstall | 0m 22s |
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt)
| hadoop-yarn-api in the patch failed. |
| -1 :x: | compile | 0m 40s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| -1 :x: | javac | 0m 40s |
[/patch-compile-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-compile-root.txt)
| root in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -1 :x: | mvnsite | 0m 31s |
[/patch-mvnsite-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-mvnsite-root.txt)
| root in the patch failed. |
| +1 :green_heart: | xmllint | 0m 0s | | No new issues. |
| -1 :x: | javadoc | 0m 33s |
[/patch-javadoc-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-javadoc-root.txt)
| root in the patch failed. |
| -1 :x: | shadedclient | 2m 9s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 6m 48s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/patch-unit-root.txt)
| root in the patch failed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 164m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4418/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4418 |
| Optional Tests | dupname asflicense codespell detsecrets shellcheck
shelldocs hadolint mvnsite unit compile javac javadoc mvninstall shadedclient
xmllint |
| uname | Linux 67905c1ae8ab 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d272048600695c0005e2fcd4dd22aa6449393c1a |
|
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17615157#comment-17615157
]
Steve Loughran commented on HADOOP-18197:
-
I'm doing a version of thirdparty jar where the protobuf lib is called
protobuf_3_21; here is a release with storediag hdfs://localhost/
{code}
class: com.google.protobuf.ExtensionRegistry
resource: com/google/protobuf/ExtensionRegistry.class
jar:file:/Users/stevel/Projects/Releases/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/lib/protobuf-java-2.5.0.jar!/com/google/protobuf/ExtensionRegistry.class
file:/Users/stevel/Projects/Releases/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/lib/protobuf-java-2.5.0.jar
class: org.apache.hadoop.shaded.com.google.protobuf.ExtensionRegistry
resource: org/apache/hadoop/shaded/com/google/protobuf/ExtensionRegistry.class
Not found on classpath:
org.apache.hadoop.shaded.com.google.protobuf.ExtensionRegistry
class: org.apache.hadoop.thirdparty.protobuf.ExtensionRegistry
resource: org/apache/hadoop/thirdparty/protobuf/ExtensionRegistry.class
jar:file:/Users/stevel/Projects/Releases/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0-SNAPSHOT.jar!/+.class
file:/Users/stevel/Projects/Releases/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0-SNAPSHOT.jar
{code}
this shows things have moved to org/apache/hadoop/thirdparty/protobuf in the
jar hadoop-shaded-protobuf_3_21-1.2.0-SNAPSHOT.jar
> Update protobuf 3.7.1 to a version without CVE-2021-22569
> -
>
> Key: HADOOP-18197
> URL: https://issues.apache.org/jira/browse/HADOOP-18197
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ivan Viaznikov
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available, security
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The artifact `org.apache.hadoop:hadoop-common` brings in a dependency
> `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version
> released in 2013 and it contains a vulnerability
> [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569].
> Therefore, requesting you to clarify if this library version is going to be
> updated in the following releases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577362#comment-17577362 ] Steve Loughran commented on HADOOP-18197: - that unshaded protobuf 2.5 has primarily been there to stop breaking other things. We could cut it and say "if you really need this here it is but really you should rebuild with your own version of Protobuf." Internally, we need a new version of that shaded library. I don't believe that putting a new version into our shaded lib with the same class names is the right thing to do. Instead I think we need a new shaded protobuf release with a different package name, and all our code rebuilt to link against that version. As for the shaded 3.7.1 package -we can cut it. If we have made any guarantees to maintain it (Have we?) Then we could release it as a self-contained Library which we don't include in our package, or we somehow get it into the at jar. Though that is implicitly committing to including not just it but all later Proto both versions which we release. Just upgrading our own package and saying "let's release this and rebuild Hadoop 3.3.9+ against it" would be the easiest. # I want to fork the next 3.3.x release off branch-3.3 by the end of the month. # I am not in a position to personally do the migration. If anyone else can put in the time it would be wonderful. > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 50m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577359#comment-17577359 ] Steve Loughran commented on HADOOP-18197: - bq. Just out of curiosity: what's the plan for protobuf 2.5.0 in older releases, e.g.: branch-2.10.2 or branch-3.2.3/4. Do we plan to update it to 2.6.1 or would that break things because it was not shaded? And what about newer branches and trunk should we just not ship the 2.5.0 jar? nothing. if you search through the mail archives of "the great protobuf upgrade", some time before hadoop 2 shipped, yoy will understand why. only with a private shaded protobuf lib or simultaneous rebuild of every application can you upgrade > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 50m > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[
https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577223#comment-17577223
]
Tamas Domok commented on HADOOP-18197:
--
Hi [~ste...@apache.org],
Based on [this|https://github.com/advisories/GHSA-wrvw-hg22-4m67] the affected
versions of CVE-2021-22569 are:
{quote}
com.google.protobuf:protobuf-java
Affected versions
< 3.16.1
>= 3.18.0, < 3.18.2
>= 3.19.0, < 3.19.2
Patched versions
3.16.1
3.18.2
3.19.2
{quote}
Which conforms the link in the description. So the protobuf-java-2.5.0.jar is
not affected by CVE-2021-22569, but it is vulnerable to CVE-2015-5237,
CVE-2019-15544.
I see that we ship the following protobuf related jars in the 3.3.4 release:
{code}
./hadoop-3.3.4/share/hadoop/yarn/csi/lib/protobuf-java-3.6.1.jar
./hadoop-3.3.4/share/hadoop/yarn/csi/lib/grpc-protobuf-lite-1.26.0.jar
./hadoop-3.3.4/share/hadoop/yarn/csi/lib/grpc-protobuf-1.26.0.jar
./hadoop-3.3.4/share/hadoop/common/lib/protobuf-java-2.5.0.jar
./hadoop-3.3.4/share/hadoop/common/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar
./hadoop-3.3.4/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar
./hadoop-3.3.4/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar
{code}
The csi was changed in: YARN-10747. Bump YARN CSI protobuf version to 3.7.1
(#2946)
Just out of curiosity: what's the plan for protobuf 2.5.0 in older releases,
e.g.: branch-2.10.2 or branch-3.2.3/4. Do we plan to update it to 2.6.1 or
would that break things because it was not shaded? And what about newer
branches and trunk should we just not ship the 2.5.0 jar?
Thanks.
> Update protobuf 3.7.1 to a version without CVE-2021-22569
> -
>
> Key: HADOOP-18197
> URL: https://issues.apache.org/jira/browse/HADOOP-18197
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ivan Viaznikov
>Priority: Major
> Labels: pull-request-available, security
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The artifact `org.apache.hadoop:hadoop-common` brings in a dependency
> `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version
> released in 2013 and it contains a vulnerability
> [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569].
> Therefore, requesting you to clarify if this library version is going to be
> updated in the following releases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17520557#comment-17520557 ] Steve Loughran commented on HADOOP-18197: - [~ivan.viaznikov] HADOOP-16557 upgraded our internal binaries to compile against 3.7.1, as we shade the classes we can update/upgrade without the risk of breaking every other app. we do still ship the old jar, which is something we can revisit. we will need to update our own protobuf version though > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: security > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
