[
https://issues.apache.org/jira/browse/BEAM-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17293819#comment-17293819
]
Tomo Suzuki edited comment on BEAM-11227 at 3/5/21, 8:07 PM:
-------------------------------------------------------------
Looking at Boury Mbodj's
[activity|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=bmbodj&selectedTab=com.atlassian.streams.streams-jira-plugin:user-profile-stream-panel],
it seems that this is one-off ticket (not by automation).
[~kenn] Sure. Let me continue https://github.com/apache/beam/pull/14028 to see
what would break.
h1. Memo
Previous emails
* [[VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for BEAM-9288 RC
#3|https://lists.apache.org/thread.html/rea4a27c47529a27936ab2c51162c8e532b8b625c4d70c4f7f485c7cd%40%3Cdev.beam.apache.org%3E]
(vote passed)
* [[VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for BEAM-9288 RC
#2|https://lists.apache.org/thread.html/rc6372c9873a2b00cf5dc30efeeb0b13bb1aa92a0f93e2417211effc4%40%3Cdev.beam.apache.org%3E]
(commit id mistake)
* [Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for
BEAM-9288|https://lists.apache.org/thread.html/r31fb38e8480889ecb23db7135771d419c9cf43fd20be96c4aa179e54%40%3Cdev.beam.apache.org%3E]
(-> conscrypt was in JAR)
{code}
suztomo@suztomo:~/beam$ jar tvf
vendor/grpc-1_36_0/build/libs/beam-vendor-grpc-1_36_0-0.1.jar | grep conscrypt
suztomo@suztomo:~/beam$
{code}
Previous version (beam-vendor-grpc-1_26_0-0.3):
{code}
suztomo-macbookpro44% jar tvf ~/Downloads/beam-vendor-grpc-1_26_0-0.3.jar|grep
'\.so$'
2626449 Sat Sep 21 10:06:14 EDT 2019
META-INF/native/liborg_apache_beam_vendor_grpc_v1p26p0_netty_tcnative_linux_x86_64.so
59545 Wed Nov 26 20:02:18 EST 2014 linux/amd64/liblz4-java.so
68840 Wed Nov 26 20:02:18 EST 2014 linux/i386/liblz4-java.so
161360 Wed Nov 26 20:02:18 EST 2014 win32/amd64/liblz4-java.so
{code}
Current proposal
{code}
suztomo@suztomo:~/beam$ jar tvf
vendor/grpc-1_36_0/build/libs/beam-vendor-grpc-1_36_0-0.1.jar | grep '\.so$'
2628280 Fri Aug 21 11:19:38 UTC 2020
META-INF/native/liborg_apache_beam_vendor_grpc_v1p36p0_netty_tcnative_linux_x86_64.so
1933284 Fri Aug 21 11:19:38 UTC 2020
META-INF/native/liborg_apache_beam_vendor_grpc_v1p36p0_netty_tcnative_linux_aarch64.so
59545 Wed Nov 26 20:02:18 UTC 2014 linux/amd64/liblz4-java.so
68840 Wed Nov 26 20:02:18 UTC 2014 linux/i386/liblz4-java.so
161360 Wed Nov 26 20:02:18 UTC 2014 win32/amd64/liblz4-java.so
{code}
h1. Consideration
h2. protobuf-java version
gRPC 1.36 uses protobuf-java 3.12.0 and
com.google.api.grpc:proto-google-common-protos:2.0.1 draws protobuf-java
3.13.0. Gradle's dependency mediation chooses 3.13.0 (higher) which has the
Java8-incompatibility problem
(https://github.com/protocolbuffers/protobuf/issues/7827). We need to set a
higher version to avoid having the problem in the vendored gRPC.
h1. Linkage Errors
I ran checkJavaLinkage task (
[output|https://gist.github.com/suztomo/c1d9c587ea58ad4cfd25c853ad921c93#file-beam-vendor-grpc-1_36_0_with_protobuf_3-15-txt-L6].
Apart from the errors on loggers, it shows the following linkage errors on
{{io.netty.handler.ssl.ConscryptAlpnSslEngine}}:
{code}
Class org.junit.runners.model.Statement is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.runners.model.MultipleFailureException is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.rules.ExternalResource is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcServerRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.rules.TestRule is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.AllocatedBuffer is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.Conscrypt is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.BufferAllocator is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.HandshakeListener is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
{code}
The {{grpc.testing}} package seems fine. How about the
{{org.conscrypt.BufferAllocator}}? Are they safe?
* Why aren't they part of the vendored JAR in version 1.36?
How it it declared?
* Were the class files part of the vendored gRPC 1.26?
* Does Beam use it?
was (Author: suztomo):
Looking at Boury Mbodj's
[activity|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=bmbodj&selectedTab=com.atlassian.streams.streams-jira-plugin:user-profile-stream-panel],
it seems that this is one-off ticket (not by automation).
[~kenn] Sure. Let me continue https://github.com/apache/beam/pull/14028 to see
what would break.
h1. Memo
Previous emails
* [[VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for BEAM-9288 RC
#3|https://lists.apache.org/thread.html/rea4a27c47529a27936ab2c51162c8e532b8b625c4d70c4f7f485c7cd%40%3Cdev.beam.apache.org%3E]
(vote passed)
* [[VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for BEAM-9288 RC
#2|https://lists.apache.org/thread.html/rc6372c9873a2b00cf5dc30efeeb0b13bb1aa92a0f93e2417211effc4%40%3Cdev.beam.apache.org%3E]
(commit id mistake)
* [Re: [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.3 for
BEAM-9288|https://lists.apache.org/thread.html/r31fb38e8480889ecb23db7135771d419c9cf43fd20be96c4aa179e54%40%3Cdev.beam.apache.org%3E]
(-> conscrypt was in JAR)
{code}
suztomo@suztomo:~/beam$ jar tvf
vendor/grpc-1_36_0/build/libs/beam-vendor-grpc-1_36_0-0.1.jar | grep conscrypt
suztomo@suztomo:~/beam$
{code}
Previous version (beam-vendor-grpc-1_26_0-0.3):
{code}
suztomo-macbookpro44% jar tvf ~/Downloads/beam-vendor-grpc-1_26_0-0.3.jar|grep
'\.so$'
2626449 Sat Sep 21 10:06:14 EDT 2019
META-INF/native/liborg_apache_beam_vendor_grpc_v1p26p0_netty_tcnative_linux_x86_64.so
59545 Wed Nov 26 20:02:18 EST 2014 linux/amd64/liblz4-java.so
68840 Wed Nov 26 20:02:18 EST 2014 linux/i386/liblz4-java.so
161360 Wed Nov 26 20:02:18 EST 2014 win32/amd64/liblz4-java.so
{code}
Current proposal
{code}
suztomo@suztomo:~/beam$ jar tvf
vendor/grpc-1_36_0/build/libs/beam-vendor-grpc-1_36_0-0.1.jar | grep '\.so$'
2628280 Fri Aug 21 11:19:38 UTC 2020
META-INF/native/liborg_apache_beam_vendor_grpc_v1p36p0_netty_tcnative_linux_x86_64.so
1933284 Fri Aug 21 11:19:38 UTC 2020
META-INF/native/liborg_apache_beam_vendor_grpc_v1p36p0_netty_tcnative_linux_aarch64.so
59545 Wed Nov 26 20:02:18 UTC 2014 linux/amd64/liblz4-java.so
68840 Wed Nov 26 20:02:18 UTC 2014 linux/i386/liblz4-java.so
161360 Wed Nov 26 20:02:18 UTC 2014 win32/amd64/liblz4-java.so
{code}
h1. Consideration
h2. protobuf-java version
gRPC 1.36 uses protobuf-java 3.12.0 and
com.google.api.grpc:proto-google-common-protos:2.0.1 draws protobuf-java
3.13.0. Gradle's dependency mediation chooses 3.13.0 (higher) which has the
Java8-incompatibility problem
(https://github.com/protocolbuffers/protobuf/issues/7827). We need to set a
higher version to avoid having the problem in the vendored gRPC.
h1. Linkage Errors
I ran checkJavaLinkage task (
[output|https://gist.github.com/suztomo/c1d9c587ea58ad4cfd25c853ad921c93#file-beam-vendor-grpc-1_36_0_with_protobuf_3-15-txt-L6].
Apart from the errors on loggers, it shows the following linkage errors on
{{io.netty.handler.ssl.ConscryptAlpnSslEngine}}:
{code}
Class org.junit.runners.model.Statement is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.runners.model.MultipleFailureException is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.rules.ExternalResource is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcServerRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.junit.rules.TestRule is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.grpc.testing.GrpcCleanupRule
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.AllocatedBuffer is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.Conscrypt is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.BufferAllocator is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
Class org.conscrypt.HandshakeListener is not found;
referenced by 1 class file
org.apache.beam.vendor.grpc.v1p36p0.io.netty.handler.ssl.ConscryptAlpnSslEngine
(org.apache.beam:beam-vendor-grpc-1_36_0:0.1)
Cause:
Unknown
{code}
> Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216
> ---------------------------------------------------------
>
> Key: BEAM-11227
> URL: https://issues.apache.org/jira/browse/BEAM-11227
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Affects Versions: 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0
> Reporter: Boury Mbodj
> Priority: P1
> Labels: apache-beam, beam
> Fix For: 2.29.0
>
> Time Spent: 14h 50m
> Remaining Estimate: 0h
>
> *+Description+**:* [Apache Beam :: Vendored Dependencies :: GRPC ::
> 1.26.0|https://mvnrepository.com/artifact/org.apache.beam/beam-vendor-grpc-1_26_0]
> »
> [0.3|https://mvnrepository.com/artifact/org.apache.beam/beam-vendor-grpc-1_26_0/0.3]
> uses the dependency Eclipse Jetty (9.2.10.v20150310), which is prone to a
> privilege escalation vulnerability. This issue (CVE-2020-27216) was published
> on 23/10/2020.
> *+Affected Versions:+*
> Eclipse Jetty versions 9.4.32.v20200930 and prior, 10.0.0.beta2 and prior
> and 11.0.0.beta2 and prior.
> *+Recommendation/+* *+Update Suggestion:+*
> Update the Eclipse Jetty dependency to version 9.4.33.v20201020,
> 10.0.0.beta3, 11.0.0.beta3 or later.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
