[
https://issues.apache.org/jira/browse/BEAM-11227?focusedWorklogId=579718&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-579718
]
ASF GitHub Bot logged work on BEAM-11227:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Apr/21 03:32
Start Date: 09/Apr/21 03:32
Worklog Time Spent: 10m
Work Description: suztomo edited a comment on pull request #14474:
URL: https://github.com/apache/beam/pull/14474#issuecomment-816372956
https://ci-beam.apache.org/job/beam_PreCommit_Java_Phrase/3389/ got stuck at
22:53:00.
```
suztomo@apache-ci-beam-jenkins-11:~$ ps -ef |grep 'Gradle Test Executor'
jenkins 4993 11152 3 02:41 ? 00:00:48
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-Djava.security.manager=worker.org.gradle.process.internal.worker.child.Bootstra
pSecurityManager -Dorg.gradle.native=false
-javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.8.6.jar_a26f6511a7813217be4cd6439d66563b/jacocoagent.jar=destfile=build
/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false
-Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.va
riant -ea -cp /home/jenkins/.gradle/caches/6.8/workerMain/gradle-worker.jar
worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test
Executor 320'
jenkins 5751 5472 0 00:33 ? 00:01:05
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-DbeamTestPipelineOptions=["--project=apache-beam-testing","--tempLocation=gs://
temp-storage-for-perf-tests/loadtests","--tempRoot=gs://temp-storage-for-perf-tests/loadtests","--writeMethod=STREAMING_INSERTS","--writeFormat=JSON","--testBigQueryDatase
t=beam_performance","--testBigQueryTable=bqio_write_10GB_java_stream_0409002212","--metricsBigQueryDataset=beam_performance","--metricsBigQueryTable=bqio_10GB_results_java
_stream","--influxMeasurement=bqio_10GB_results_java_stream","--sourceOptions={\"numRecords\":\"10485760\",\"keySizeBytes\":\"1\",\"valueSizeBytes\":\"1024\"}","--runner=D
ataflowRunner","--maxNumWorkers=5","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--workerHarn
essContainerImage=","--dataflowWorkerJar=/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Streaming_Performance_Test_Java/src/runners/google-cloud-dataflow-java/worke
r/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.30.0-SNAPSHOT.jar","--region=us-central1"]
-Djava.security.manager=worker.org.gradle.pro
cess.internal.worker.child.BootstrapSecurityManager
-Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US
-Duser.language=en -Duser.variant -ea -cp /hom
e/jenkins/.gradle/caches/6.8/workerMain/gradle-worker.jar
worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test
Executor 1'
suztomo 20574 20550 0 03:07 pts/0 00:00:00 grep --color=auto Gradle
Test Executor
jenkins 21632 11377 3 00:03 ? 00:07:03
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-
testing","--region=us-central1","--tempRoot=gs://temp-storage-for-validates-runner-tests","--dataflowWorkerJar=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_V
alidatesRunner_Dataflow/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.30.0-SNAPSHOT.jar","
--workerHarnessContainerImage="]
-Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
-Dorg.gradle.native=false -Xmx2g -Dfile.e
ncoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp
/home/jenkins/.gradle/caches/6.8/workerMain/gradle-worker.jar
worker.org.gradle.process.internal.
worker.GradleWorkerMain 'Gradle Test Executor 6'
jenkins 21652 11377 3 00:03 ? 00:07:09
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-DbeamTestPipelineOptions=["--runner=TestDataflowRunner","--project=apache-beam-
testing","--region=us-central1","--tempRoot=gs://temp-storage-for-validates-runner-tests","--dataflowWorkerJar=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Java_V
alidatesRunner_Dataflow/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.30.0-SNAPSHOT.jar","
--workerHarnessContainerImage="]
-Djava.security.manager=worker.org.gradle.process.internal.worker.child.BootstrapSecurityManager
-Dorg.gradle.native=false -Xmx2g -Dfile.e
ncoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp
/home/jenkins/.gradle/caches/6.8/workerMain/gradle-worker.jar
worker.org.gradle.process.internal.
worker.GradleWorkerMain 'Gradle Test Executor 2'
jenkins 25731 11152 3 02:49 ? 00:00:42
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-Djava.security.manager=worker.org.gradle.process.internal.worker.child.Bootstra
pSecurityManager -Dorg.gradle.native=false
-javaagent:build/tmp/expandedArchives/org.jacoco.agent-0.8.6.jar_a26f6511a7813217be4cd6439d66563b/jacocoagent.jar=destfile=build
/jacoco/test.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false
-Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.va
riant -ea -cp /home/jenkins/.gradle/caches/6.8/workerMain/gradle-worker.jar
worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Test
Executor 361'
```
With `sudo jstack 4993` (for "Gradle Test Executor 320"), I got
https://gist.github.com/suztomo/6b7678f1c36cc67bf4e84df55c4a9611 in Jenkins's
console.
### Test that got stuck
```
23:10:59 at
java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:244)
23:10:59 at
org.apache.beam.runners.dataflow.worker.fn.logging.BeamFnLoggingServiceTest.testMultipleClientsFailingIsHandledGracefullyByServer(BeamFnLoggingServiceTest.java:150)
```
### gRPC classes in the stack traces
Interestingly the output includes gRPC-related classes:
```
23:10:59 "grpc-nio-worker-ELG-3-8" #106 daemon prio=5 os_prio=0
tid=0x00007f7bc403d800 nid=0x2028 runnable [0x00007f7c94cc5000]
23:10:59 java.lang.Thread.State: RUNNABLE
23:10:59 at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
23:10:59 at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
23:10:59 at
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
23:10:59 at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
23:10:59 - locked <0x00000000f967b450> (a
org.apache.beam.vendor.grpc.v1p36p0.io.netty.channel.nio.SelectedSelectionKeySet)
23:10:59 - locked <0x00000000f967c508> (a
java.util.Collections$UnmodifiableSet)
23:10:59 - locked <0x00000000f967b358> (a sun.nio.ch.EPollSelectorImpl)
23:10:59 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
23:10:59 at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:805)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
23:10:59 at
org.apache.beam.vendor.grpc.v1p36p0.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 579718)
Time Spent: 112h 20m (was: 112h 10m)
> Upgrade beam-vendor-grpc-1_26_0-0.3 to fix CVE-2020-27216
> ---------------------------------------------------------
>
> Key: BEAM-11227
> URL: https://issues.apache.org/jira/browse/BEAM-11227
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Affects Versions: 2.21.0, 2.22.0, 2.23.0, 2.24.0, 2.25.0
> Reporter: Boury Mbodj
> Assignee: Kenneth Knowles
> Priority: P1
> Labels: apache-beam, beam
> Fix For: 2.29.0
>
> Time Spent: 112h 20m
> Remaining Estimate: 0h
>
> *+Description+**:* [Apache Beam :: Vendored Dependencies :: GRPC ::
> 1.26.0|https://mvnrepository.com/artifact/org.apache.beam/beam-vendor-grpc-1_26_0]
> »
> [0.3|https://mvnrepository.com/artifact/org.apache.beam/beam-vendor-grpc-1_26_0/0.3]
> uses the dependency Eclipse Jetty (9.2.10.v20150310), which is prone to a
> privilege escalation vulnerability. This issue (CVE-2020-27216) was published
> on 23/10/2020.
> *+Affected Versions:+*
> Eclipse Jetty versions 9.4.32.v20200930 and prior, 10.0.0.beta2 and prior
> and 11.0.0.beta2 and prior.
> *+Recommendation/+* *+Update Suggestion:+*
> Update the Eclipse Jetty dependency to version 9.4.33.v20201020,
> 10.0.0.beta3, 11.0.0.beta3 or later.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
