[
https://issues.apache.org/jira/browse/BEAM-7688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Weaver updated BEAM-7688:
------------------------------
Description:
When parallelism > 1:
"MapPartition (MapPartition at [37]\{Analyze, RandomizeData, ReadFromText,
DecodeForAnalyze}) (9/12)" #2855 prio=5 os_prio=0 tid=0x00007f9184022800
nid=0x2b58 waiting on condition [0x00007f9091592000]
java.lang.Thread.State: WAITING (parking)
at (C/C++) 0x00007f926a97a9f2 (Unknown Source)
at (C/C++) 0x00007f9269f1dd99 (Unknown Source)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ca218030> (a
java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
at
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.beam.sdk.fn.data.CompletableFutureInboundDataClient.awaitCompletion(CompletableFutureInboundDataClient.java:48)
at
org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:90)
at
org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:298)
at
org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:209)
at
org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:209)
at
org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
at java.lang.Thread.run(Thread.java:748)
was:
When parallelism > 1:
{{"MapPartition (MapPartition at [37]\{Analyze, RandomizeData, ReadFromText,
DecodeForAnalyze}) (9/12)" #2855 prio=5 os_prio=0 tid=0x00007f9184022800
nid=0x2b58 waiting on condition [0x00007f9091592000]}}
{{ java.lang.Thread.State: WAITING (parking)}}
{{ at (C/C++) 0x00007f926a97a9f2 (Unknown Source)}}
{{ at (C/C++) 0x00007f9269f1dd99 (Unknown Source)}}
{{ at sun.misc.Unsafe.park(Native Method)}}
{{ - parking to wait for <0x00000000ca218030> (a
java.util.concurrent.CompletableFuture$Signaller)}}
{{ at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)}}
{{ at
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)}}
{{ at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)}}
{{ at
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)}}
{{ at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)}}
{{ at
org.apache.beam.sdk.fn.data.CompletableFutureInboundDataClient.awaitCompletion(CompletableFutureInboundDataClient.java:48)}}
{{ at
org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:90)}}
{{ at
org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:298)}}
{{ at
org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:209)}}
{{ at
org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:209)}}
{{ at
org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)}}
{{ at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)}}
{{ at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)}}
{{ at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)}}
{{ at java.lang.Thread.run(Thread.java:748)}}
> Flink portable runner gets stuck when waiting for SDK Harness to close
> ----------------------------------------------------------------------
>
> Key: BEAM-7688
> URL: https://issues.apache.org/jira/browse/BEAM-7688
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: Major
>
> When parallelism > 1:
> "MapPartition (MapPartition at [37]\{Analyze, RandomizeData, ReadFromText,
> DecodeForAnalyze}) (9/12)" #2855 prio=5 os_prio=0 tid=0x00007f9184022800
> nid=0x2b58 waiting on condition [0x00007f9091592000]
> java.lang.Thread.State: WAITING (parking)
> at (C/C++) 0x00007f926a97a9f2 (Unknown Source)
> at (C/C++) 0x00007f9269f1dd99 (Unknown Source)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000000ca218030> (a
> java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.beam.sdk.fn.data.CompletableFutureInboundDataClient.awaitCompletion(CompletableFutureInboundDataClient.java:48)
> at
> org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:90)
> at
> org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:298)
> at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:209)
> at
> org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:209)
> at
> org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
> at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
