Jiayi Zhao created BEAM-6956:
--------------------------------
Summary: --experiments=worker_threads=100 issue
Key: BEAM-6956
URL: https://issues.apache.org/jira/browse/BEAM-6956
Project: Beam
Issue Type: New Feature
Components: runner-flink
Reporter: Jiayi Zhao
I noticed that without this "–experiments=worker_threads=100", pipeline will
stuck,
The weird thing is, I tried some complex pipeline using the in thread flink
method (./gradlew :beam-runners-flink_2.11-job-server:runShadow)
"–experiments=worker_threads=100" doesn't work, but
"–experiments=worker_threads=1000" works fine
Then I tried the same pipeline using the separate local flink cluster
(./gradlew :beam-runners-flink_2.11-job-server:runShadow
-PflinkMasterUrl=localhost:8081), flink version is 1.5.6 (other version doesn't
work, see BEAM-6915)
Neither did "–experiments=worker_threads=1000" or
"–experiments=worker_threads=10000" work, pipeline stuck at certain stage
(shows running in flink UI but won't finish forever)
any real fix to that? Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
