[
https://issues.apache.org/jira/browse/BEAM-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990113#comment-16990113
]
Heejong Lee commented on BEAM-8901:
-----------------------------------
Interesting. Does Flink also create a new local environment for every test? I
could see Flink local environment did some cleanup job when I reuse the
existing environment since there are logs that the mini cluster automatically
shuts down and reboots after each pipeline run. I'm still not sure what happens
when multiple local environments are created (do they share some backend?) I'll
try to test with -XX:MaxMetaspaceSize.
> add experimental flag for reusing flink local environment
> ---------------------------------------------------------
>
> Key: BEAM-8901
> URL: https://issues.apache.org/jira/browse/BEAM-8901
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Heejong Lee
> Assignee: Heejong Lee
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Flink job server launches a new mini cluster every time we run the pipeline
> on Flink local environment. To prevent OOM, we need to reuse existing Flink
> local environment if possible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
