[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-08-10 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17396691#comment-17396691 ] Till Rohrmann commented on FLINK-16069: --- These are great results. Thanks a lot for improving the

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-08-08 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17395728#comment-17395728 ] Zhu Zhu commented on FLINK-16069: - Thanks for making the improvements and sharing the results!

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-08-05 Thread Zhilong Hong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393869#comment-17393869 ] Zhilong Hong commented on FLINK-16069: -- After the optimization we've done in FLINK-23005, the

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-31 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354757#comment-17354757 ] Zhu Zhu commented on FLINK-16069: - Even if the main thread can have the highest priority, GC problem can

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-31 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354477#comment-17354477 ] Chesnay Schepler commented on FLINK-16069: -- Have we ever experimented with thread priorities

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-31 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354434#comment-17354434 ] Till Rohrmann commented on FLINK-16069: --- Maybe one could start by taking {{#cpus/2}} as a default

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-31 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354420#comment-17354420 ] Zhu Zhu commented on FLINK-16069: - Yes a dedicated {{serializationExecutor}} is an alternative. One

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-31 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354373#comment-17354373 ] Till Rohrmann commented on FLINK-16069: --- This is a good finding [~zhuzh]. I like the ideas.

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-05-27 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352943#comment-17352943 ] Zhu Zhu commented on FLINK-16069: - Thanks for the suggestion and sorry for the late reply! [~trohrmann]

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-03-31 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312220#comment-17312220 ] Till Rohrmann commented on FLINK-16069: --- Maybe we should remove {{AkkaRpcService.getExecutor}} and

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-03-31 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312218#comment-17312218 ] Till Rohrmann commented on FLINK-16069: --- Could it be that we use somehow the {{ActorSystems}}

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-03-30 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17311960#comment-17311960 ] Zhu Zhu commented on FLINK-16069: - >From what I can see, heartbeat timeout happens because the scheduled

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-03-30 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17311563#comment-17311563 ] Till Rohrmann commented on FLINK-16069: --- Thanks for providing the feedback [~zhuzh]. The result

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-02-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291607#comment-17291607 ] Zhu Zhu commented on FLINK-16069: - I had a PoC to verify the idea to cache and reuse ShuffleDescriptor

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-02-26 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291544#comment-17291544 ] Till Rohrmann commented on FLINK-16069: --- This ticket has been touched by the backlog grooming

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-08-09 Thread Yumeng Zhang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174053#comment-17174053 ] Yumeng Zhang commented on FLINK-16069: -- [~zhuzh] Yes, we encountered the same problem when

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-08-09 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173885#comment-17173885 ] Zhu Zhu commented on FLINK-16069: - Hi Yumeng, I think we have not reached a consensus. I once did a PoC

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-08-07 Thread Yumeng Zhang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173036#comment-17173036 ] Yumeng Zhang commented on FLINK-16069: -- Hello guys, do you reach a consensus about how to optimize

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045531#comment-17045531 ] Zhu Zhu commented on FLINK-16069: - >> Is there anything in the ShuffleDescriptor that would be different

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-26 Thread Stephan Ewen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045395#comment-17045395 ] Stephan Ewen commented on FLINK-16069: -- "each InputGateDeploymentDescriptor of B instances should

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-25 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045135#comment-17045135 ] Zhu Zhu commented on FLINK-16069: - Making building deployment descriptor more static seems not to be an

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-25 Thread Yingjie Cao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045107#comment-17045107 ] Yingjie Cao commented on FLINK-16069: - Maybe we can split the tasks to be deployed into multi mini

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-21 Thread Stephan Ewen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041812#comment-17041812 ] Stephan Ewen commented on FLINK-16069: -- It it possible to make more of the deployment descriptor

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-19 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039916#comment-17039916 ] Till Rohrmann commented on FLINK-16069: --- Whenever we access state which is not immutable belonging

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-18 Thread huweihua (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039191#comment-17039191 ] huweihua commented on FLINK-16069: -- Hi, [~trohrmann], thanks for your time. Yes, The iteration over the

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2020-02-18 Thread Till Rohrmann (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038976#comment-17038976 ] Till Rohrmann commented on FLINK-16069: --- Hi [~huwh], do you know what exactly is taking so long.