[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364439#comment-16364439 ] Till Rohrmann commented on FLINK-8044: -- Thanks for letting me know. We currently also follow the simple heuristic that whenever you have a previous location, try to be scheduled to the same place again. I think this should already help a lot with recovering. > Introduce scheduling mechanism to satisfy both state locality and input > --- > > Key: FLINK-8044 > URL: https://issues.apache.org/jira/browse/FLINK-8044 > Project: Flink > Issue Type: New Feature > Components: Scheduler >Affects Versions: 1.4.0 >Reporter: Sihua Zhou >Priority: Major > > For local recovery we need a scheduler to satisfy both state locality and > input, but current scheduler allocate resources according to the order of > topologically, this can cause bad result if we allocate base on both state > and input, so some revision for scheduler is necessary now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364215#comment-16364215 ] Sihua Zhou commented on FLINK-8044: --- Sorry [~till.rohrmann], finally I can't come up with a perfect algorithm and finally we use a naively mechanism which only schedule according to the state size for local recovery. So I think I can't give your a good document... > Introduce scheduling mechanism to satisfy both state locality and input > --- > > Key: FLINK-8044 > URL: https://issues.apache.org/jira/browse/FLINK-8044 > Project: Flink > Issue Type: New Feature > Components: Scheduler >Affects Versions: 1.4.0 >Reporter: Sihua Zhou >Priority: Major > > For local recovery we need a scheduler to satisfy both state locality and > input, but current scheduler allocate resources according to the order of > topologically, this can cause bad result if we allocate base on both state > and input, so some revision for scheduler is necessary now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input
[ https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247263#comment-16247263 ] Sihua Zhou commented on FLINK-8044: --- Hi [~till.rohrmann] I have addressed [FLINK-7866|https://issues.apache.org/jira/browse/FLINK-7866], How about to start revision on scheduler from here? Will attach a doc to describe my thought later. > Introduce scheduling mechanism to satisfy both state locality and input > --- > > Key: FLINK-8044 > URL: https://issues.apache.org/jira/browse/FLINK-8044 > Project: Flink > Issue Type: New Feature > Components: Scheduler >Affects Versions: 1.4.0 >Reporter: Sihua Zhou > > For local recovery we need a scheduler to satisfy both state locality and > input, but current scheduler allocate resources according to the order of > topologically, this can cause bad result if we allocate base on both state > and input, so some revision for scheduler is necessary now. -- This message was sent by Atlassian JIRA (v6.4.14#64029)