[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2018-02-14 Thread Till Rohrmann (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364439#comment-16364439
 ] 

Till Rohrmann commented on FLINK-8044:
--

Thanks for letting me know. We currently also follow the simple heuristic that 
whenever you have a previous location, try to be scheduled to the same place 
again. I think this should already help a lot with recovering.

> Introduce scheduling mechanism to satisfy both state locality and input
> ---
>
> Key: FLINK-8044
> URL: https://issues.apache.org/jira/browse/FLINK-8044
> Project: Flink
>  Issue Type: New Feature
>  Components: Scheduler
>Affects Versions: 1.4.0
>Reporter: Sihua Zhou
>Priority: Major
>
> For local recovery we need a scheduler to satisfy both state locality and 
> input, but current scheduler allocate resources according to the order of 
> topologically, this can cause bad result if we allocate base on both state 
> and input, so some revision for scheduler is necessary now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2018-02-14 Thread Sihua Zhou (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364215#comment-16364215
 ] 

Sihua Zhou commented on FLINK-8044:
---

Sorry [~till.rohrmann], finally I can't come up with a perfect algorithm  
and finally we use a naively mechanism which only schedule according to the 
state size for local recovery. So I think I can't give your a good document...

> Introduce scheduling mechanism to satisfy both state locality and input
> ---
>
> Key: FLINK-8044
> URL: https://issues.apache.org/jira/browse/FLINK-8044
> Project: Flink
>  Issue Type: New Feature
>  Components: Scheduler
>Affects Versions: 1.4.0
>Reporter: Sihua Zhou
>Priority: Major
>
> For local recovery we need a scheduler to satisfy both state locality and 
> input, but current scheduler allocate resources according to the order of 
> topologically, this can cause bad result if we allocate base on both state 
> and input, so some revision for scheduler is necessary now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (FLINK-8044) Introduce scheduling mechanism to satisfy both state locality and input

2017-11-10 Thread Sihua Zhou (JIRA)

[ 
https://issues.apache.org/jira/browse/FLINK-8044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247263#comment-16247263
 ] 

Sihua Zhou commented on FLINK-8044:
---

Hi [~till.rohrmann] I have addressed 
[FLINK-7866|https://issues.apache.org/jira/browse/FLINK-7866], How about to 
start revision on scheduler from here? Will attach a doc to describe my thought 
later.

> Introduce scheduling mechanism to satisfy both state locality and input
> ---
>
> Key: FLINK-8044
> URL: https://issues.apache.org/jira/browse/FLINK-8044
> Project: Flink
>  Issue Type: New Feature
>  Components: Scheduler
>Affects Versions: 1.4.0
>Reporter: Sihua Zhou
>
> For local recovery we need a scheduler to satisfy both state locality and 
> input, but current scheduler allocate resources according to the order of 
> topologically, this can cause bad result if we allocate base on both state 
> and input, so some revision for scheduler is necessary now.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)