[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann updated FLINK-15626: -- Fix Version/s: (was: 1.12.0) 1.13.0 > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.13.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG > and RestartIndividualStrategy. These implementations and their tests heavily > relies on the legacy scheduling logics in ExecutionGraph so removing them can > make #3 much easier. > 3. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed. So before removing the legacy logics, it's > better to cleanup all the tests relying on the legacy scheduling first, i.e. > remove tests that are no needed and rewrite the tests which are still needed. > 4. remove legacy FailoverStrategy and RestartStrategy > 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Fix Version/s: (was: 1.11.0) 1.12.0 > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Assignee: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.12.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG > and RestartIndividualStrategy. These implementations and their tests heavily > relies on the legacy scheduling logics in ExecutionGraph so removing them can > make #3 much easier. > 3. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed. So before removing the legacy logics, it's > better to cleanup all the tests relying on the legacy scheduling first, i.e. > remove tests that are no needed and rewrite the tests which are still needed. > 4. remove legacy FailoverStrategy and RestartStrategy > 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Description: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG and RestartIndividualStrategy. These implementations and their tests heavily relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier. 3. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed. So before removing the legacy logics, it's better to cleanup all the tests relying on the legacy scheduling first, i.e. remove tests that are no needed and rewrite the tests which are still needed. 4. remove legacy FailoverStrategy and RestartStrategy 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. was: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG and RestartIndividualStrategy. These implementations and their tests heavily relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier. 3. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 4. remove legacy FailoverStrategy, RestartStrategy and utils 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG > and RestartIndividualStrategy. These implementations and their tests heavily > relies on the legacy scheduling logics in ExecutionGraph so removing them can > make #3 much easier. > 3. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed. So before removing the legacy logics, it's > better to cleanup all the tests relying on the legacy scheduling first, i.e. > remove tests that are no needed and rewrite the tests which are still needed. > 4. remove legacy FailoverStrategy and RestartStrategy > 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Description: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG and RestartIndividualStrategy. These implementations and their tests heavily relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier. 3. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 4. remove legacy FailoverStrategy, RestartStrategy and utils 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. was: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy failover strategy implementations except for NoOpFailoverStrategy. These implementations relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier. 3. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 4. remove legacy FailoverStrategy, RestartStrategy and utils 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG > and RestartIndividualStrategy. These implementations and their tests heavily > relies on the legacy scheduling logics in ExecutionGraph so removing them can > make #3 much easier. > 3. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed > 4. remove legacy FailoverStrategy, RestartStrategy and utils > 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Description: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy failover strategy implementations except for NoOpFailoverStrategy. These implementations relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier. 3. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 4. remove legacy FailoverStrategy, RestartStrategy and utils 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. was: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 3. remove legacy FailoverStrategy, RestartStrategy and utils 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy failover strategy implementations except for > NoOpFailoverStrategy. These implementations relies on the legacy scheduling > logics in ExecutionGraph so removing them can make #3 much easier. > 3. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed > 4. remove legacy FailoverStrategy, RestartStrategy and utils > 5. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Description: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 3. remove legacy FailoverStrategy, RestartStrategy and utils 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. was: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 3. remove legacy FailoverStrategy, RestartStrategy and utils 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed > 3. remove legacy FailoverStrategy, RestartStrategy and utils > 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Description: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. Here's a rough plan: 1. remove LegacyScheduler class * related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler 2. remove legacy scheduling/failover logics in ExecutionGraph * note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed 3. remove legacy FailoverStrategy, RestartStrategy and utils 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one. was: This umbrella ticket is to track the tickets to remove the legacy scheduler and related components. So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling. > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. > Here's a rough plan: > 1. remove LegacyScheduler class > * related tests should also be removed, including > AlsoRunWithLegacyScheduler which is used to enable integration tests for > legacy scheduler > 2. remove legacy scheduling/failover logics in ExecutionGraph > * note that many tests still relies on these logics(e.g. > ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite > tests which are still needed > 3. remove legacy FailoverStrategy, RestartStrategy and utils > 4. Factoring out the scheduling logics in ExecutionGraph to make it a plain > data structure. This can be a long term goal though. There can be several > pieces and we may need to examine them one by one. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Issue Type: Task (was: New Feature) > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: Task > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Priority: Critical (was: Blocker) > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: New Feature > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Critical > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-15626) Remove legacy scheduler
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-15626: Priority: Blocker (was: Major) > Remove legacy scheduler > --- > > Key: FLINK-15626 > URL: https://issues.apache.org/jira/browse/FLINK-15626 > Project: Flink > Issue Type: New Feature > Components: Runtime / Coordination >Affects Versions: 1.11.0 >Reporter: Zhu Zhu >Priority: Blocker > Labels: Umbrella > Fix For: 1.11.0 > > > This umbrella ticket is to track the tickets to remove the legacy scheduler > and related components. > So that we can have a much cleaner scheduler framework which significantly > simplifies our next development work on job scheduling. -- This message was sent by Atlassian Jira (v8.3.4#803005)
