[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636404#comment-15636404 ] ASF GitHub Bot commented on FLINK-4596: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/2592 > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Wish > Components: Distributed Coordination >Affects Versions: 1.2.0 >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636298#comment-15636298 ] ASF GitHub Bot commented on FLINK-4596: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/2592 Thanks for your contribution @nguraja. Will merge this PR. > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Wish > Components: Distributed Coordination >Affects Versions: 1.2.0 >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15636294#comment-15636294 ] ASF GitHub Bot commented on FLINK-4596: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/2592#discussion_r86542348 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -1114,7 +1114,8 @@ class JobManager( Option(jobGraph.getSerializedExecutionConfig() --- End diff -- True, you're right because it would prevent the `StreamingJobGraphGenerator` from setting the `FixedDelayRestartStrategy` in case that no other restarting strategy has been set. > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Wish > Components: Distributed Coordination >Affects Versions: 1.2.0 >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[
https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548006#comment-15548006
]
ASF GitHub Bot commented on FLINK-4596:
---
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/2592#discussion_r81918259
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/restartstrategy/RestartStrategies.java
---
@@ -173,4 +177,13 @@ public String getDescription() {
+ " and fixed delay " +
delayBetweenAttemptsInterval.toString();
}
}
+
+ final public static class FallbackRestartStrategyConfiguration extends
RestartStrategyConfiguration{
--- End diff --
Javadocs missing
> RESTART_STRATEGY is not really pluggable
>
>
> Key: FLINK-4596
> URL: https://issues.apache.org/jira/browse/FLINK-4596
> Project: Flink
> Issue Type: Wish
> Components: Distributed Coordination
>Affects Versions: 1.2.0
>Reporter: Nagarjun Guraja
>
> Standalone cluster config accepts an implementation(class) as part of the
> yaml config file but that does not work either as cluster level restart
> strategy or streaming job level restart strategy
> CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value
> of strategyname to lowercase and searches for class name using lowercased
> string.
> JOB LEVEL CAUSE: Checkpointed streams have specific code to add
> fixeddelayrestartconfiguration if no RestartConfiguration is specified in
> the job env. Also, jobs cannot provide their own custom restart strategy
> implementation and are constrained to pick up one of the three restart
> strategies provided by flink.
> FIX: Do not lower case the strategy config value, support a new
> restartconfiguration to fallback to cluster level restart strategy and
> support jobs to provide custom implementation of the strategy class itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548007#comment-15548007 ] ASF GitHub Bot commented on FLINK-4596: --- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/2592#discussion_r81918886 --- Diff: flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala --- @@ -1114,7 +1114,8 @@ class JobManager( Option(jobGraph.getSerializedExecutionConfig() --- End diff -- I think the `FallbackRestartStrategy` could be set by default in the `ExecutionConfig`. Then we could get rid of the `Option` here. > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Wish > Components: Distributed Coordination >Affects Versions: 1.2.0 >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[
https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547946#comment-15547946
]
ASF GitHub Bot commented on FLINK-4596:
---
GitHub user nguraja opened a pull request:
https://github.com/apache/flink/pull/2592
FLINK-4596: Fallback restart strategy config to let jobs choose resta…
Thanks for contributing to Apache Flink. Before you open your pull request,
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your
pull request. For more information and/or questions please refer to the [How To
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful
description of your changes.
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
…rt configuration set at cluster level
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nguraja/flink FLINK-4596
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2592.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2592
commit f511ac9adc789f75761f2f997c294966cfe05b93
Author: Nagarjun
Date: 2016-10-05T07:26:05Z
FLINK-4596: Fallback restart strategy config to let jobs choose restart
configuration set at cluster level
> RESTART_STRATEGY is not really pluggable
>
>
> Key: FLINK-4596
> URL: https://issues.apache.org/jira/browse/FLINK-4596
> Project: Flink
> Issue Type: Wish
> Components: Distributed Coordination
>Affects Versions: 1.2.0
>Reporter: Nagarjun Guraja
>
> Standalone cluster config accepts an implementation(class) as part of the
> yaml config file but that does not work either as cluster level restart
> strategy or streaming job level restart strategy
> CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value
> of strategyname to lowercase and searches for class name using lowercased
> string.
> JOB LEVEL CAUSE: Checkpointed streams have specific code to add
> fixeddelayrestartconfiguration if no RestartConfiguration is specified in
> the job env. Also, jobs cannot provide their own custom restart strategy
> implementation and are constrained to pick up one of the three restart
> strategies provided by flink.
> FIX: Do not lower case the strategy config value, support a new
> restartconfiguration to fallback to cluster level restart strategy and
> support jobs to provide custom implementation of the strategy class itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[
https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522615#comment-15522615
]
Till Rohrmann commented on FLINK-4596:
--
Hi [~nguraja], you're right that the class loading will fail due to the lower
casing of the restart strategy. This is clearly a bug and I'll fix it.
At the moment, the restart strategies are not intended to be pluggable. That's
also why you are not allowed create you own {{RestartStrategyConfigurations}}.
A problem, for example, is that the default restart strategy will be created
when you start the job manager. Thus, there won't be any user code classes
around. If you want to use this, then you would have to put the custom restart
strategy manually in the classpath of the {{JobManager}}.
If you need custom restart strategies, then we can change this restriction. We
could then introduce a {{CustomRestartStrategyConfiguration}} interface which
is capable to instantiate {{RestartStrategies}}. However, this class would have
to be serializable.
> RESTART_STRATEGY is not really pluggable
>
>
> Key: FLINK-4596
> URL: https://issues.apache.org/jira/browse/FLINK-4596
> Project: Flink
> Issue Type: Bug
>Reporter: Nagarjun Guraja
>
> Standalone cluster config accepts an implementation(class) as part of the
> yaml config file but that does not work either as cluster level restart
> strategy or streaming job level restart strategy
> CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value
> of strategyname to lowercase and searches for class name using lowercased
> string.
> JOB LEVEL CAUSE: Checkpointed streams have specific code to add
> fixeddelayrestartconfiguration if no RestartConfiguration is specified in
> the job env. Also, jobs cannot provide their own custom restart strategy
> implementation and are constrained to pick up one of the three restart
> strategies provided by flink.
> FIX: Do not lower case the strategy config value, support a new
> restartconfiguration to fallback to cluster level restart strategy and
> support jobs to provide custom implementation of the strategy class itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515992#comment-15515992 ] Stephan Ewen commented on FLINK-4596: - Adding [~till.rohrmann] to this - he created the restart strategies. > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Bug >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4596) RESTART_STRATEGY is not really pluggable
[ https://issues.apache.org/jira/browse/FLINK-4596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513734#comment-15513734 ] Monal Daxini commented on FLINK-4596: - Hi [~StephanEwen] what are your thoughts on this? > RESTART_STRATEGY is not really pluggable > > > Key: FLINK-4596 > URL: https://issues.apache.org/jira/browse/FLINK-4596 > Project: Flink > Issue Type: Bug >Reporter: Nagarjun Guraja > > Standalone cluster config accepts an implementation(class) as part of the > yaml config file but that does not work either as cluster level restart > strategy or streaming job level restart strategy > CLUSTER LEVEL CAUSE: createRestartStrategyFactory converts configured value > of strategyname to lowercase and searches for class name using lowercased > string. > JOB LEVEL CAUSE: Checkpointed streams have specific code to add > fixeddelayrestartconfiguration if no RestartConfiguration is specified in > the job env. Also, jobs cannot provide their own custom restart strategy > implementation and are constrained to pick up one of the three restart > strategies provided by flink. > FIX: Do not lower case the strategy config value, support a new > restartconfiguration to fallback to cluster level restart strategy and > support jobs to provide custom implementation of the strategy class itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
