[GitHub] flink pull request #4232: [FLINK-7041] Deserialize StateBackend from JobChec...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4232 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #4232: [FLINK-7041] Deserialize StateBackend from JobChec...
Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/4232#discussion_r125034182 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionGraphBuilder.java --- @@ -223,19 +223,21 @@ public static ExecutionGraph buildGraph( // if specified in the application, use from there, otherwise load from configuration final StateBackend metadataBackend; - final StateBackend applicationConfiguredBackend = snapshotSettings.getDefaultStateBackend(); + final SerializedValue applicationConfiguredBackend = snapshotSettings.getDefaultStateBackend(); if (applicationConfiguredBackend != null) { - metadataBackend = applicationConfiguredBackend; + try { + metadataBackend = applicationConfiguredBackend.deserializeValue(classLoader); + } catch (IOException | ClassNotFoundException e) { + throw new JobExecutionException(jobId, "Could not instantiate configured state backend.", e); + } log.info("Using application-defined state backend for checkpoint/savepoint metadata: {}.", applicationConfiguredBackend); --- End diff -- Good catch! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #4232: [FLINK-7041] Deserialize StateBackend from JobChec...
Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/4232#discussion_r125004426 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionGraphBuilder.java --- @@ -223,19 +223,21 @@ public static ExecutionGraph buildGraph( // if specified in the application, use from there, otherwise load from configuration final StateBackend metadataBackend; - final StateBackend applicationConfiguredBackend = snapshotSettings.getDefaultStateBackend(); + final SerializedValue applicationConfiguredBackend = snapshotSettings.getDefaultStateBackend(); if (applicationConfiguredBackend != null) { - metadataBackend = applicationConfiguredBackend; + try { + metadataBackend = applicationConfiguredBackend.deserializeValue(classLoader); + } catch (IOException | ClassNotFoundException e) { + throw new JobExecutionException(jobId, "Could not instantiate configured state backend.", e); + } log.info("Using application-defined state backend for checkpoint/savepoint metadata: {}.", applicationConfiguredBackend); --- End diff -- Need to change the argument for this log to `metadataBackend` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #4232: [FLINK-7041] Deserialize StateBackend from JobChec...
GitHub user aljoscha opened a pull request: https://github.com/apache/flink/pull/4232 [FLINK-7041] Deserialize StateBackend from JobCheckpointingSettings with user classloader R: @tzulitai CC: @StephanEwen I noticed that the state backend that is in the configuration is never used anywhere. I'm guessing this is for future work or is something missing in the code? You can merge this pull request into a Git repository by running: $ git pull https://github.com/aljoscha/flink jira-7041-fix-state-backend-classloader Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4232.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4232 commit 5ed5300b510f1439ceb516b44592e433dbc46931 Author: Aljoscha KrettekDate: 2017-06-30T09:19:30Z [FLINK-7041] Deserialize StateBackend from JobCheckpointingSettings with user classloader This also adds a test for serializability with a class that's not in the default classloader. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---