[
https://issues.apache.org/jira/browse/FLINK-29146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shuiqiang Chen updated FLINK-29146:
-----------------------------------
Description:
Currently, when building an ExecutionGraph, it requires to set the job specific
information (like job id, job name, job configuration, etc) and most of them
are from JobGraph.But I find that the configuration in JobGraph is a new
Configuration instance when the JobGraph is built, and it does not contain any
user set configuration. As a result, we are not able retrieve the use specified
job configuration in ExecutionGraph built from JobGraph during execution
runtime.
BTW, in StreamExecutionEnvironment, it seems that job configurations that not
contained in built-in options will be ignored when calling
StreamExecutionEnvironment.configure(ReadableConfig[, ClassLoader]). However,
it will be included when constructing a StreamExecutionEnvironment, which seems
a bit inconsistent. Is it by design?
{code:java}
Configuration configuration = new Configuration();
// These configured string will take effect.
configuration.setString("k1", "v1");
configuration.setString("k2", "v2");
configuration.setString("k3", "v3");
configuration.set(HeartbeatManagerOptions.HEARTBEAT_TIMEOUT, 300000L);
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment(configuration);
// These configured string will be ignored.
configuration.setString("k4", "v4");
configuration.setString("k5", "v5");
configuration.setString("k6", "v6");
env.configure(configuration);
{code}
was:
Currently, when building an ExecutionGraph, it requires to set the job specific
information (like job id, job name, job configuration, etc) and most of them
are from JobGraph.But I find that the configuraiton in JobGraph is a new
Configuration instance that does not contain any user set configuration. As a
result, we are not able retrieve the use specified job configuration in
ExecutionGraph built from JobGraph during runtime execution.
BTW, in StreamExecutionEnvironment, it seems that job configuraitons that not
contained in built-in options will be igored when calling
StreamExecutionEnvironment.configure(ReadableConfig[, ClassLoader]). However,
it will be included when constructing a StreamExecutionEnvironment, which seems
a bit inconsistent.
{code:java}
Configuration configuration = new Configuration();
// These configured string will take effect.
configuration.setString("k1", "v1");
configuration.setString("k2", "v2");
configuration.setString("k3", "v3");
configuration.set(HeartbeatManagerOptions.HEARTBEAT_TIMEOUT, 300000L);
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment(configuration);
// These configured string will be ignored.
configuration.setString("k4", "v4");
configuration.setString("k5", "v5");
configuration.setString("k6", "v6");
env.configure(configuration);
{code}
> User set job configuration can not be retirieved from JobGraph and
> ExecutionGraph
> ---------------------------------------------------------------------------------
>
> Key: FLINK-29146
> URL: https://issues.apache.org/jira/browse/FLINK-29146
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Shuiqiang Chen
> Priority: Major
>
> Currently, when building an ExecutionGraph, it requires to set the job
> specific information (like job id, job name, job configuration, etc) and most
> of them are from JobGraph.But I find that the configuration in JobGraph is a
> new Configuration instance when the JobGraph is built, and it does not
> contain any user set configuration. As a result, we are not able retrieve the
> use specified job configuration in ExecutionGraph built from JobGraph during
> execution runtime.
> BTW, in StreamExecutionEnvironment, it seems that job configurations that not
> contained in built-in options will be ignored when calling
> StreamExecutionEnvironment.configure(ReadableConfig[, ClassLoader]). However,
> it will be included when constructing a StreamExecutionEnvironment, which
> seems a bit inconsistent. Is it by design?
> {code:java}
> Configuration configuration = new Configuration();
> // These configured string will take effect.
> configuration.setString("k1", "v1");
> configuration.setString("k2", "v2");
> configuration.setString("k3", "v3");
> configuration.set(HeartbeatManagerOptions.HEARTBEAT_TIMEOUT, 300000L);
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment(configuration);
> // These configured string will be ignored.
> configuration.setString("k4", "v4");
> configuration.setString("k5", "v5");
> configuration.setString("k6", "v6");
> env.configure(configuration);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
