[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321486#comment-17321486
]
Flink Jira Bot commented on FLINK-13406:
This issue and all of its Sub-Tasks have not been updated for 180 days. So, it
has been labeled "stale-minor". If you are still affected by this bug or are
still interested in this issue, please give an update and remove the label. In
7 days the issue will be closed automatically.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Minor
> Labels: stale-minor
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909024#comment-16909024
]
Chesnay Schepler commented on FLINK-13406:
--
This is not really a new option, I did cover changes to the Configuration that
affect how the MetricConfig is assembled in my last comment.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Minor
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909019#comment-16909019
]
Till Rohrmann commented on FLINK-13406:
---
I think there would be a 3. option: Only store {{Strings}} in {{Properties}}.
This effectively means to replace {{put}} calls with {{setProperty}}. That's
also what the JavaDocs of {{Properties}} say. It is a bit unfortunate that
{{Properties extends {{Hashtable}} because it obviously is not a {{Hashtable}}.
Consequently, one could argue that we are misusing {{Properties}} and the
{{MetricConfig}} by calling
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/configuration/Configuration.java#L661.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Minor
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909002#comment-16909002
]
Chesnay Schepler commented on FLINK-13406:
--
Not necessarily.
So let's go through all scenarios:
# An actual cluster, with a job submitted through the CLI, with only the
flink-conf.yaml as an input for config parameters.
In this case, the Configuration only contains strings, the MetricConfig only
contains strings, and everything works as expected.
This is the case that the current code was written for.
# A local cluster, where a custom Configuration was passed to the
ExecutionEnvironment.
Here, the Configuration may contain values that are not strings, in turn the
MetricConfig may contain values that are not strings, and reporters may run
into issues when trying to retrieve a non-string value. This is the originally
reported issue.
# A MetricConfig is manually created by putting in arbitrary objects.
The MetricConfig does not do any type conversions, and reporters may run into
the same issues as in 2).
This case should mostly apply to unit tests.
So, with that said:
Changing the Configuration, or the transformation of Configuration ->
MetricConfig can only cover case 2). I would regard this option as undesirable
for that reason, although I would generally prefer if the Configuration were
entirely based on strings. (But IIRC the Streaming API does some funky things
with the Configuration, so I don't know how feasible this is).
Thus we effectively have to touch the MetricConfig instead, and there are 2
approaches we can opt for:
# Sanitize inputs; override all methods that add entries and convert objects to
strings if necessary.
# Beef up accessors, to account for cases where properties may not be strings.
(Basically, if (property is String) -> parse; elif (property is targetType) ->
return; else -> fail)
Or, we just don't do anything and attribute it to user error.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Minor
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908931#comment-16908931
]
Till Rohrmann commented on FLINK-13406:
---
Isn't the problem that {{Configuration}} uses {{HashMap
confData}} to store values. Hence, depending on whether you load the
{{Configuration}} from disk or set it programmatically, the internal state of
it looks different (storing an {{Integer}} object instead of a {{String}}).
Hence, I can understand the confusion.
As a quick fix, I would propose to simply use
{{configuration.setString("metrics.reporter.my.foo", "42")}} [~oripwk].
But we should discuss whether the behaviour of the {{Configuration}} is
actually correct or not.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897207#comment-16897207
]
Chesnay Schepler commented on FLINK-13406:
--
note that or testing reporters you should naturally relies on strings since
that is what the reporter is actually given at runtime in an actual cluster.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897205#comment-16897205
]
Chesnay Schepler commented on FLINK-13406:
--
Only when running things in local mode, which I consider as a testing purpose.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897188#comment-16897188
]
Ori Popowski commented on FLINK-13406:
--
{quote}I'm not sure; it will never affect users running against actual
clusters; it's just a gotcha that exists in tests.
{quote}
That's not quite true. It affects people who develop custom Metrics Reporters.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897152#comment-16897152
]
Chesnay Schepler commented on FLINK-13406:
--
I'm not sure; it will never affect users running against actual clusters; it's
just a gotcha that exists in tests.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16897138#comment-16897138
]
Robert Metzger commented on FLINK-13406:
Does this mean we can close this ticket?
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13406) MetricConfig.getInteger() always returns null
[
https://issues.apache.org/jira/browse/FLINK-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892654#comment-16892654
]
Chesnay Schepler commented on FLINK-13406:
--
This isn't quite incorrect. The {{MetricConfig}} primarily provides access to
what is configured in the {{flink-conf.yaml}}, which is read in a way such that
all values _are_ strings, and as such {{MetricConfig#getInteger}} does work.
> MetricConfig.getInteger() always returns null
> -
>
> Key: FLINK-13406
> URL: https://issues.apache.org/jira/browse/FLINK-13406
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics
>Affects Versions: 1.6.3
>Reporter: Ori Popowski
>Priority: Major
>
> {{MetricConfig}}'s {{getInteger}} will always return the default value.
> The reason is, since it delegates to Java's {{Properties.getProperty}} which
> returns null if the type of the value is not {{String}}.
> h3. Reproduce
> # Create a class {{MyReporter}} implementing {{MetricReporter}}
> # Implment the {{open()}} method so that you do {{config.getInteger("foo",
> null)}}
> # Start an {{ExecutionEnvironment}} with and give it the following
> Configuration object:
> {code:java}
> configuration.setString("metrics.reporters", "my");
> configuration.setClass("metrics.reporter.my.class", MyReporter.class)
> configuration.setInteger("metrics.reporter.my.foo", 42);{code}
> # In {{open()}} the value of {{foo}} is null.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
