[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:13 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
{color:red}The default value of this configuration options didn't take effect.
{color}checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
I checked the flink source code for this configuration, and the default is
indeed 1.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
I checked the flink source code for this configuration, and the default is
indeed 1.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:13 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
{color:red}The default value of this configuration options didn't take effect.
{color}
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
I checked the flink source code for this configuration, and the default is
indeed 1.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
{color:red}The default value of this configuration options didn't take effect.
{color}checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
I checked the flink source code for this configuration, and the default is
indeed 1.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:11 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
I checked the flink source code for this configuration, and the default is
indeed 1.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:09 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/0291345f4624476a24dcb5c03e3a303e/chk-3
..
{color:red}As specified in the configuration, only one file is retained
{color}
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
checkpoints files:
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-1
hdfs://flinkcluster/flink/checkpoint/c03ac3a71c52f2f33fd518089cc78c76/chk-2
..
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:04 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
But When I remove this configuration,now the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
But When I remove this configuration(-Dstate.checkpoints.num-retained=1 ),now
the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468375#comment-17468375
]
chenqizhu edited comment on FLINK-25296 at 1/4/22, 6:03 AM:
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.checkpoints.num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
But When I remove this configuration(-Dstate.checkpoints.num-retained=1 ),now
the command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
The default value of this configuration options didn't take effect.
{code:java}
public static final ConfigOption MAX_RETAINED_CHECKPOINTS =
ConfigOptions.key("state.checkpoints.num-retained")
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
{code}
was (Author: libra_816):
Hi [~pnowojski] You are right. i mean adding -Dstate.checkpoints.num-retained
via command-Line Interface, like this :
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
-Dstate.cache. num-retained=1 examples/streaming/TopSpeedWindowing.jar.
{code}
This configuration options is valid.
But When I remove this configuration(-Dstate.cache. num-retained=1 ),now the
command line becomes like this:
{code:java}
bin/flink run -t yarn-per-job -Dexecution.checkpointing.interval=60s
examples/streaming/TopSpeedWindowing.jar.
{code}
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.checkpoints.num-retained=n
> via command-Line Interface
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25296) [state.checkpoints.num-retained ]The default value does not take effect
[
https://issues.apache.org/jira/browse/FLINK-25296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17467258#comment-17467258
]
Piotr Nowojski edited comment on FLINK-25296 at 12/31/21, 2:15 PM:
---
Hi [~libra_816]. What do you mean by {{-Dstate.cache.num-retained}}? I couldn't
find such property/option defined/documented anywhere. Am I missing something?
Did you mean adding {{-Dstate.checkpoints.num-retained}}? If so, I'm also not
sure where are you passing such property, because as far as I know Flink only
supports passing configuration options via {{flink-conf.yaml}} file (please
check [the
docs|https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/config/]),
or via {{FLINK_PROPERTIES}} on docker-based deployments. Or maybe I am missing
something?
Also for the future, I would suggest reaching out via user mailing list with
questions like that first:
https://flink.apache.org/community.html#mailing-lists
It usually offers faster response times.
was (Author: pnowojski):
Hi [~libra_816]. What do you mean by {{-Dstate.cache.num-retained}}? I couldn't
find such property/option defined/documented anywhere. Am I missing something?
Did you mean adding {{-Dstate.checkpoints.num-retained}}? If so, I'm also not
sure where are you passing such property, because as far as I know Flink only
supports passing configuration options via {{flink-conf.yaml}} file (please
check [the
docs|https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/config/]),
or via {{FLINK_PROPERTIES}} on docker-based deployments.
Also for the future, I would suggest reaching out via user mailing list with
questions like that first:
https://flink.apache.org/community.html#mailing-lists
It usually offers faster response times.
> [state.checkpoints.num-retained ]The default value does not take effect
> ---
>
> Key: FLINK-25296
> URL: https://issues.apache.org/jira/browse/FLINK-25296
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.3
>Reporter: chenqizhu
>Priority: Major
>
> The configration is valid only when I add -Dstate.cache. num-retained=n.
> If I do not specify this configuration, the default value does not take
> effect , is it a bug ?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
