Hello -
In
https://github.com/apache/flink/blob/master/flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesConfigOptions.java
the various supported options are declared as constants.
I see that there is no support for options like Volumes and VolumeMounts.
Also I se
Hi Rex,
I think Latency Marker is what you need [1].
Best,
Yun
[1]
https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#latency-tracking
--
Sender:Rex Fenley
Date:2020/12/21 04:57:59
Recipient:user
Cc:Brad
IIUC, "state.checkpoints.dir" is specifying an external checkpoint path,
which will not be cleaned up unless
the users configured it explicitly[1].
However, for "high-availability.storageDir", it will be cleaned up
automatically when all the jobs in the application
reaches to the terminal state. M
That worked. Thankyou, Kostas.
From: Kostas Kloudas
Date: Sunday, December 20, 2020 at 7:21 AM
To: Ben Beasley
Cc: user@flink.apache.org
Subject: Re: No execution.target specified in your configuration file
Hi Ben,
You can try using StreamExecutionEnvironment
streamExecutionEnvironment =
Strea
I understand this.
State storage Is defined defined by state.checkpointing.dir, for example
state.checkpoints.dir: file:///mnt/flink/storage/checkpoints
I am talking about reference defined in 2 places
> On Dec 20, 2020, at 8:05 PM, Yang Wang wrote:
>
> I am afraid only the state handle is sto
I am afraid only the state handle is stored in the ConfigMap. The real
state data is stored in
the distributed storage configured via "high-availability.storageDir". I
believe you could find
more information in this class KubernetesStateHandleStore[1].
How could you find that the checkpointing inf
Hello,
Is there some proxy to seeing the relative time it takes for records to
make it through an entire job plan? Maybe checkpoint alignment time would
be a proxy for this? Is there metrics for that or something else that would
provide signal here?
Thanks!
--
Rex Fenley | Software Engineer
Hi Ben,
You can try using StreamExecutionEnvironment
streamExecutionEnvironment =
StreamExecutionEnvironment.getExecutionEnvironment();
instead of directly creating a new one. This will allow to pick up the
configuration parameters you pass through the command line.
I hope this helps,
Kostas
On
