Hi Fabian and Chesnay,
Thank you guys.
Fabian : Unfortunately, as Chesnay said, MetricGroup doesn't allow for
ProcessWindowFunction to access to a counter defined in Trigger.
Chesnay : I'm going to follow your advice on how to modify Flink. Thank you
very much!
Best,
- Dongwon
On Thu, Jun 21,
Without modifications to Flink? No. By design nothing can intercept or
retrieve metrics with the metrics API.
For this pattern the usual recommendation is to explicitly pass the
metric to components that require it.
If modifications are an option, what you could do is
* define a counter in the
Hi Dongwon,
Yes, the counter state should be stored in operator state which is not
available on Triggers.
Chesnay: Can a window function (like ProcessWindowFunction) access (read,
write) the counter of its associated Trigger to checkpoint and restore it?
Best, Fabian
2018-06-20 16:59 GMT+02:00
Hi Fabian and Chesnay,
As Chesnay pointed out, it seems that I need to write the current counter
(which is defined inside Trigger) into state which I think should be the
operator state of the window operator.
However, as I previously said, TriggerContext allows for users to access
only the
Checkpointing of metrics is a manual process.
The operator must write the current value into state, retrieve it on
restore and restore the counter's count.
On 20.06.2018 12:10, Fabian Hueske wrote:
Hi Dongwon,
You are of course right! We need to decrement the counter when the
window is
Hi Dongwon,
You are of course right! We need to decrement the counter when the window
is closed.
The idea of using Trigger.clear() (the clean up method is called clear()
instead of onClose()) method is great!
It will be called when the window is closed but also when it is merged.
So, I think you
Hi Fabian,
Thanks a lot for your reply.
Do you need to number of active session windows as a DataStream or would
> you like to have it as a metric that you can expose.
> I possible, I would recommend to expose it as a metric because they are
> usually easier to collect.
I want to have it as a
Hi Dongwon,
Do you need to number of active session windows as a DataStream or would
you like to have it as a metric that you can expose.
I possible, I would recommend to expose it as a metric because they are
usually easier to collect.
SessionWindows work internally as follows:
- every new
Hi Fabian,
I'm still eager to expose # of active sessions as a key metric of our service
but I haven’t figured it out yet.
First of all, I want to ask you some questions regarding your suggestion.
> You could implement a Trigger that fires when a new window is created and
> when the window is
Hi Dongwon Kim,
That's an interesting question.
I don't have a solution blueprint for you, but a few ideas that should help
to solve the problem.
I would start with a separate job first and later try to integrate it with
the other job.
You could implement a Trigger that fires when a new window
Hi,
It could be a totally stupid question but I currently have no idea how to get
the number of active session windows from a running job.
Our traffic trajectory application (which handles up to 10,000 tps) uses
event-time session window on KeyedStream (keyed by userID).
Should I write
11 matches
Mail list logo
