Re: Custom Counter on Flink File Source
Hi, Kirti.
We could find these information in the 1.18 release wiki page[1].
Its timeline is as follows.
Feature Freeze: July 11, 2023, end of business CEST
Release: End of September 2023
Best,
Hang
[1]
https://cwiki.apache.org/confluence/display/FLINK/1.18+Release#id-1.18Release-Summary
Kirti Dhar Upadhyay K 于2023年6月7日周三
15:49写道：
> Thanks Hang.
>
> Any expected date for Flink 1.18.0 release?
>
>
>
> Regards,
>
> Kirti Dhar
>
>
>
> *From:* Hang Ruan
> *Sent:* 07 June 2023 07:34
> *To:* Kirti Dhar Upadhyay K
> *Cc:* user@flink.apache.org
> *Subject:* Re: Custom Counter on Flink File Source
>
>
>
> Hi, Kirti Dhar Upadhyay K.
>
>
>
> I check the FLIP-274[1]. This issue will be released in the 1.18.0. It is
> not contained in any release now.
>
>
>
> Best,
>
> Hang
>
>
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-274%3A+Introduce+metric+group+for+OperatorCoordinator
>
>
>
> Kirti Dhar Upadhyay K 于2023年6月7日周三
> 02:51写道：
>
> Hi Hang,
>
>
>
> Thanks for reply.
>
> I tried using SplitEnumeratorContext passed in
> AbstractFileSource#createEnumerator but resulted as NullPointerException.
>
> As SplitEnumeratorContext provides its implementation as
> SourceCoordinatorContext having metricGroup() as below-
>
>
>
>
>
> @Override
>
> *public* SplitEnumeratorMetricGroup metricGroup() {
>
> *return* *null*;
>
> }
>
>
>
> Am I doing any mistake?
>
>
>
> Regards,
>
> Kirti Dhar
>
>
>
> *From:* Hang Ruan
> *Sent:* 06 June 2023 08:12
> *To:* Kirti Dhar Upadhyay K
> *Cc:* user@flink.apache.org
> *Subject:* Re: Custom Counter on Flink File Source
>
>
>
> Hi, Kirti Dhar Upadhyay K.
>
>
>
> We could get the metric group from the context, like `SourceReaderContext`
> and `SplitEnumeratorContext`. These contexts could be found when creating
> readers and enumerators. See `AbstractFileSource#createReader` and
> `AbstractFileSource#createEnumerator`.
>
>
>
> Best,
>
> Hang
>
>
>
> Kirti Dhar Upadhyay K via user 于2023年6月5日周一 22:57
> 写道：
>
> Hi Community,
>
>
>
> I am trying to add a new counter for number of files collected on Flink
> File Source.
>
> Referring the doc
> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I
> understand how to add a new counter on any operator.
>
>
>
> *this.*counter *=* *getRuntimeContext().*getMetricGroup*().*counter*(*
> "myCounter"*);*
>
>
>
> But not able to get this RuntimeContext on FileSource.
>
> Can someone give some clue on this?
>
>
>
> Regards,
>
> Kirti Dhar
>
>
RE: Custom Counter on Flink File Source
Thanks Hang.
Any expected date for Flink 1.18.0 release?
Regards,
Kirti Dhar
From: Hang Ruan
Sent: 07 June 2023 07:34
To: Kirti Dhar Upadhyay K
Cc: user@flink.apache.org
Subject: Re: Custom Counter on Flink File Source
Hi, Kirti Dhar Upadhyay K.
I check the FLIP-274[1]. This issue will be released in the 1.18.0. It is not
contained in any release now.
Best,
Hang
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-274%3A+Introduce+metric+group+for+OperatorCoordinator
Kirti Dhar Upadhyay K
mailto:kirti.k.dhar.upadh...@ericsson.com>>
于2023年6月7日周三 02:51写道：
Hi Hang,
Thanks for reply.
I tried using SplitEnumeratorContext passed in
AbstractFileSource#createEnumerator but resulted as NullPointerException.
As SplitEnumeratorContext provides its implementation as
SourceCoordinatorContext having metricGroup() as below-
@Override
public SplitEnumeratorMetricGroup metricGroup() {
return null;
}
Am I doing any mistake?
Regards,
Kirti Dhar
From: Hang Ruan mailto:ruanhang1...@gmail.com>>
Sent: 06 June 2023 08:12
To: Kirti Dhar Upadhyay K
mailto:kirti.k.dhar.upadh...@ericsson.com>>
Cc: user@flink.apache.org<mailto:user@flink.apache.org>
Subject: Re: Custom Counter on Flink File Source
Hi, Kirti Dhar Upadhyay K.
We could get the metric group from the context, like `SourceReaderContext` and
`SplitEnumeratorContext`. These contexts could be found when creating readers
and enumerators. See `AbstractFileSource#createReader` and
`AbstractFileSource#createEnumerator`.
Best,
Hang
Kirti Dhar Upadhyay K via user
mailto:user@flink.apache.org>> 于2023年6月5日周一 22:57写道：
Hi Community,
I am trying to add a new counter for number of files collected on Flink File
Source.
Referring the doc
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I
understand how to add a new counter on any operator.
this.counter = getRuntimeContext().getMetricGroup().counter("myCounter");
But not able to get this RuntimeContext on FileSource.
Can someone give some clue on this?
Regards,
Kirti Dhar
Re: Custom Counter on Flink File Source
Hi, Kirti Dhar Upadhyay K.
I check the FLIP-274[1]. This issue will be released in the 1.18.0. It is
not contained in any release now.
Best,
Hang
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-274%3A+Introduce+metric+group+for+OperatorCoordinator
Kirti Dhar Upadhyay K 于2023年6月7日周三
02:51写道：
> Hi Hang,
>
>
>
> Thanks for reply.
>
> I tried using SplitEnumeratorContext passed in
> AbstractFileSource#createEnumerator but resulted as NullPointerException.
>
> As SplitEnumeratorContext provides its implementation as
> SourceCoordinatorContext having metricGroup() as below-
>
>
>
>
>
> @Override
>
> *public* SplitEnumeratorMetricGroup metricGroup() {
>
> *return* *null*;
>
> }
>
>
>
> Am I doing any mistake?
>
>
>
> Regards,
>
> Kirti Dhar
>
>
>
> *From:* Hang Ruan
> *Sent:* 06 June 2023 08:12
> *To:* Kirti Dhar Upadhyay K
> *Cc:* user@flink.apache.org
> *Subject:* Re: Custom Counter on Flink File Source
>
>
>
> Hi, Kirti Dhar Upadhyay K.
>
>
>
> We could get the metric group from the context, like `SourceReaderContext`
> and `SplitEnumeratorContext`. These contexts could be found when creating
> readers and enumerators. See `AbstractFileSource#createReader` and
> `AbstractFileSource#createEnumerator`.
>
>
>
> Best,
>
> Hang
>
>
>
> Kirti Dhar Upadhyay K via user 于2023年6月5日周一 22:57
> 写道：
>
> Hi Community,
>
>
>
> I am trying to add a new counter for number of files collected on Flink
> File Source.
>
> Referring the doc
> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I
> understand how to add a new counter on any operator.
>
>
>
> *this.*counter *=* *getRuntimeContext().*getMetricGroup*().*counter*(*
> "myCounter"*);*
>
>
>
> But not able to get this RuntimeContext on FileSource.
>
> Can someone give some clue on this?
>
>
>
> Regards,
>
> Kirti Dhar
>
>
RE: Custom Counter on Flink File Source
Hi Hang,
Thanks for reply.
I tried using SplitEnumeratorContext passed in
AbstractFileSource#createEnumerator but resulted as NullPointerException.
As SplitEnumeratorContext provides its implementation as
SourceCoordinatorContext having metricGroup() as below-
@Override
public SplitEnumeratorMetricGroup metricGroup() {
return null;
}
Am I doing any mistake?
Regards,
Kirti Dhar
From: Hang Ruan
Sent: 06 June 2023 08:12
To: Kirti Dhar Upadhyay K
Cc: user@flink.apache.org
Subject: Re: Custom Counter on Flink File Source
Hi, Kirti Dhar Upadhyay K.
We could get the metric group from the context, like `SourceReaderContext` and
`SplitEnumeratorContext`. These contexts could be found when creating readers
and enumerators. See `AbstractFileSource#createReader` and
`AbstractFileSource#createEnumerator`.
Best,
Hang
Kirti Dhar Upadhyay K via user
mailto:user@flink.apache.org>> 于2023年6月5日周一 22:57写道：
Hi Community,
I am trying to add a new counter for number of files collected on Flink File
Source.
Referring the doc
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I
understand how to add a new counter on any operator.
this.counter = getRuntimeContext().getMetricGroup().counter("myCounter");
But not able to get this RuntimeContext on FileSource.
Can someone give some clue on this?
Regards,
Kirti Dhar
Re: Custom Counter on Flink File Source
Hi, Kirti Dhar Upadhyay K. We could get the metric group from the context, like `SourceReaderContext` and `SplitEnumeratorContext`. These contexts could be found when creating readers and enumerators. See `AbstractFileSource#createReader` and `AbstractFileSource#createEnumerator`. Best, Hang Kirti Dhar Upadhyay K via user 于2023年6月5日周一 22:57写道： > Hi Community, > > > > I am trying to add a new counter for number of files collected on Flink > File Source. > > Referring the doc > https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I > understand how to add a new counter on any operator. > > > > *this.*counter *=* *getRuntimeContext().*getMetricGroup*().*counter*(* > "myCounter"*);* > > > > But not able to get this RuntimeContext on FileSource. > > Can someone give some clue on this? > > > > Regards, > > Kirti Dhar >
Custom Counter on Flink File Source
Hi Community,
I am trying to add a new counter for number of files collected on Flink File
Source.
Referring the doc
https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ I
understand how to add a new counter on any operator.
this.counter = getRuntimeContext().getMetricGroup().counter("myCounter");
But not able to get this RuntimeContext on FileSource.
Can someone give some clue on this?
Regards,
Kirti Dhar
