The only way to access this info from the client is the REST API
<https://ci.apache.org/projects/flink/flink-docs-master/monitoring/rest_api.html#details-of-a-running-or-completed-job>
or the Metrics REST API
<https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#rest-api-integration>.
On 14.02.2018 12:38, Flavio Pompermaier wrote:
Actually I'd like to get this number from my Java class in order to
update some external dataset "catalog",
so I'm asking if there's some programmatic way to access this info
(from JobExecutionResult for example).
On Wed, Feb 14, 2018 at 12:25 PM, Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> wrote:
Do you want to know how many records the sink received, or how
many the sink wrote to the DB?
If it's the first you're in luck because we measure that already,
check out the metrics documentation.
If it's the latter, then this issue is essentially covered by
FLINK-7286 which aims at allowing functions
to modify the numRecordsIn/numRecordsOut counts.
On 14.02.2018 12:22, Flavio Pompermaier wrote:
Hi to all,
I have a (batch) job that writes to 1 or more sinks.
Is there a way to retrieve, once the job has terminated, the
number of records written to each sink?
Is there any better way than than using an accumulator for each sink?
If that is the only way to do that, the Sink API could be
enriched in order to automatically create an accumulator when
required. E.g.
dataset.output(JDBCOutputFormat.buildJDBCOutputFormat()
.setDrivername(...)
.setDBUrl(...)
.setQuery(...)
*.addRecordsCountAccumulator("some-name")*
.finish())
Best,
Flavio
--
Flavio Pompermaier
Development Department
OKKAM S.r.l.
Tel. +(39) 0461 041809
