[
https://issues.apache.org/jira/browse/FLINK-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Cranmer reassigned FLINK-16003:
-------------------------------------
Assignee: Emre Kartoglu
> ShardConsumer errors cannot be logged
> -------------------------------------
>
> Key: FLINK-16003
> URL: https://issues.apache.org/jira/browse/FLINK-16003
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
> Affects Versions: 1.6.3
> Reporter: Ori Popowski
> Assignee: Emre Kartoglu
> Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> pull-request-available
>
> Some of the errors in the Kinesis connector show on Flink UI, but are not
> logged. This causes a serious problem since we cannot see them in our logging
> aggregation platform, and we cannot create alerts on them. One of the errors
> is the following:
>
> {code:java}
> java.lang.RuntimeException: Rate Exceeded for getRecords operation - all 3
> retry attempts returned ProvisionedThroughputExceededException.
> at
> org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getRecords(KinesisProxy.java:234)
> at
> org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.getRecords(ShardConsumer.java:311)
> at
> org.apache.flink.streaming.connectors.kinesis.internals.ShardConsumer.run(ShardConsumer.java:219)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
>
>
> The code shows that this exception is not caught and hence is not logged and
> can be detected only from the "exceptions" tab in Flink UI. People who use
> the connector cannot leverage logging and metrics when such an exception
> occurs.
> It can be useful to catch all the throwables from the ShardConsumer's run()
> method and log them.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
