[
https://issues.apache.org/jira/browse/FLINK-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011874#comment-16011874
]
Tzu-Li (Gordon) Tai commented on FLINK-5946:
--------------------------------------------
Great news! Thanks for the info Scott.
> Kinesis Producer uses KPL that orphans threads that consume 100% CPU
> --------------------------------------------------------------------
>
> Key: FLINK-5946
> URL: https://issues.apache.org/jira/browse/FLINK-5946
> Project: Flink
> Issue Type: Bug
> Components: Kinesis Connector
> Affects Versions: 1.2.0
> Reporter: Scott Kidder
>
> It's possible for the Amazon Kinesis Producer Library (KPL) to leave orphaned
> threads running after the producer has been instructed to shutdown via the
> `destroy()` method. These threads run in a very tight infinite loop that can
> push CPU usage to 100%. I've seen this happen on several occasions, though it
> does not happen all of the time. Once these threads are orphaned, the only
> solution to bring CPU utilization back down is to restart the Flink Task
> Manager.
> When a KPL producer is instantiated, it creates several threads: one to
> execute and monitor the native sender process, and two threads to monitor the
> process' stdout and stderr output. It's possible for the process-monitor
> thread to stop in such a way that leaves the output monitor threads orphaned.
> I've submitted a Github issue and pull-request against the KPL project:
> https://github.com/awslabs/amazon-kinesis-producer/issues/93
> https://github.com/awslabs/amazon-kinesis-producer/pull/94
> This issue is rooted in the Amazon Kinesis Producer Library (KPL) that the
> Flink Kinesis streaming connector depends upon. It ought to be fixed in the
> KPL, but I want to document it on the Flink project. The Flink KPL dependency
> should be updated once the KPL has been fixed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
