[
https://issues.apache.org/jira/browse/FLINK-30418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daren Wong updated FLINK-30418:
-------------------------------
Description:
h2. What?
Use synchronous KinesisClient for consumer
registration/de-registration/describe.
h2.
h2. Why?
KinesisProxyV2 supports the following operations:
* subscribeToShard
* describeStreamSummary
* describeStreamConsumer
* registerStreamConsumer
* deregisterStreamConsumer
All the above operations are currently performed via an AsyncHTTPClient.
However, only subsribeToShard needs to be asynchronous, the rest are called
synchronously.
In addition, there are known issue with async client as follows:
* Async client can cause the following RuntimeException when app parallelism
is high.
{quote}Unable to execute HTTP request: The channel was closed before the
protocol could be determined.
{quote}
* It can also cause deadlock situation as described in
https://issues.apache.org/jira/browse/FLINK-30304
was:
Use synchronous KinesisClient for consumer registration/de-registration.
h3.
Why?
This is because current async client can cause the following RuntimeException
when app parallelism is high.
{quote}Unable to execute HTTP request: The channel was closed before the
protocol could be determined.
{quote}
In addition, it can also cause deadlock situation as described in
https://issues.apache.org/jira/browse/FLINK-30304
> Implement synchronous KinesisClient in EFO
> ------------------------------------------
>
> Key: FLINK-30418
> URL: https://issues.apache.org/jira/browse/FLINK-30418
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / AWS
> Reporter: Daren Wong
> Assignee: Daren Wong
> Priority: Major
> Labels: pull-request-available
> Fix For: aws-connector-4.1.0
>
>
> h2. What?
> Use synchronous KinesisClient for consumer
> registration/de-registration/describe.
> h2.
> h2. Why?
> KinesisProxyV2 supports the following operations:
> * subscribeToShard
> * describeStreamSummary
> * describeStreamConsumer
> * registerStreamConsumer
> * deregisterStreamConsumer
>
> All the above operations are currently performed via an AsyncHTTPClient.
> However, only subsribeToShard needs to be asynchronous, the rest are called
> synchronously.
>
> In addition, there are known issue with async client as follows:
> * Async client can cause the following RuntimeException when app parallelism
> is high.
> {quote}Unable to execute HTTP request: The channel was closed before the
> protocol could be determined.
> {quote}
> * It can also cause deadlock situation as described in
> https://issues.apache.org/jira/browse/FLINK-30304
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
