[
https://issues.apache.org/jira/browse/IGNITE-19910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-19910:
-----------------------------------
Description:
Currently, in CDC through Kafka applications, single timeout property
({{kafkaRequestTimeout)}} is used for all Kafka related operations instead of
built-in timeouts of Kafka clients (moreover, default value of 3 seconds does
not correspond to Kafka clients defaults):
# {{KafkaProducer}}:
** {{delivery.timeout.ms}}, defaults to 120s [1]
** {{request.timeout.ms}}, defaults to 30s [2]
# {{KafkaKonsumer}}:
** {{default.api.timeout.ms}}, defaults to 60s [3]
** {{request.timeout.ms}}, defaults to 30s [4]
----
Places where {{kafkaRequestTimeout}} is used:
*In ignite-cdc.sh* ({{IgniteToKafkaCdcStreamer}}):
# Waiting for {{KafkaProducer#send}} future to finish.
*In kafka-to-ignite.sh:*
# {{KafkaToIgniteCdcStreamerApplier}} as parameter of methods:
## {{KafkaConsumer#poll}}
## {{KafkaConsumer#commitSync}}
# {{KafkaToIgniteMetadataUpdater}} as parameter of methods.
## {{KafkaConsumer#poll}}
## {{KafkaConsumer#endOffsets}}
## {{KafkaConsumer#partitionsFor}}
----
Also, there is difference of behavior of {{KafkaConsumer#poll}} and other used
{{KafkaConsumer}} methods. Poll just waits for data until specified timeout
expires, but other methods will raise an exception, thus, timeout should not be
to low for them.
Links:
#
https://kafka.apache.org/27/documentation.html#producerconfigs_delivery.timeout.ms
#
https://kafka.apache.org/27/documentation.html#producerconfigs_request.timeout.ms
#
https://kafka.apache.org/27/documentation.html#consumerconfigs_default.api.timeout.ms
#
https://kafka.apache.org/27/documentation.html#consumerconfigs_request.timeout.ms
was:
Currently, in CDC through Kafka applications, single timeout property
({{kafkaRequestTimeout)}} is used for all Kafka related operations instead of
built-in timeouts of Kafka clients (moreover, default value of 3 seconds does
not correspond to Kafka clients defaults):
# {{KafkaProducer}}:
** {{delivery.timeout.ms}}, defaults to 120s [1]
** {{request.timeout.ms}}, defaults to 30s [2]
# {{KafkaKonsumer}}:
** {{default.api.timeout.ms}}, defaults to 60s [3]
** {{request.timeout.ms}}, defaults to 30s [4]
----
Places where {{kafkaRequestTimeout}} is used:
*In ignite-cdc.sh* ({{IgniteToKafkaCdcStreamer}}):
# Waiting for {{KafkaProducer#send}} future to finish.
*In kafka-to-ignite.sh:*
# {{KafkaToIgniteCdcStreamerApplier}} as parameter of methods:
## {{KafkaConsumer#poll}}
## {{KafkaConsumer#commitSync}}
# {{KafkaToIgniteMetadataUpdater}} as parameter of methods.
## {{KafkaConsumer#poll}}
## {{KafkaConsumer#endOffsets}}
## {{KafkaConsumer#partitionsFor}}
----
Also, there is difference of behavior of {{KafkaConsumer#poll}} and other used
{{KafkaConsumer}} methods. Poll just waits for data until specified timeout
expires, but other methods will raise an exception, thus, timeout should not be
to low for them.
Links:
#
https://kafka.apache.org/27/documentation.html#producerconfigs_delivery.timeout.ms
#
https://kafka.apache.org/27/documentation.html#producerconfigs_request.timeout.ms
#
https://kafka.apache.org/27/documentation.html#consumerconfigs_default.api.timeout.ms
#
https://kafka.apache.org/27/documentation.html#consumerconfigs_request.timeout.ms
> CDC through Kafka: refactor timeouts
> ------------------------------------
>
> Key: IGNITE-19910
> URL: https://issues.apache.org/jira/browse/IGNITE-19910
> Project: Ignite
> Issue Type: Task
> Components: extensions
> Reporter: Ilya Shishkov
> Priority: Minor
> Labels: IEP-59, ise
>
> Currently, in CDC through Kafka applications, single timeout property
> ({{kafkaRequestTimeout)}} is used for all Kafka related operations instead of
> built-in timeouts of Kafka clients (moreover, default value of 3 seconds does
> not correspond to Kafka clients defaults):
> # {{KafkaProducer}}:
> ** {{delivery.timeout.ms}}, defaults to 120s [1]
> ** {{request.timeout.ms}}, defaults to 30s [2]
> # {{KafkaKonsumer}}:
> ** {{default.api.timeout.ms}}, defaults to 60s [3]
> ** {{request.timeout.ms}}, defaults to 30s [4]
> ----
> Places where {{kafkaRequestTimeout}} is used:
> *In ignite-cdc.sh* ({{IgniteToKafkaCdcStreamer}}):
> # Waiting for {{KafkaProducer#send}} future to finish.
> *In kafka-to-ignite.sh:*
> # {{KafkaToIgniteCdcStreamerApplier}} as parameter of methods:
> ## {{KafkaConsumer#poll}}
> ## {{KafkaConsumer#commitSync}}
> # {{KafkaToIgniteMetadataUpdater}} as parameter of methods.
> ## {{KafkaConsumer#poll}}
> ## {{KafkaConsumer#endOffsets}}
> ## {{KafkaConsumer#partitionsFor}}
> ----
> Also, there is difference of behavior of {{KafkaConsumer#poll}} and other
> used {{KafkaConsumer}} methods. Poll just waits for data until specified
> timeout expires, but other methods will raise an exception, thus, timeout
> should not be to low for them.
> Links:
> #
> https://kafka.apache.org/27/documentation.html#producerconfigs_delivery.timeout.ms
> #
> https://kafka.apache.org/27/documentation.html#producerconfigs_request.timeout.ms
> #
> https://kafka.apache.org/27/documentation.html#consumerconfigs_default.api.timeout.ms
> #
> https://kafka.apache.org/27/documentation.html#consumerconfigs_request.timeout.ms
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
