[
https://issues.apache.org/jira/browse/SPARK-18431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829625#comment-15829625
]
Genmao Yu commented on SPARK-18431:
-----------------------------------
[~srowen] As this is far from a complex issue, there is no need to leave it
open. There are two points to concern about:
- Is 500 a reasonable value? I do not know, or say i do not encounter any
problem. But, I found AWS gave a default value with 1000 * 10. Here,
https://github.com/awslabs/amazon-kinesis-connectors/blob/master/src/main/java/com/amazonaws/services/kinesis/connectors/KinesisConnectorConfiguration.java#L110
- Make it configurable is reasonable, the less hardcode the better. But we need
to change the definition of {{KinesisReceiver}} to pass
{{taskBackoffTimeMillisas}} or {{configuration}} in.
Any suggestion is appreciated.
> Hard coded value in org.apache.spark.streaming.kinesis.KinesisReceiver
> ----------------------------------------------------------------------
>
> Key: SPARK-18431
> URL: https://issues.apache.org/jira/browse/SPARK-18431
> Project: Spark
> Issue Type: Improvement
> Components: DStreams
> Affects Versions: 2.0.1
> Reporter: Shushant Arora
> Priority: Minor
>
> There is a hardcoded value of taskBackoffTimeMillisas 500in onstart method of
> org.apache.spark.streaming.kinesis.KinesisReceiver.Instead of hardcoded value
> it must be configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
