[
https://issues.apache.org/jira/browse/STORM-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Zowalla closed STORM-3547.
----------------------------------
Resolution: Won't Fix
Kinesis is gone with 2.6.0
> KinesisSpout incorrectly handles closed shards
> ----------------------------------------------
>
> Key: STORM-3547
> URL: https://issues.apache.org/jira/browse/STORM-3547
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kinesis
> Affects Versions: 2.1.0
> Reporter: Vijay Bang
> Priority: Major
>
> The KinesisSpout throws an exception when consuming closed Kinesis shards,
> which return null from `GetShardIterator`, as it tries to call `GetRecords`
> with the null `shardIterator` value instead of abandoning the closed shard.
> This means KinesisSpout fails after re-sharding a stream with an exception
> like the following:
>
> {code:java}
> java.lang.RuntimeException:
> com.amazonaws.services.kinesis.model.AmazonKinesisException: 1 validation
> error detected: Value null at 'shardIterator' failed to satisfy constraint:
> Member must not be null (S ervice: AmazonKinesis; Status Code: 400; Error
> Code: ValidationException; Request ID: d85076a4-3953-fb6e-8e08-331e7d91ef0f)
> at org.apache.storm.utils.Utils$1.run(Utils.java:407)
> ~[storm-client-2.1.0.jar:2.1.0] at java.lang.Thread.run(Thread.java:745)
> [?:1.8.0_101] Caused by:
> com.amazonaws.services.kinesis.model.AmazonKinesisException: 1 validation
> error detected: Value null at 'shardIterator' failed to satisfy constraint:
> Member must not be null (Service: AmazonKin esis; Status Code: 400; Error
> Code: ValidationException; Request ID: d85076a4-3953-fb6e-8e08-331e7d91ef0f)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2803)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2772)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2761)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.services.kinesis.AmazonKinesisClient.executeGetRecords(AmazonKinesisClient.java:1288)
> ~[stormjar.jar:3.4.6-1569965] at
> com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:1259)
> ~[stormjar.jar:3.4.6-1569965] at
> org.apache.storm.kinesis.spout.KinesisConnection.fetchRecords(KinesisConnection.java:113)
> ~[stormjar.jar:3.4.6-1569965] at
> org.apache.storm.kinesis.spout.KinesisRecordsManager.fetchNewRecords(KinesisRecordsManager.java:329)
> ~[stormjar.jar:3.4.6-1569965] at
> org.apache.storm.kinesis.spout.KinesisRecordsManager.next(KinesisRecordsManager.java:135)
> ~[stormjar.jar:3.4.6-1569965] at
> org.apache.storm.kinesis.spout.KinesisSpout.nextTuple(KinesisSpout.java:82)
> ~[stormjar.jar:3.4.6-1569965] at
> org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:192)
> ~[storm-client-2.1.0.jar:2.1.0] at
> org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:159)
> ~[storm-client-2.1.0.jar:2.1.0] at
> org.apache.storm.utils.Utils$1.run(Utils.java:392)
> ~[storm-client-2.1.0.jar:2.1.0]
> {code}
>
> Same Problem was resolved in https://issues.apache.org/jira/browse/BEAM-2582
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
