[ https://issues.apache.org/jira/browse/STORM-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Zowalla closed STORM-3547. ---------------------------------- Resolution: Won't Fix Kinesis is gone with 2.6.0 > KinesisSpout incorrectly handles closed shards > ---------------------------------------------- > > Key: STORM-3547 > URL: https://issues.apache.org/jira/browse/STORM-3547 > Project: Apache Storm > Issue Type: Bug > Components: storm-kinesis > Affects Versions: 2.1.0 > Reporter: Vijay Bang > Priority: Major > > The KinesisSpout throws an exception when consuming closed Kinesis shards, > which return null from `GetShardIterator`, as it tries to call `GetRecords` > with the null `shardIterator` value instead of abandoning the closed shard. > This means KinesisSpout fails after re-sharding a stream with an exception > like the following: > > {code:java} > java.lang.RuntimeException: > com.amazonaws.services.kinesis.model.AmazonKinesisException: 1 validation > error detected: Value null at 'shardIterator' failed to satisfy constraint: > Member must not be null (S ervice: AmazonKinesis; Status Code: 400; Error > Code: ValidationException; Request ID: d85076a4-3953-fb6e-8e08-331e7d91ef0f) > at org.apache.storm.utils.Utils$1.run(Utils.java:407) > ~[storm-client-2.1.0.jar:2.1.0] at java.lang.Thread.run(Thread.java:745) > [?:1.8.0_101] Caused by: > com.amazonaws.services.kinesis.model.AmazonKinesisException: 1 validation > error detected: Value null at 'shardIterator' failed to satisfy constraint: > Member must not be null (Service: AmazonKin esis; Status Code: 400; Error > Code: ValidationException; Request ID: d85076a4-3953-fb6e-8e08-331e7d91ef0f) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2803) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2772) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2761) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.services.kinesis.AmazonKinesisClient.executeGetRecords(AmazonKinesisClient.java:1288) > ~[stormjar.jar:3.4.6-1569965] at > com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:1259) > ~[stormjar.jar:3.4.6-1569965] at > org.apache.storm.kinesis.spout.KinesisConnection.fetchRecords(KinesisConnection.java:113) > ~[stormjar.jar:3.4.6-1569965] at > org.apache.storm.kinesis.spout.KinesisRecordsManager.fetchNewRecords(KinesisRecordsManager.java:329) > ~[stormjar.jar:3.4.6-1569965] at > org.apache.storm.kinesis.spout.KinesisRecordsManager.next(KinesisRecordsManager.java:135) > ~[stormjar.jar:3.4.6-1569965] at > org.apache.storm.kinesis.spout.KinesisSpout.nextTuple(KinesisSpout.java:82) > ~[stormjar.jar:3.4.6-1569965] at > org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:192) > ~[storm-client-2.1.0.jar:2.1.0] at > org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:159) > ~[storm-client-2.1.0.jar:2.1.0] at > org.apache.storm.utils.Utils$1.run(Utils.java:392) > ~[storm-client-2.1.0.jar:2.1.0] > {code} > > Same Problem was resolved in https://issues.apache.org/jira/browse/BEAM-2582 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)