[
https://issues.apache.org/jira/browse/BEAM-6262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on BEAM-6262 started by Alexey Romanenko.
----------------------------------------------
> KinesisIO - ShardReadersPool - Unexpected exception occurred
> ------------------------------------------------------------
>
> Key: BEAM-6262
> URL: https://issues.apache.org/jira/browse/BEAM-6262
> Project: Beam
> Issue Type: Bug
> Components: io-java-kinesis
> Affects Versions: 2.7.0, 2.8.0, 2.9.0
> Reporter: Alexey Romanenko
> Assignee: Alexey Romanenko
> Priority: Minor
>
> While reading a stream with limited number of records or time it throws an
> exception in the end. Though, it seems that there is no data loss, so it's a
> minor issue.
> {noformat}
> INFO: Creating new reader using [Checkpoint AT_TIMESTAMP for stream
> apache-beam, shard shardId-000000000002: null, Checkpoint AT_TIMESTAMP for
> stream apache-beam, shard shardId-000000000001: null]
> Dec 18, 2018 6:41:44 PM org.apache.beam.sdk.io.kinesis.KinesisReader start
> INFO: Starting reader using [Checkpoint AT_TIMESTAMP for stream apache-beam,
> shard shardId-000000000002: null, Checkpoint AT_TIMESTAMP for stream
> apache-beam, shard shardId-000000000001: null]
> Dec 18, 2018 6:41:46 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool stop
> INFO: Closing shard iterators pool
> Dec 18, 2018 6:41:46 PM org.apache.beam.sdk.io.kinesis.ShardReadersPool
> readLoop
> SEVERE: Unexpected exception occurred
> java.lang.RuntimeException: Not retryable client failure
> at
> org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.wrapExceptions(SimplifiedKinesisClient.java:225)
> at
> org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.getRecords(SimplifiedKinesisClient.java:134)
> at
> org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.getRecords(SimplifiedKinesisClient.java:119)
> at
> org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.fetchRecords(ShardRecordsIterator.java:86)
> at
> org.apache.beam.sdk.io.kinesis.ShardRecordsIterator.readNextBatch(ShardRecordsIterator.java:76)
> at
> org.apache.beam.sdk.io.kinesis.ShardReadersPool.readLoop(ShardReadersPool.java:123)
> at
> org.apache.beam.sdk.io.kinesis.ShardReadersPool.lambda$startReadingShards$0(ShardReadersPool.java:114)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: com.amazonaws.AbortedException:
> at
> com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:53)
> at
> com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:81)
> at
> com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180)
> at
> com.fasterxml.jackson.dataformat.cbor.CBORParserBootstrapper.ensureLoaded(CBORParserBootstrapper.java:196)
> at
> com.fasterxml.jackson.dataformat.cbor.CBORParserBootstrapper.constructParser(CBORParserBootstrapper.java:90)
> at
> com.fasterxml.jackson.dataformat.cbor.CBORFactory._createParser(CBORFactory.java:350)
> at
> com.fasterxml.jackson.dataformat.cbor.CBORFactory.createParser(CBORFactory.java:287)
> at
> com.fasterxml.jackson.dataformat.cbor.CBORFactory.createParser(CBORFactory.java:26)
> at
> com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:109)
> at
> com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:43)
> at
> com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1554)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1272)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
> at
> com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
> at
> com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2276)
> at
> com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2252)
> at
> com.amazonaws.services.kinesis.AmazonKinesisClient.executeGetRecords(AmazonKinesisClient.java:1062)
> at
> com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:1038)
> at
> org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.lambda$getRecords$2(SimplifiedKinesisClient.java:137)
> at
> org.apache.beam.sdk.io.kinesis.SimplifiedKinesisClient.wrapExceptions(SimplifiedKinesisClient.java:210)
> ... 11 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
