[ https://issues.apache.org/jira/browse/HADOOP-14071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran resolved HADOOP-14071. ------------------------------------- Resolution: Duplicate > S3a: Failed to reset the request input stream > --------------------------------------------- > > Key: HADOOP-14071 > URL: https://issues.apache.org/jira/browse/HADOOP-14071 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 > Affects Versions: 3.0.0-alpha2 > Reporter: Seth Fitzsimmons > > When using the patch from HADOOP-14028, I fairly consistently get {{Failed to > reset the request input stream}} exceptions. They're more likely to occur the > larger the file that's being written (70GB in the extreme case, but it needs > to be one file). > {code} > 2017-02-10 04:21:43 WARN S3ABlockOutputStream:692 - Transfer failure of block > FileBlock{index=416, > destFile=/tmp/hadoop-root/s3a/s3ablock-0416-4228067786955989475.tmp, > state=Upload, dataSize=11591473, limit=104857600} > 2017-02-10 04:21:43 WARN S3AInstrumentation:777 - Closing output stream > statistics while data is still marked as pending upload in > OutputStreamStatistics{blocksSubmitted=416, blocksInQueue=0, blocksActive=0, > blockUploadsCompleted=416, blockUploadsFailed=3, > bytesPendingUpload=209747761, bytesUploaded=43317747712, blocksAllocated=416, > blocksReleased=416, blocksActivelyAllocated=0, > exceptionsInMultipartFinalize=0, transferDuration=1389936 ms, > queueDuration=519 ms, averageQueueTime=1 ms, totalUploadDuration=1390455 ms, > effectiveBandwidth=3.1153649497466657E7 bytes/s} > at org.apache.hadoop.fs.s3a.S3AUtils.extractException(S3AUtils.java:200) > at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:128) > Exception in thread "main" org.apache.hadoop.fs.s3a.AWSClientIOException: > Multi-part upload with id > 'Xx.ezqT5hWrY1W92GrcodCip88i8rkJiOcom2nuUAqHtb6aQX__26FYh5uYWKlRNX5vY5ktdmQWlOovsbR8CLmxUVmwFkISXxDRHeor8iH9nPhI3OkNbWJJBLrvB3xLUuLX0zvGZWo7bUrAKB6IGxA--' > to 2017/planet-170206.orc on 2017/planet-170206.orc: > com.amazonaws.ResetException: Failed to reset the request input stream; If > the request involves an input stream, the maximum stream buffer size can be > configured via request.getRequestClientOptions().setReadLimit(int): Failed to > reset the request input stream; If the request involves an input stream, the > maximum stream buffer size can be configured via > request.getRequestClientOptions().setReadLimit(int) > at > org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.waitForAllPartUploads(S3ABlockOutputStream.java:539) > at > org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$100(S3ABlockOutputStream.java:456) > at > org.apache.hadoop.fs.s3a.S3ABlockOutputStream.close(S3ABlockOutputStream.java:351) > at > org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) > at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101) > at org.apache.orc.impl.PhysicalFsWriter.close(PhysicalFsWriter.java:221) > at org.apache.orc.impl.WriterImpl.close(WriterImpl.java:2827) > at net.mojodna.osm2orc.standalone.OsmPbf2Orc.convert(OsmPbf2Orc.java:296) > at net.mojodna.osm2orc.Osm2Orc.main(Osm2Orc.java:47) > Caused by: com.amazonaws.ResetException: Failed to reset the request input > stream; If the request involves an input stream, the maximum stream buffer > size can be configured via request.getRequestClientOptions().setReadLimit(int) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.resetRequestInputStream(AmazonHttpClient.java:1221) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1042) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:948) > at > org.apache.hadoop.fs.s3a.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:222) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:635) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:618) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:661) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:573) > at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:445) > at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4041) > at > com.amazonaws.services.s3.AmazonS3Client.doUploadPart(AmazonS3Client.java:3041) > at > com.amazonaws.services.s3.AmazonS3Client.uploadPart(AmazonS3Client.java:3026) > at org.apache.hadoop.fs.s3a.S3AFileSystem.uploadPart(S3AFileSystem.java:1114) > at > org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload$1.call(S3ABlockOutputStream.java:501) > at > org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload$1.call(S3ABlockOutputStream.java:492) > at > org.apache.hadoop.fs.s3a.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:222) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:586) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > ... 20 more > Caused by: java.io.IOException: Resetting to invalid mark > at java.io.BufferedInputStream.reset(BufferedInputStream.java:448) > at > com.amazonaws.internal.SdkFilterInputStream.reset(SdkFilterInputStream.java:102) > at com.amazonaws.event.ProgressInputStream.reset(ProgressInputStream.java:169) > at > com.amazonaws.internal.SdkFilterInputStream.reset(SdkFilterInputStream.java:102) > at > com.amazonaws.http.AmazonHttpClient$RequestExecutor.resetRequestInputStream(AmazonHttpClient.java:1219) > at > com.amazonaws.internal.SdkBufferedInputStream.reset(SdkBufferedInputStream.java:106) > {code} > Potentially relevant: https://github.com/aws/aws-sdk-java/issues/427 -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org