[
https://issues.apache.org/jira/browse/HADOOP-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612250#comment-16612250
]
Steve Loughran commented on HADOOP-15751:
-----------------------------------------
{code}
[ERROR]
testParallelJobsToAdjacentPaths(org.apache.hadoop.fs.s3a.commit.staging.integration.ITestDirectoryCommitProtocol)
Time elapsed: 26.849 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSClientIOException: put on
fork-0002/test/DELAY_LISTING_ME/ITestDirectoryCommitProtocol-testParallelJobsToAdjacentPathsjob2Dest/_SUCCESS:
com.amazonaws.SdkClientException: Data read has a different length than the
expected: dataLength=0; expectedLength=4130; includeSkipped=false;
in.getClass()=class com.amazonaws.internal.ReleasableInputStream;
markedSupported=true; marked=0; resetSinceLastMarked=false; markCount=1;
resetCount=0: Data read has a different length than the expected: dataLength=0;
expectedLength=4130; includeSkipped=false; in.getClass()=class
com.amazonaws.internal.ReleasableInputStream; markedSupported=true; marked=0;
resetSinceLastMarked=false; markCount=1; resetCount=0
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:188)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
at
org.apache.hadoop.fs.s3a.WriteOperationHelper.retry(WriteOperationHelper.java:123)
at
org.apache.hadoop.fs.s3a.WriteOperationHelper.putObject(WriteOperationHelper.java:428)
at
org.apache.hadoop.fs.s3a.S3ABlockOutputStream.lambda$putObject$0(S3ABlockOutputStream.java:438)
at
org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
at
org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.amazonaws.SdkClientException: Data read has a different length
than the expected: dataLength=0; expectedLength=4130; includeSkipped=false;
in.getClass()=class com.amazonaws.internal.ReleasableInputStream;
markedSupported=true; marked=0; resetSinceLastMarked=false; markCount=1;
resetCount=0
at
com.amazonaws.util.LengthCheckInputStream.checkLength(LengthCheckInputStream.java:151)
at
com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:109)
at
com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82)
at
com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream.read(MD5DigestCalculatingInputStream.java:128)
at
com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82)
at
com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:180)
at
com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:82)
at
com.amazonaws.thirdparty.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:140)
at
com.amazonaws.http.RepeatableInputStreamRequestEntity.writeTo(RepeatableInputStreamRequestEntity.java:160)
at
com.amazonaws.thirdparty.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at
com.amazonaws.thirdparty.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160)
at
com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at
com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63)
at
com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at
com.amazonaws.thirdparty.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at
com.amazonaws.thirdparty.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at
com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at
com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at
com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at
com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4368)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4315)
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1758)
at
org.apache.hadoop.fs.s3a.InconsistentAmazonS3Client.putObject(InconsistentAmazonS3Client.java:226)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.putObjectDirect(S3AFileSystem.java:1534)
at
org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$putObject$5(WriteOperationHelper.java:430)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
... 13 more
{code}
> AWS Data read stack trace in S3a putObjectDirect
> ------------------------------------------------
>
> Key: HADOOP-15751
> URL: https://issues.apache.org/jira/browse/HADOOP-15751
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Minor
>
> (transient) failure in a put to AWS london from UK Bristol; 120+ miles away
> (I.e not that far).
> Error "Data read has a different length than the expected:"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
