[
https://issues.apache.org/jira/browse/HADOOP-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17352480#comment-17352480
]
Steve Loughran commented on HADOOP-17737:
-----------------------------------------
This looks like a retry caused by a (real world) network connection reset, so
the stats were out.
Going to close as a cannot reproduce; the JIRA is here for others to see if
they search for it.
{code}
2021-05-27 13:10:40,058 DEBUG [pool-44-thread-2]: services.AbfsClient
(AbfsRestOperation.java:executeHttpOperation(295)) - HttpRequestFailure:
0,,,cid=65a6ca53-dccc-43a9-9991-dd6f553fb956,rid=,sent=1572864,recv=0,PUT,https://stevelukwest.dfs.core.windows.net/abfs-testcontainer-57c8928e-2313-4b11-bb3e-76f331a08c53/user/stevel/testSmallWriteOptimization%5BOptmOFF_FlushCloseTest_EmptyFile_MultiSmallWritesStillLessThanBufferSize%5D375a3beb-a78c-4d94-9453-04c250d71cb1?action=append&position=1572864&timeout=90,
{}
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.wildfly.openssl.OpenSSLSocket.read(OpenSSLSocket.java:423)
at
org.wildfly.openssl.OpenSSLInputStream.read(OpenSSLInputStream.java:41)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at
org.apache.hadoop.fs.azurebfs.services.AbfsHttpOperation.processResponse(AbfsHttpOperation.java:330)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.executeHttpOperation(AbfsRestOperation.java:274)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.completeExecute(AbfsRestOperation.java:205)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.lambda$execute$0(AbfsRestOperation.java:181)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfInvocation(IOStatisticsBinding.java:464)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:179)
at
org.apache.hadoop.fs.azurebfs.services.AbfsClient.append(AbfsClient.java:604)
at
org.apache.hadoop.fs.azurebfs.services.AbfsOutputStream.lambda$writeCurrentBufferToService$0(AbfsOutputStream.java:462)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-05-27 13:10:40,061 DEBUG [pool-44-thread-2]: services.AbfsClient
(AbfsRestOperation.java:completeExecute(208)) - Retrying REST operation Append.
RetryCount = 1
2021-05-27 13:10:45,484 DEBUG [pool-44-thread-2]: services.AbfsClient
(AbfsRestOperation.java:executeHttpOperation(248)) - Signing request with
shared key
2021-05-27 13:10:57,700 DEBUG [pool-44-thread-2]: services.AbfsClient
(AbfsRestOperation.java:executeHttpOperation(307)) - HttpRequest: Append:
202,,,cid=965182bf-0d06-4a3c-9673-83a2afd81992,rid=59d17b3d-501f-002a-58f1-522f4c000000,sent=1572864,recv=0,PUT,https://stevelukwest.dfs.core.windows.net/abfs-testcontainer-57c8928e-2313-4b11-bb3e-76f331a08c53/user/stevel/testSmallWriteOptimization%5BOptmOFF_FlushCloseTest_EmptyFile_MultiSmallWritesStillLessThanBufferSize%5D375a3beb-a78c-4d94-9453-04c250d71cb1?action=append&position=1572864&timeout=90
{code}
> abfs ITestSmallWriteOptimization.testSmallWriteOptimization
> -----------------------------------------------------------
>
> Key: HADOOP-17737
> URL: https://issues.apache.org/jira/browse/HADOOP-17737
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Priority: Minor
> Attachments:
> org.apache.hadoop.fs.azurebfs.ITestSmallWriteOptimization-output.txt
>
>
> (transient) failure of
> ITestSmallWriteOptimization.testSmallWriteOptimization; Azure cardiff over a
> long-haul link
> This is on my manifest committer branch -but I've not done any changes to
> input streams there
> {code}
> [ERROR] Failures:
> [ERROR]
> ITestSmallWriteOptimization.testSmallWriteOptimization:324->formulateSmallWriteTestAppendPattern:437->assertOpStats:499->AbstractAbfsIntegrationTest.assertAbfsStatistics:453->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
> Mismatch in connections_made expected:<7> but was:<8>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
