[
https://issues.apache.org/jira/browse/HADOOP-16622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950216#comment-16950216
]
Kai Xie commented on HADOOP-16622:
----------------------------------
Hi [[email protected]],
I'm using Singapore endpoint (southeast-1) and 6 out of 10 I can reproduce the
failure in this ticket.
I put a log in S3AFileSystem.innerGetFileStatus and it looks like
* s3 mod time is in second unit
* s3 mod time is rounded, +/- 1 second compared to metastore mod time
Example:
When test passed --
s3modtime=1570939532000; msModTime=1570939532634
When test failed --
s3modtime=1570939675000; msModTime=1570939674909
s3modtime=1570940114000; msModTime=1570940113793
while etags and versionIds from both s3 and metastore remain the same. Do you
think it worths a patch to detect modification using etag & versionId
introduced by HADOOP-16085?
> intermittent failure of ITestCommitOperations: too many s3guard writes
> ----------------------------------------------------------------------
>
> Key: HADOOP-16622
> URL: https://issues.apache.org/jira/browse/HADOOP-16622
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Minor
>
> intermittent failure of ITestCommitOperations; expected 2 s3guard writes, saw
> 7
> the logged commit state shows that only two entries were added, so I'm not
> sure what is up. Proposed: in HADOOP-16207 I will add s3guard.operations log
> to debug so we get a trace of all DDB put/delete calls; this will let us
> debug it when it surfaces again
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
