[
https://issues.apache.org/jira/browse/HADOOP-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856647#comment-16856647
]
Steve Loughran commented on HADOOP-16349:
-----------------------------------------
Jstack
{code}
"main" #1 prio=5 os_prio=31 tid=0x00007fcb4e000000 nid=0x1f03 waiting on
condition [0x000070000e11f000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.getVersionMarkerItem(DynamoDBMetadataStore.java:1319)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:1255)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:383)
at
org.apache.hadoop.fs.s3a.s3guard.S3Guard.getMetadataStore(S3Guard.java:102)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:383)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3338)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:136)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3387)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3355)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:497)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:352)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:250)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:233)
at
org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:104)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
{code}
> DynamoDBMetadataStore.getVersionMarkerItem() to log at info/warn on retry
> -------------------------------------------------------------------------
>
> Key: HADOOP-16349
> URL: https://issues.apache.org/jira/browse/HADOOP-16349
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> If you delete the version marker from a S3Guard table, it appears to hang for
> 5 minutes.
> Only if you restart and turn logging to debug do you see that
> {{DynamoDBMetadataStore.getVersionMarkerItem()}} is sleeping and retrying.
> # log at warn
> # add entry to troubleshooting doc on the topic
> The cause of the failure can be any of
> * table being inited elsewhere: expectation, fast recovery
> * it's not a S3Guard table: it won't recover
> * it's a S3Guard table without a version marker: it won't recover.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
