[
https://issues.apache.org/jira/browse/HADOOP-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957642#comment-15957642
]
Mingliang Liu commented on HADOOP-14215:
----------------------------------------
During test, I found a corner case beyond v0 patch: when the table is being
created by another process, it waits it for active w/ v0 patch, and it will try
to verify the table version compatibility immediately. This can happen *before*
the version marker is put by the other process that is creating the table. This
is a race condition. The exception is like:
{code}
java.io.IOException: S3Guard table lacks version marker. Table: mliu-s3guard
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.verifyVersionCompatibility(DynamoDBMetadataStore.java:764)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:727)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:239)
at
org.apache.hadoop.fs.s3a.s3guard.S3Guard.getMetadataStore(S3Guard.java:94)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:282)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3257)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:473)
{code}
So I think v0 patch is not enough to address this race condition.
p.s. I also remember that [~rajesh.balamohan] met the similar problem while he
was testing S3Guard with Hive. That is to say, w/o v0 patch, the problem is
existing (though hidden mostly by not active).
> DynamoDB client should waitForActive on existing tables
> -------------------------------------------------------
>
> Key: HADOOP-14215
> URL: https://issues.apache.org/jira/browse/HADOOP-14215
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: HADOOP-14215-HADOOP-13345.000.patch
>
>
> I saw a case where 2 separate applications tried to use the same
> non-pre-existing table with table.create = true at about the same time. One
> failed with a ResourceInUse exception. If a table does not exist, we attempt
> to create it and then wait for it to enter the active state. If another jumps
> in in the middle of that, the table may exist, thus bypassing our call to
> waitForActive(), and then try to use the table immediately.
> While we're at it, let's also make sure that the race condition where a table
> might get created between checking if it exists and attempting to create it
> is handled gracefully.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
