[
https://issues.apache.org/jira/browse/HADOOP-11523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297656#comment-14297656
]
Chris Nauroth commented on HADOOP-11523:
----------------------------------------
Hi Duo. Thank you for the patch.
The current patch would acquire the lease for all {{rename}} operations,
covering both block blobs and page blobs. During initial development of atomic
rename, we were careful to limit the scope to only page blobs (typically used
by HBase logs). Existing applications using block blobs might not be expecting
the leases, so I think we need to make sure the lease acquisition only happens
after checking {{AzureNativeFileSystemStore#isAtomicRenameKey}}.
In the error handling, if an exception is thrown from the try block, and then
another exception is also thrown from freeing the lease, then this would drop
the original exception and throw the exception from freeing the lease. I
suspect in general the main exception from the try block is going to be more
interesting for root cause analysis, so I recommend throwing that one and just
logging the one from freeing the lease, like so:
{code}
} finally {
try {
if (lease != null) {
lease.free();
}
} catch (Exception e) {
LOG.error("Unable to free lease on " + parentKey, e);
}
}
{code}
In addition to the unit tests, I ran the test suite against a live Azure
storage account and confirmed that everything passed.
The release audit warning from Jenkins is unrelated to this patch.
> StorageException complaining " no lease ID" when updating
> FolderLastModifiedTime in WASB
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-11523
> URL: https://issues.apache.org/jira/browse/HADOOP-11523
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
> Affects Versions: 2.2.0, 2.4.0, 2.6.0
> Reporter: Duo Xu
> Assignee: Duo Xu
> Attachments: HADOOP-11523.1.patch
>
>
> In current WASB (Windows Azure Storage - Blob) implementation, when rename
> operation succeeds, WASB will update the parent folder's "last modified time"
> property. By default we do not acquire lease on this folder when updating its
> property and simply pass "null" to it.
> In HBase scenario, when doing distributed log splitting, there might be a
> case that multiple processes from different region servers will access the
> same folder, and randomly we will see this exception in regionserver's log,
> which makes log splitting fail.
> So we should acquire the lease when updating the folder property rather than
> pass "null" to it.
> {code}
> ERROR org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Couldn't rename
> wasb://asvhbase22-2015-01-2808-29...@hwxasvtesting.blob.core.windows.net/hbase/data/default/tdelrowtbl/3c842e8823c192d1028dc72ac3f22886/recovered.edits/0000000000000000015.temp
> to
> wasb://asvhbase22-2015-01-2808-29...@hwxasvtesting.blob.core.windows.net/hbase/data/default/tdelrowtbl/3c842e8823c192d1028dc72ac3f22886/recovered.edits/0000000000000000015
> org.apache.hadoop.fs.azure.AzureException:
> com.microsoft.windowsazure.storage.StorageException: There is currently a
> lease on the blob and no lease ID was specified in the request.
> at
> org.apache.hadoop.fs.azurenative.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2558)
> at
> org.apache.hadoop.fs.azurenative.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2569)
> at
> org.apache.hadoop.fs.azurenative.NativeAzureFileSystem.updateParentFolderLastModifiedTime(NativeAzureFileSystem.java:2016)
> at
> org.apache.hadoop.fs.azurenative.NativeAzureFileSystem.rename(NativeAzureFileSystem.java:1983)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink$2.call(HLogSplitter.java:1161)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink$2.call(HLogSplitter.java:1121)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
