[
https://issues.apache.org/jira/browse/HADOOP-11523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297697#comment-14297697
]
Duo Xu commented on HADOOP-11523:
---------------------------------
Hi, [~cnauroth]
Current HBase clusters in Azure, our configuration is
{code}
"Name": "fs.azure.page.blob.dir", "Value": "\/hbase\/WALs,\/hbase\/oldWALs"
{code}
In the exception above, WASB tried to update folder/blob
"hbase/data/default/tdelrowtbl/3c842e8823c192d1028dc72ac3f22886/recovered.edits".
This is apparently not a page blob. So lease acquisition is needed in block
blob, at least for HBase.
For the error handling, agree and I have attached a new patch.
> StorageException complaining " no lease ID" when updating
> FolderLastModifiedTime in WASB
> ----------------------------------------------------------------------------------------
>
> Key: HADOOP-11523
> URL: https://issues.apache.org/jira/browse/HADOOP-11523
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools
> Affects Versions: 2.2.0, 2.4.0, 2.6.0
> Reporter: Duo Xu
> Assignee: Duo Xu
> Attachments: HADOOP-11523.1.patch, HADOOP-11523.2.patch
>
>
> In current WASB (Windows Azure Storage - Blob) implementation, when rename
> operation succeeds, WASB will update the parent folder's "last modified time"
> property. By default we do not acquire lease on this folder when updating its
> property and simply pass "null" to it.
> In HBase scenario, when doing distributed log splitting, there might be a
> case that multiple processes from different region servers will access the
> same folder, and randomly we will see this exception in regionserver's log,
> which makes log splitting fail.
> So we should acquire the lease when updating the folder property rather than
> pass "null" to it.
> {code}
> ERROR org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Couldn't rename
> wasb://asvhbase22-2015-01-2808-29...@hwxasvtesting.blob.core.windows.net/hbase/data/default/tdelrowtbl/3c842e8823c192d1028dc72ac3f22886/recovered.edits/0000000000000000015.temp
> to
> wasb://asvhbase22-2015-01-2808-29...@hwxasvtesting.blob.core.windows.net/hbase/data/default/tdelrowtbl/3c842e8823c192d1028dc72ac3f22886/recovered.edits/0000000000000000015
> org.apache.hadoop.fs.azure.AzureException:
> com.microsoft.windowsazure.storage.StorageException: There is currently a
> lease on the blob and no lease ID was specified in the request.
> at
> org.apache.hadoop.fs.azurenative.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2558)
> at
> org.apache.hadoop.fs.azurenative.AzureNativeFileSystemStore.updateFolderLastModifiedTime(AzureNativeFileSystemStore.java:2569)
> at
> org.apache.hadoop.fs.azurenative.NativeAzureFileSystem.updateParentFolderLastModifiedTime(NativeAzureFileSystem.java:2016)
> at
> org.apache.hadoop.fs.azurenative.NativeAzureFileSystem.rename(NativeAzureFileSystem.java:1983)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink$2.call(HLogSplitter.java:1161)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink$2.call(HLogSplitter.java:1121)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
