[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595943#comment-15595943
]
Mikhail Antonov commented on HBASE-15385:
-
FYI this commit actually refers to HBASE-15386, bucket cache improvements.
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251073#comment-15251073
]
Hudson commented on HBASE-15385:
ABORTED: Integrated in HBase-1.2 #605 (See
[https://builds.apache.org/job/HBase-1.2/605/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
b2e0008cf34aaf6a41beff85c342b835493908f0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250963#comment-15250963
]
Hudson commented on HBASE-15385:
SUCCESS: Integrated in HBase-1.3 #661 (See
[https://builds.apache.org/job/HBase-1.3/661/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
328c7c4c16201c5e022b6163c6f15b08a0971ea1)
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250872#comment-15250872
]
Hudson commented on HBASE-15385:
SUCCESS: Integrated in HBase-1.4 #104 (See
[https://builds.apache.org/job/HBase-1.4/104/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
a331a57ef82b97995e50f917b678063aea941e22)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250591#comment-15250591
]
Hudson commented on HBASE-15385:
SUCCESS: Integrated in HBase-1.2-IT #486 (See
[https://builds.apache.org/job/HBase-1.2-IT/486/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
b2e0008cf34aaf6a41beff85c342b835493908f0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250564#comment-15250564
]
Hudson commented on HBASE-15385:
FAILURE: Integrated in HBase-Trunk_matrix #858 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/858/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
fa215a67e20da8c1a450b16db27c73ee3f9d02c0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
* src/main/asciidoc/_chapters/performance.adoc
*
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250506#comment-15250506
]
Hudson commented on HBASE-15385:
SUCCESS: Integrated in HBase-1.3-IT #625 (See
[https://builds.apache.org/job/HBase-1.3-IT/625/])
HBASE-15385 PREFETCH_BLOCKS_ON_OPEN in HColumnDescriptor is ignored (stack: rev
328c7c4c16201c5e022b6163c6f15b08a0971ea1)
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestPrefetch.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177837#comment-15177837
]
Liu Shaohui commented on HBASE-15385:
-
Sorry. It is a issue for Hadoop. Close it~
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
> Fix For: 3.0.0
>
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15385) A failed atomic folder rename operation can never recovery for the destination file is deleted in Wasb filesystem
[
https://issues.apache.org/jira/browse/HBASE-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177822#comment-15177822
]
Liu Shaohui commented on HBASE-15385:
-
[~cnauroth]
> A failed atomic folder rename operation can never recovery for the
> destination file is deleted in Wasb filesystem
> -
>
> Key: HBASE-15385
> URL: https://issues.apache.org/jira/browse/HBASE-15385
> Project: HBase
> Issue Type: Bug
> Components: hadoop-azure
>Reporter: Liu Shaohui
>Priority: Critical
> Fix For: 3.0.0
>
>
> When using Wsab file system, we found that a failed atomic folder rename
> operation can never recovery for the destination file deleted in Wasb
> filesystem.
> {quota}
> ls: Attempting to complete rename of file
> hbase/azurtst-xiaomi/data/default/YCSBTest/.tabledesc during folder rename
> redo, and file was not found in source or destination.
> {quote}
> The reason is the the file is renamed to the destination file before the
> crash, and the destination file is deleted by another process after crash. So
> the recovery is blocked during finishing the rename operation of this file
> when found the source and destination files all don't exist.
> See: NativeAzureFileSystem.java #finishSingleFileRename
> Another serious problem is that the recovery of atomic rename operation may
> delete new created file which is same name as the source file, because the
> file system don't check if there are rename operation need be redo.
> Suggestions are welcomed~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
