[
https://issues.apache.org/jira/browse/HBASE-17437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510394#comment-16510394
]
Ted Yu commented on HBASE-17437:
--------------------------------
When hbase.wal.dir is configured to be hdfs://mycluster/walontest ,
[~rpednekar] reported seeing the following:
{code}
wal.WALSplitter: This region's directory doesn't exist:
hdfs://mycluster/walontest/data/default/usertable/835ef9135a290a22e530e6d6971f9013.
It is very likely that it was already split so it's safe to discard those
edits.
{code}
It is strange that hbase.wal.dir is used in formulating region dir.
That led me to this change:
{code}
- if (!WALSplitter.splitLogFile(rootdir, fs.getFileStatus(new
Path(rootdir, filename)),
+ if (!WALSplitter.splitLogFile(walDir, fs.getFileStatus(new
Path(walDir, filename)),
{code}
[~zyork]:
How should I interpret the log in the first snippet ?
> Support specifying a WAL directory outside of the root directory
> ----------------------------------------------------------------
>
> Key: HBASE-17437
> URL: https://issues.apache.org/jira/browse/HBASE-17437
> Project: HBase
> Issue Type: Improvement
> Components: Filesystem Integration, wal
> Affects Versions: 1.2.4
> Reporter: Yishan Yang
> Assignee: Zach York
> Priority: Major
> Labels: patch
> Fix For: 1.4.0, 2.0.0
>
> Attachments: HBASE-17437.branch-1.001.patch,
> HBASE-17437.branch-1.002.patch, HBASE-17437.branch-1.003.patch,
> HBASE-17437.branch-1.004.patch, HBASE-17437.master.001.patch,
> HBASE-17437.master.002.patch, HBASE-17437.master.003.patch,
> HBASE-17437.master.004.patch, HBASE-17437.master.005.patch,
> HBASE-17437.master.006.patch, HBASE-17437.master.007.patch,
> HBASE-17437.master.008.patch, HBASE-17437.master.009.patch,
> HBASE-17437.master.010.patch, HBASE-17437.master.011.patch,
> HBASE-17437.master.012.patch, hbase-17437-branch-1.2.patch,
> hbase-17437-master.patch
>
>
> Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some
> FileSystems (such as Amazon S3) don’t support append or consistent writes.
> These two properties are imperative for the WAL in order to avoid loss of
> writes. However, StoreFiles don’t necessarily need the same consistency
> guarantees (since writes are cached locally and if writes fail, they can
> always be replayed from the WAL).
>
> This JIRA aims to allow users to configure a log directory (for WALs) that is
> outside of the root directory or even in a different FileSystem. The default
> value will still put the log directory under the root directory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
