[
https://issues.apache.org/jira/browse/HBASE-22628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-22628:
---------------------------------
Priority: Critical (was: Blocker)
> Document the custom WAL directory (hbase.wal.dir) usage
> -------------------------------------------------------
>
> Key: HBASE-22628
> URL: https://issues.apache.org/jira/browse/HBASE-22628
> Project: HBase
> Issue Type: Bug
> Components: documentation, wal
> Reporter: Pankaj Kumar
> Assignee: Pankaj Kumar
> Priority: Critical
>
> There is one data loss scenario while migrating to custom WAL directory.
> Steps to reproduce:
> # Setup HBase cluster with the default setting (all WAL files are under the
> root directory ie. /hbase/WALs).
> # Create table 't1' and insert few records
> # Flush meta table (so that table region entries persist in FS)
> # Forcibly kill HBase processes (HM & RS).
> # Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL)
> # Start the HBase servers
> # Scan 't1'
> Ideally HMaster should submit split task of old RS(s) WAL files (created
> under /hbase/WALs) and old data should be replayed.
> But currently, during HM startup we populate the previous dead servers from
> the current WAL dir ( hbase.wal.dir -> /hbaseWAL).
> In MasterFileSystem.getFailedServersFromLogFolders(),
> {code:java}
> Set<ServerName> getFailedServersFromLogFolders() {
> boolean retrySplitting = !conf.getBoolean("hbase.hlog.split.skip.errors",
> WALSplitter.SPLIT_SKIP_ERRORS_DEFAULT);
> Set<ServerName> serverNames = new HashSet<ServerName>();
> Path logsDirPath = new Path(this.walRootDir, HConstants.HREGION_LOGDIR_NAME);
> do {
> if (master.isStopped()) {
> LOG.warn("Master stopped while trying to get failed servers.");
> break;
> }
> try {
> if (!this.walFs.exists(logsDirPath)) return serverNames;
> FileStatus[] logFolders = FSUtils.listStatus(this.walFs, logsDirPath, null);
> {code}
> For backward compatibility we should consider default WAL directory path also.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
