Wellington Chevreuil created HBASE-22200:
--------------------------------------------
Summary: WALSplitter.hasRecoveredEdits should use same FS instance
from WAL region dir
Key: HBASE-22200
URL: https://issues.apache.org/jira/browse/HBASE-22200
Project: HBase
Issue Type: Bug
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
*WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region dir
when checking for recovered.edits files, instead of taking FS instance as
additional method parameter. When specifying different file systems for *wal
dir* and *root dir*, *WALSplitter.hasRecoveredEdits* current implementation
will crash or give wrong results. As of now, it's being used indirectly by
*SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and
*root dir* on S3, for example, noticed region split failing with below error:
{noformat}
2019-04-08 13:53:58,064 ERROR
org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
runtime exception: pid=98,
state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true;
SplitTableRegionProcedure table=test-tbl,
parent=4c5db01611e97e3abbe02e781e867212,
daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7,
daughterB=05fa26feaf03ebf9e87e099cbd1eabac
java.lang.IllegalArgumentException: Path
hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits
scheme must be s3a
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
at
org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
at
org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
{noformat}
Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for
the region, we can simply re-use FS instance from the path instance for the WAL
dir region, when searching for recovered.edits.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
