[
https://issues.apache.org/jira/browse/HBASE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-10378:
------------------------------------
Pardon if I'm implicating the wrong issue.
Working on a Macbook dev box.
I have found that some TestHRegion unit tests fail now because the can't get a
WAL writer:
{noformat}
$ mvn -DskipTests clean install
$ mvn test -Dtest=TestHRegion
Tests in error:
TestHRegion.testDelete_multiDeleteColumn:1836->initHRegion:4883->initHRegion:4904->initHRegion:4923
» IOException cannot get log writer
TestHRegion.testRegionReplicaSecondary:4587 » IOException cannot get log
writer
TestHRegion.testScanner_DeleteOneFamilyNotAnother:2096->initHRegion:4883->initHRegion:4904->initHRegion:4923
» IOException cannot get log writer
TestHRegion.testScanner_JoinedScannersWithLimits:3156->initHRegion:4883->initHRegion:4904->initHRegion:4923
» IOException cannot get log writer
{noformat}
For example:
{noformat}
testRegionReplicaSecondary(org.apache.hadoop.hbase.regionserver.TestHRegion)
Time elapsed: 0.063 sec <<< ERROR!
java.io.IOException: cannot get log writer
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:774)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:646)
at
org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:293)
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:451)
at
org.apache.hadoop.fs.ChecksumFileSystem.createNonRecursive(ChecksumFileSystem.java:469)
at
org.apache.hadoop.fs.FileSystem.createNonRecursive(FileSystem.java:1086)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.init(ProtobufLogWriter.java:90)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.createWriter(DefaultWALProvider.java:361)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.createWriterInstance(FSHLog.java:736)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:701)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:612)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.<init>(FSHLog.java:549)
at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)
at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:139)
at org.apache.hadoop.hbase.wal.WALFactory.<init>(WALFactory.java:171)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:4566)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:4533)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:4508)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:4585)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:4469)
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testRegionReplicaSecondary(TestHRegion.java:4587)
{noformat}
Then, I can't clean:
{noformat}
$ mvn clean
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on
project hbase-server: Failed to clean project: Failed to delete
/Users/apurtell/src/hbase/hbase-server/target/test-data/c8c12cda-613a-488d-9a9c-2463c8f884d5/WALs
-> [Help 1]
{noformat}
I need to sudo rm -rf .
It wasn't like this last week.
> Divide HLog interface into User and Implementor specific interfaces
> -------------------------------------------------------------------
>
> Key: HBASE-10378
> URL: https://issues.apache.org/jira/browse/HBASE-10378
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Himanshu Vashishtha
> Assignee: Sean Busbey
> Fix For: 2.0.0
>
> Attachments: 10378-1.patch, 10378-2.patch, HBASE-10378.3.patch.txt,
> HBASE-10378.4.patch.txt, HBASE-10378.5.patch.txt
>
>
> HBASE-5937 introduces the HLog interface as a first step to support multiple
> WAL implementations. This interface is a good start, but has some
> limitations/drawbacks in its current state, such as:
> 1) There is no clear distinction b/w User and Implementor APIs, and it
> provides APIs both for WAL users (append, sync, etc) and also WAL
> implementors (Reader/Writer interfaces, etc). There are APIs which are very
> much implementation specific (getFileNum, etc) and a user such as a
> RegionServer shouldn't know about it.
> 2) There are about 14 methods in FSHLog which are not present in HLog
> interface but are used at several places in the unit test code. These tests
> typecast HLog to FSHLog, which makes it very difficult to test multiple WAL
> implementations without doing some ugly checks.
> I'd like to propose some changes in HLog interface that would ease the multi
> WAL story:
> 1) Have two interfaces WAL and WALService. WAL provides APIs for
> implementors. WALService provides APIs for users (such as RegionServer).
> 2) A skeleton implementation of the above two interface as the base class for
> other WAL implementations (AbstractWAL). It provides required fields for all
> subclasses (fs, conf, log dir, etc). Make a minimal set of test only methods
> and add this set in AbstractWAL.
> 3) HLogFactory returns a WALService reference when creating a WAL instance;
> if a user need to access impl specific APIs (there are unit tests which get
> WAL from a HRegionServer and then call impl specific APIs), use AbstractWAL
> type casting,
> 4) Make TestHLog abstract and let all implementors provide their respective
> test class which extends TestHLog (TestFSHLog, for example).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
