[
https://issues.apache.org/jira/browse/HBASE-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207742#comment-16207742
]
Hudson commented on HBASE-14247:
--------------------------------
FAILURE: Integrated in Jenkins build HBase-2.0 #703 (See
[https://builds.apache.org/job/HBase-2.0/703/])
HBASE-14247 Separate the old WALs into different regionserver (zghao: rev
7f1cd12e8c557d23a9b93c704433435bda8094d3)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/FileCleanerDelegate.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestWALEntryStream.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/DumpReplicationQueues.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManagerZkImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationKillSlaveRS.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AsyncFSWALProvider.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationKillMasterRS.java
* (edit)
hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueInfo.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/FSHLogProvider.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/RecoveredReplicationSource.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/WALEntryStream.java
> Separate the old WALs into different regionserver directories
> -------------------------------------------------------------
>
> Key: HBASE-14247
> URL: https://issues.apache.org/jira/browse/HBASE-14247
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: Liu Shaohui
> Assignee: Guanghao Zhang
> Priority: Critical
> Fix For: 3.0.0, 2.0.0-alpha-4
>
> Attachments: HBASE-14247-v001.diff, HBASE-14247-v002.diff,
> HBASE-14247-v003.diff, HBASE-14247.master.001.patch,
> HBASE-14247.master.002.patch, HBASE-14247.master.003.patch,
> HBASE-14247.master.004.patch, HBASE-14247.master.005.patch
>
>
> Currently all old WALs of regionservers are achieved into the single
> directory of oldWALs. In big clusters, because of long TTL of WAL or disabled
> replications, the number of files under oldWALs may reach the
> max-directory-items limit of HDFS, which will make the hbase cluster crashed.
> {quote}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
> The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
> limit=1048576 items=1048576
> {quote}
> A simple solution is to separate the old WALs into different directories
> according to the server name of the WAL.
> Suggestions are welcomed~ Thanks
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
