[
https://issues.apache.org/jira/browse/HBASE-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177017#comment-16177017
]
Dave Latham commented on HBASE-14247:
-------------------------------------
Yes, that would address my concern, at least for our own clusters. I don't
think it's a great way to address the issue, though. People with large
clusters that use it would be the ones most likely to be hit by HBASE-9208. It
also doesn't seem great for HBase to need to support two different directory
layouts for WAL files. I still think it would be better to update the log
cleaner to handle multiple directories well, and then turn it on for everyone.
> Separate the old WALs into different regionserver directories
> -------------------------------------------------------------
>
> Key: HBASE-14247
> URL: https://issues.apache.org/jira/browse/HBASE-14247
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: Liu Shaohui
> Assignee: Guanghao Zhang
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-14247.master.001.patch,
> HBASE-14247.master.002.patch, HBASE-14247-v001.diff, HBASE-14247-v002.diff,
> HBASE-14247-v003.diff
>
>
> Currently all old WALs of regionservers are achieved into the single
> directory of oldWALs. In big clusters, because of long TTL of WAL or disabled
> replications, the number of files under oldWALs may reach the
> max-directory-items limit of HDFS, which will make the hbase cluster crashed.
> {quote}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
> The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
> limit=1048576 items=1048576
> {quote}
> A simple solution is to separate the old WALs into different directories
> according to the server name of the WAL.
> Suggestions are welcomed~ Thanks
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
