[
https://issues.apache.org/jira/browse/HBASE-26271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418880#comment-17418880
]
Josh Elser commented on HBASE-26271:
------------------------------------
{quote}In general, I do not think this could solve the 'flush/compaction fails'
problem. In fact, even if we add a list in store file tracker to record these
files, we could still fail before inserting these files into store file tracker
right? But the files are already there and need to be deleted. So FWIW, we
still need to list the filesystem to find them out...
{quote}
Yeah, that's a good point.
I was talking to Szabolcs (and Wellington) earlier today about this too. I
agree with him that letting the files accumulate is likely OK (for one store,
we shouldn't really get close to the HDFS directory limitation, and I assume
cloud storage limits are even bigger).
To avoid creating another pain-point like we have with the HFileCleaner now
(with all of the MOB file cleaner bugs), it would be good to have some
utility/information so that we can clearly see when and what the new cleaner
will do :)
> Cleanup the broken store files under data directory
> ---------------------------------------------------
>
> Key: HBASE-26271
> URL: https://issues.apache.org/jira/browse/HBASE-26271
> Project: HBase
> Issue Type: Sub-task
> Components: HFile
> Reporter: Duo Zhang
> Priority: Major
>
> As for some new store file tracker implementation, we allow flush/compaction
> to write directly to data directory, so if we crash in the middle, there will
> be broken store files left in the data directory.
> We should find a proper way to delete these broken files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
