[
https://issues.apache.org/jira/browse/KUDU-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413462#comment-17413462
]
Yingchun Lai commented on KUDU-3318:
------------------------------------
Another way to optimize the situation is to compact metadata at runtime, now it
is only compact at bootstrap.
> Log Block Container metadata consumed too much disk space
> ---------------------------------------------------------
>
> Key: KUDU-3318
> URL: https://issues.apache.org/jira/browse/KUDU-3318
> Project: Kudu
> Issue Type: Improvement
> Components: fs
> Reporter: Yingchun Lai
> Priority: Major
>
> In log block container, blocks in .data file are append only, there is a
> related append only .metadata file to trace blocks in .data, this type of
> entries in metadata are in CREATE type, the other type of entries in metadata
> are type of DELETE, it means mark the corresponding CREATE block as deleted.
> If there is a pair of CREATE and DELETE entries of a same block id, LBM use
> hole punch to reclaim disk space in .data file, but the entries in .metadata
> will not be compacted except bootstrap.
> Another way to limit metadata is the .data file offset reach its size
> limitation(default 10GB), or block number in metadata reach its limitation(no
> limit on default).
> I found a case in product environment that metadata consumed too many disk
> space and near to .data's disk space, it's a waste, and make users confused
> and complain that the actual disk space is far more than user's data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
