Yingchun Lai created KUDU-3318:
----------------------------------
Summary: Log Block Container metadata consumed too much disk space
Key: KUDU-3318
URL: https://issues.apache.org/jira/browse/KUDU-3318
Project: Kudu
Issue Type: Improvement
Components: fs
Reporter: Yingchun Lai
In log block container, blocks in .data file are append only, there is a
related append only .metadata file to trace blocks in .data, this type of
entries in metadata are in CREATE type, the other type of entries in metadata
are type of DELETE, it means mark the corresponding CREATE block as deleted.
If there is a pair of CREATE and DELETE entries of a same block id, LBM use
hole punch to reclaim disk space in .data file, but the entries in .metadata
will not be compacted except bootstrap.
Another way to limit metadata is the .data file offset reach its size
limitation(default 10GB), or block number in metadata reach its limitation(no
limit on default).
I found a case in product environment that metadata consumed too many disk
space and near to .data's disk space, it's a waste, and make users confused and
complain that the actual disk space is far more than user's data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
