zhaoyangming created HUDI-4794:
----------------------------------
Summary: add an option of the log file block size
Key: HUDI-4794
URL: https://issues.apache.org/jira/browse/HUDI-4794
Project: Apache Hudi
Issue Type: Improvement
Reporter: zhaoyangming
on each hoodie log append, hdfs used will be increased with the length of the
block(512M), not teh actual data length().
Consider in a scenario,I use many writers to append concurrently to a large
number of files(bucket file),but each time I append only 10 bytes.
dfs used will be increased with the length of the block(512M),this will cause
the datanode to report in-sufficient disk space on data write.
even though it related to HDFS, We should also have the option to modify the
configuration.It helps reduce the rate of increase during the du.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
