hudi-bot opened a new issue, #15410: URL: https://github.com/apache/hudi/issues/15410
on each hoodie log append, hdfs used will be increased with the length of the block(512M), not teh actual data length(). Consider in a scenario,I use many writers to append concurrently to a large number of files(bucket file),but each time I append only 10 bytes. dfs used will be increased with the length of the block(512M),this will cause the datanode to report in-sufficient disk space on data write. even though it related to HDFS, We should also have the option to modify the configuration.It helps reduce the rate of increase during the du. ## JIRA info - Link: https://issues.apache.org/jira/browse/HUDI-4794 - Type: Improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
