pengxianzi commented on issue #12589: URL: https://github.com/apache/hudi/issues/12589#issuecomment-2574568813
> @pengxianzi How may records are there in the Kudu table, can you switch to mor table instead, mor is more suitable for large table. > > And if possible, can you upgrade to 0.14.1 or 0.15.0 which fix some stability bugs. Thank you for your response! Here is our further feedback: Regarding Upgrading Hudi Version: We are currently unable to upgrade the Hudi version to resolve the issue, as our production environment relies on version 0.12.0. Regarding Data Volume: The data volume migrated from Kudu to the Hudi COW bucketed table is typically in the tens of millions. We tried increasing the number of buckets and parallelism, and kept the file size within each bucket below 100MB, but the write still fails. Regarding Switching to MOR Table: We also have a MOR table with hundreds of millions of records, but it encounters similar issues. Therefore, the problem does not seem to be related to the table type (COW or MOR). Regarding the Impact of Kudu Table: We believe this issue is not related to the Kudu table, as we have smaller tables that can be written normally. The problem seems to primarily occur with large tables. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
