n3nash commented on issue #3077: URL: https://github.com/apache/hudi/issues/3077#issuecomment-865422129
@karan867 Can you try setting the following parameters ? 1. https://hudi.apache.org/docs/configurations.html#withIndexType -> Set the index type to SIMPLE. 2. https://hudi.apache.org/docs/configurations.html#compactionSmallFileSize -> Set this config to 0. Both of the above should speed up your ingestion. Hudi attempts to find small files and then pack records into them to ensure that small files are never created. For faster ingestion, this is a trade-off you can take. For more details, you can check this blog -> https://hudi.apache.org/blog/hudi-file-sizing/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
