vinothchandar commented on issue #654: Many small files URL: https://github.com/apache/incubator-hudi/issues/654#issuecomment-487696347 are you programming agains the RDD api? https://hudi.apache.org/configurations.html#limitFileSize controls the size of the parquet file produced and setting this https://hudi.apache.org/configurations.html#compactionSmallFileSize to a value close the limitFileSize (say 100MB) should ensure small file handling kicks in
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
