nsivabalan commented on issue #6866: URL: https://github.com/apache/hudi/issues/6866#issuecomment-1283466393
and btw, general advise on your partitioning strategy. Try to keep the cardinality of total partitions to 10k or less. If not, you might have lot of bottlenecks around it. I see you are doing hour based partitioning. Even for 1 year of data, it comes to ~9k partitions. So, would be good to think of a diff strategy or use day based strategy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
