Gatsby-Lee commented on issue #11742: URL: https://github.com/apache/hudi/issues/11742#issuecomment-2302783068
Hi @ankit0811 that's great news that the cost dropped. ( BTW, you need the cleaner at minimum ) The cost is not constant. As you can imagine, there are many factors. In short, I always think this way. If ETL does less work, the cost is less. I don't use clustering since it can add extra complicity. I only use CoW + Hudi optimization ( like lazy listing ). And, this is already fast enough without using the Metadata Table. I've experimented the Metadata Table. And, I think It is NOT always good for everything. If you don't need it, then you don't need to enable it. It can slow down ETL and increase cost. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
