yuzhaojing commented on PR #12642: URL: https://github.com/apache/hudi/pull/12642#issuecomment-2646983743
> One consensus that needs to be discussed is: Should the relation that hudi provides to spark be variable? The current relation is immutable. In the same spark context, once the relation is created, spark will reuse the relation in some cases. For example, the same query sql will cache the corresponding execution plan. Then the same relation will always be reused, and the query result will always be the same, even if there are data changes during these queries. @yuzhaojing @danny0405 @yihua Excellent summary. I believe that the Hudi relation should be mutable to support the Spark engine's reuse of the relation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
