vinothchandar commented on issue #2498: URL: https://github.com/apache/hudi/issues/2498#issuecomment-776028785
Folks, this is due to version mismatch between aws spark and apache spark. Hudi releases are built against apache spark and aws typically follows up with a EMR release. The problematic access is only on the MOR query, that explains why MOR is problematic, while COW is not. https://dev.to/bytearray/using-your-own-apache-spark-hudi-versions-with-aws-emr-40a0 if interested on steps. If one of you could help verify, that the issue does not exist when querying from Apache Spark, we can route the issue accordingly. cc @umehrot2 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
