umehrot2 commented on issue #2057: URL: https://github.com/apache/hudi/issues/2057#issuecomment-712454567
> @umehrot2 It looks like for 0.6.0 where this issue is fixed, @WTa-hash is seeing the exception `java.lang.NoSuchMethodError: org.apache.spark.sql.execution.datasources.PartitionedFile.` originating from `at org.apache.hudi.MergeOnReadSnapshotRelation.buildFileIndex(MergeOnReadSnapshotRelation.scala:142) `. Any ideas if this is wrong spark versions ? @n3nash this issue is because of EMR environment. The jar @WTa-hash was building for 0.6.0 is not compiled against EMR's spark version. EMR has its own spark with various modifications. This issue should not be there in emr-5.31.0 where hudi 0.6.0 is officially supported. No need to replace that jar's there. The jar's we provide there are compiled against our own spark and it fixes this issue. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
