sivabalan narayanan created HUDI-2001:
-----------------------------------------
Summary: NoSuchMethodError ... PartitionedFile in
MergeOnReadSnapshotRelation
Key: HUDI-2001
URL: https://issues.apache.org/jira/browse/HUDI-2001
Project: Apache Hudi
Issue Type: Bug
Reporter: sivabalan narayanan
In latest master, I was trying out hudi test suite job and ran into
NoSuchMethodError. My compiled scala is 2.11 with spark 2.4.4. Run time spark
version is 2.4.7. I tried in EMR.
Stacktrace:
[https://gist.github.com/nsivabalan/9a276c3c0dc8a2ac93310fedccaa2fea]
This happens when ValidateDatasetNode is being used.
```
// read from hudi and remove meta columns.
Dataset<Row> hudiDf = session.read().format("hudi").load(hudiPath);
Dataset<Row> trimmedDf =
hudiDf.drop(HoodieRecord.COMMIT_TIME_METADATA_FIELD).drop(HoodieRecord.COMMIT_SEQNO_METADATA_FIELD).drop(HoodieRecord.RECORD_KEY_METADATA_FIELD)
.drop(HoodieRecord.PARTITION_PATH_METADATA_FIELD).drop(HoodieRecord.FILENAME_METADATA_FIELD);
```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
