yihua opened a new pull request, #7938: URL: https://github.com/apache/hudi/pull/7938
### Change Logs Previously, we found that Spark Datasource read of metadata table was broken and the issue is fixed by #7924. However, the `TestMetadataTableWithSparkDataSource` guarding the exact same functionality did not fail in CI or local mvn command below. After investigation, the Hudi Spark configs (`spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog`, `spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension`) are not properly added to the Spark session in the test environment. This PR sets the proper Hudi Spark configs for Spark Datasource tests and adds one more test on reading metadata table through Spark Datasource. ### Impact After this change, without the fix #7924, the following test fails which is consistent with the behavior of spark-shell (previously it passed without raising the alarm). ``` mvn clean test -Dspark3.3 -Dscala-2.12 -DwildcardSuites="abc" -Dtest=TestMetadataTableWithSparkDataSource -DfailIfNoTests=false -pl hudi-spark-datasource/hudi-spark -am ``` ### Risk level low ### Documentation Update N/A ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
