nsivabalan commented on a change in pull request #2927:
URL: https://github.com/apache/hudi/pull/2927#discussion_r632977033
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkUtils.scala
##########
@@ -90,22 +91,45 @@ object HoodieSparkUtils {
new InMemoryFileIndex(sparkSession, globbedPaths, Map(), Option.empty,
fileStatusCache)
}
- def createRdd(df: DataFrame, structName: String, recordNamespace: String):
RDD[GenericRecord] = {
+ def createRdd(df: DataFrame, structName: String, recordNamespace: String,
mayBeUseLatestTableSchema: Boolean): RDD[GenericRecord] = {
Review comment:
I haven't fixed all callers to set the last arg from config. for eg:
SparkParquetBootstrapDataProvider and SourceFormatAdaptor calls with false as
last arg. not sure if its required and if yes, we can take it as a follow up
and not bloat this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
