prashantwason commented on code in PR #19914:
URL: https://github.com/apache/hudi/pull/19914#discussion_r4210904622
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieFileGroupReaderBasedFileFormat.scala:
##########
@@ -366,6 +387,12 @@ class HoodieFileGroupReaderBasedFileFormat(tablePath:
String,
.getSparkPartitionedFileUtils.getPathFromPartitionedFile(file))
fileSliceMapping.getSlice(fileGroupName) match {
case Some(fileSlice) if !isCount && (requiredSchema.nonEmpty ||
fileSlice.getLogFiles.findAny().isPresent) =>
+ if (!fileSlice.getLogFiles.findAny().isPresent && !isBootstrap
&& broadcastStockReader.isDefined) {
Review Comment:
Done in b7a2388e7b94. The stock `ParquetFileFormat` bypass is gone; a COW
file slice without log files is now routed to the existing `readBaseFile(file,
state, conf)` path with the vectorized Hudi `baseFileReader`, so
schema-on-read, type promotion, mandatory partition fields, vector columns and
ORC/multi-format tables go through the Hudi reader as before. I'll re-run the
benchmark through this path and update the description with the numbers, and
only look into the Hudi reader if a material gap remains.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
