the-other-tim-brown commented on code in PR #17904:
URL: https://github.com/apache/hudi/pull/17904#discussion_r2746747162
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieParquetFileFormatHelper.scala:
##########
@@ -174,24 +175,55 @@ object HoodieParquetFileFormatHelper {
}
}
- if (typeChangeInfos.isEmpty) {
- GenerateUnsafeProjection.generate(fullSchema, fullSchema)
+ def generateProjectionWithPadding(): UnsafeProjection = {
Review Comment:
The logic can exist outside of the `SparkLanceReaderBase` since it is more
of a spark row transformation than a specific Lance transformation. The idea is
to keep them separated so the code is more modular and easily testable.
##########
hudi-common/src/test/java/org/apache/hudi/common/table/read/TestHoodieFileGroupReaderBase.java:
##########
@@ -111,7 +113,7 @@
* Tests {@link HoodieFileGroupReader} with different engines
*/
public abstract class TestHoodieFileGroupReaderBase<T> {
- private static final List<HoodieFileFormat> DEFAULT_SUPPORTED_FILE_FORMATS =
Arrays.asList(HoodieFileFormat.PARQUET, HoodieFileFormat.ORC);
+ private static final List<HoodieFileFormat> DEFAULT_SUPPORTED_FILE_FORMATS =
Arrays.asList(HoodieFileFormat.PARQUET, HoodieFileFormat.LANCE);
Review Comment:
Yes, that's right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
