nsivabalan commented on code in PR #17601:
URL: https://github.com/apache/hudi/pull/17601#discussion_r2735128403
##########
hudi-common/src/main/java/org/apache/hudi/internal/schema/convert/AvroInternalSchemaConverter.java:
##########
@@ -510,4 +525,68 @@ private static int computeMinBytesForPrecision(int
precision) {
}
return numBytes;
}
+
+ /**
+ * Checks if a logical type is an instance of LocalTimestampMillis using
reflection.
+ * Returns false if the class doesn't exist (e.g., in Avro 1.8.2).
+ */
+ private static boolean isLocalTimestampMillis(LogicalType logicalType) {
+ if (logicalType == null) {
+ return false;
+ }
+ try {
+ Class<?> localTimestampMillisClass =
Class.forName("org.apache.avro.LogicalTypes$LocalTimestampMillis");
+ return localTimestampMillisClass.isInstance(logicalType);
+ } catch (ClassNotFoundException e) {
+ // Class doesn't exist (e.g., Avro 1.8.2)
+ return false;
+ }
+ }
+
+ /**
+ * Checks if a logical type is an instance of LocalTimestampMicros using
reflection.
+ * Returns false if the class doesn't exist (e.g., in Avro 1.8.2).
+ */
+ private static boolean isLocalTimestampMicros(LogicalType logicalType) {
+ if (logicalType == null) {
+ return false;
+ }
+ try {
+ Class<?> localTimestampMicrosClass =
Class.forName("org.apache.avro.LogicalTypes$LocalTimestampMicros");
Review Comment:
same comment as before.
lets move this to a class static block and cache the values.
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -483,6 +483,11 @@ protected boolean shouldReadAsPartitionedTable() {
return (partitionColumns.length > 0 && canParsePartitionValues()) ||
HoodieTableMetadata.isMetadataTable(basePath);
}
+ protected PartitionPath convertToPartitionPath(String partitionPath) {
Review Comment:
not used anywhere
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/postprocessor/ChainedSchemaPostProcessor.java:
##########
@@ -48,6 +48,6 @@ public Schema processSchema(Schema schema) {
for (SchemaPostProcessor processor : processors) {
targetSchema = processor.processSchema(targetSchema);
}
- return targetSchema;
+ return schema;
Review Comment:
this should be targetSchema right
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/ColumnStatsIndexSupport.scala:
##########
@@ -147,6 +147,62 @@ class ColumnStatsIndexSupport(spark: SparkSession,
}
}
+ /**
+ * Loads view of the Column Stats Index in a transposed format where single
row coalesces every columns'
+ * statistics for a single file, returning it as [[DataFrame]]
+ *
+ * Please check out scala-doc of the [[transpose]] method explaining this
view in more details
+ */
+ def loadTransposed[T](targetColumns: Seq[String],
Review Comment:
this is never used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
