jonvex commented on code in PR #13711:
URL: https://github.com/apache/hudi/pull/13711#discussion_r2352890055
##########
hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/BaseSparkInternalRecordContext.java:
##########
@@ -56,17 +68,59 @@ public BaseSparkInternalRecordContext(HoodieTableConfig
tableConfig) {
}
public static Object getFieldValueFromInternalRow(InternalRow row, Schema
recordSchema, String fieldName) {
+ return getFieldValueFromInternalRowInternal(row, recordSchema, fieldName,
false);
+ }
+
+ public static Object getFieldValueFromInternalRowAsJava(InternalRow row,
Schema recordSchema, String fieldName) {
Review Comment:
In the method description it says column stats explicitly. There is
something bad going on here though. The existence of this method in the hoodie
record api:
```
/**
* This method converts a value for a column with certain Avro Logical
data types that require special handling.
* <p>
* E.g., Logical Date Type is converted to actual Date value instead of
Epoch Integer which is how it is
* represented/stored in parquet.
* <p>
* E.g., Decimal Data Type is converted to actual decimal value instead of
bytes/fixed which is how it is
* represented/stored in parquet.
*/
public abstract Object convertColumnValueForLogicalType(
Schema fieldSchema, Object fieldValue, boolean
keepConsistentLogicalTimestamp);
```
shows the lack of consistency and agreed-upon types
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
