cshuo commented on code in PR #12967:
URL: https://github.com/apache/hudi/pull/12967#discussion_r2004672288
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java:
##########
@@ -70,6 +76,32 @@ public static boolean isAppendMode(Configuration conf) {
return isInsertOperation(conf) && ((isCowTable(conf) &&
!conf.getBoolean(FlinkOptions.INSERT_CLUSTER)) || isMorTable(conf));
}
+ /**
+ * Returns whether the RowData append is enabled with given configuration
{@code conf}.
+ * <p>
+ * todo:
+ * <p> support RowData append for COW, see HUDI-9149
+ * <p> support RowData append for operation besides UPSERT, see HUDI-9149
+ * <p> support RowData append for nested complex DataType field, see
HUDI-9146, currently RowData
+ * reader for MOR table is not supported yet, Flink uses Avro parquet reader
to read parquet data
+ * blocks in log file written by RowData writer, there exists some
discrepancies between Avro parquet
+ * support and RowData parquet support regarding nested complex type,
leading to failure during reading.
+ * After the RowData reader is supported, there would be no problems.
+ */
+ public static boolean supportRowDataAppend(Configuration conf, RowType
rowType) {
+ return conf.get(FlinkOptions.INSERT_ROWDATA_MODE_ENABLED)
+ && HoodieTableType.valueOf(conf.get(FlinkOptions.TABLE_TYPE)) ==
HoodieTableType.MERGE_ON_READ
+ &&
(WriteOperationType.valueOf(conf.get(FlinkOptions.OPERATION).toUpperCase()) ==
WriteOperationType.UPSERT
+ ||
WriteOperationType.valueOf(conf.get(FlinkOptions.OPERATION).toUpperCase()) ==
WriteOperationType.DELETE)
+ && !DataTypeUtils.containsNestedComplexType(rowType);
Review Comment:
I already fixed the naming issue in this PR, and it only solves problem of
simple ARRAY type, such as ARRAY of primitive, but not solving problem of
nested complex type, such as ARRAY of ROW.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
