cshuo commented on code in PR #13372:
URL: https://github.com/apache/hudi/pull/13372#discussion_r2113325113
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/FormatUtils.java:
##########
@@ -128,6 +131,7 @@ public static HoodieFileGroupReader<RowData>
createFileGroupReader(
instantRangeOption);
final TypedProperties typedProps =
FlinkClientUtil.getMergedTableAndWriteProps(metaClient.getTableConfig(),
writeConfig);
typedProps.put(HoodieReaderConfig.MERGE_TYPE.key(), mergeType);
+ typedProps.put(HoodieMemoryConfig.MAX_MEMORY_FOR_MERGE.key(),
maxMemoryInBytes);
Review Comment:
> shouldn't it already be include in the writeConfig?
Yes, config `MAX_MEMORY_FOR_MERGE` is already in writeConfig, and is
configured as the value of `FlinkOptions#WRITE_MERGE_MAX_MEMORY`, but Flink
reader before actually use `FlinkOptions#COMPACTION_MAX_MEMORY` as the size of
merging memory (stored in `MergeOnReadInputSplit#maxCompactionMemoryInBytes`),
so this pr is align the memory configuration as before.
> And there are 5 options related with the spillable map:
These configurations are in writeConfig, and are not set explicitly through
flink conf, so value for there configurations are aligned with before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
