JingsongLi commented on code in PR #1770:
URL: https://github.com/apache/incubator-paimon/pull/1770#discussion_r1289506969
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/FlinkConnectorOptions.java:
##########
@@ -224,6 +224,18 @@ public class FlinkConnectorOptions {
.withDescription(
"If the new snapshot has not been generated when
the checkpoint starts to trigger, the enumerator will block the checkpoint and
wait for the new snapshot. Set the maximum waiting time to avoid infinite
waiting, if timeout, the checkpoint will fail. Note that it should be set
smaller than the checkpoint timeout.");
+ public static final ConfigOption<Boolean> LOOKUP_ASYNC =
+ ConfigOptions.key("lookup.async")
+ .booleanType()
+ .defaultValue(false)
+ .withDescription("Whether to enable async lookup join.");
+
+ public static final ConfigOption<Integer> LOOKUP_ASYNC_THREAD_NUMBER =
+ ConfigOptions.key("lookup.async-thread-number")
+ .intType()
+ .defaultValue(16)
Review Comment:
So there is a lock to lookup.
This parameter is due to the implementation of Flink. Currently, in the case
of join missing, the Flink implementation will sleep and wait, which will cause
the thread to be blocked. So we need more threads to ensure allow_unordered
availability.
See `RetryableAsyncLookupFunctionDelegator.lookupWithRetry` in Flink.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
