yihua commented on code in PR #10073:
URL: https://github.com/apache/hudi/pull/10073#discussion_r1403778359
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/config/CloudSourceConfig.java:
##########
@@ -137,4 +147,12 @@ public class CloudSourceConfig extends HoodieConfig {
.sinceVersion("0.14.1")
.withDocumentation("specify this value in bytes, to coalesce partitions
of source dataset not greater than specified limit");
+ public static final ConfigProperty<Integer> MAX_FETCH_TIME_PER_SYNC_MS =
ConfigProperty
+ .key(STREAMER_CONFIG_PREFIX +
"source.cloud.meta.max.fetch.time.per.sync.ms")
+ .defaultValue(1)
Review Comment:
Is 1 millisecond too small as the default value?
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/config/CloudSourceConfig.java:
##########
@@ -53,7 +53,17 @@ public class CloudSourceConfig extends HoodieConfig {
.defaultValue(10)
.withAlternatives(DELTA_STREAMER_CONFIG_PREFIX +
"source.cloud.meta.batch.size")
.markAdvanced()
- .withDocumentation("Number of metadata messages to pull at a time");
+ .withDocumentation("Number of metadata messages to pull in one API call
to the cloud events queue. "
+ + "Multiple API calls with this batch size are sent to cloud events
queue, until we consume
hoodie.streamer.source.cloud.meta.max.num.messages.per.sync"
Review Comment:
```suggestion
+ "Multiple API calls with this batch size are sent to cloud
events queue, until we consume
hoodie.streamer.source.cloud.meta.max.num.messages.per.sync "
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
