rahil-c commented on code in PR #19677:
URL: https://github.com/apache/hudi/pull/19677#discussion_r3834371629
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/config/UnstructuredFileSourceConfig.java:
##########
@@ -122,6 +122,15 @@ public class UnstructuredFileSourceConfig extends
HoodieConfig {
.sinceVersion("1.2.0")
.withDocumentation("Number of characters consecutive chunks overlap
by.");
+ public static final ConfigProperty<Integer> MAX_FILES_PER_BATCH =
ConfigProperty
Review Comment:
I am not sure i understand why we would bound by file count, i would assume
bounding by bytes seems to be safer? Can you explain the pros and cons for each
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
