[
https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976320#comment-16976320
]
Pratyaksh Sharma commented on HUDI-340:
---------------------------------------
My bad, I forgot to take into account that default value for sourceLimit is
Long.MAX_VALUE.
Ok, what you are saying makes sense, I will make upper cap configurable with
default value of 10M and whenever sourceLimit is configured to be less than
Long.MAX_VALUE, we respect sourceLimit otherwise upper cap is used. :)
> Increase Default max events to read from kafka source
> -----------------------------------------------------
>
> Key: HUDI-340
> URL: https://issues.apache.org/jira/browse/HUDI-340
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: deltastreamer
> Reporter: Pratyaksh Sharma
> Assignee: Pratyaksh Sharma
> Priority: Major
>
> Right now, DEFAULT_MAX_EVENTS_TO_READ is set to 1M in case of kafka source in
> KafkaOffsetGen.java class. DeltaStreamer can handle much more incoming
> records than this.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
