anand-h-codes commented on PR #3679: URL: https://github.com/apache/storm/pull/3679#issuecomment-2334574436
@reiabreu I understand your concern. The main problem with the existing method is, even though if we set `max.poll.records`, we can't guarantee that one trident batch will have not more than `max.poll.records` number of tuples. It will only guarantee that it is true per partition. So currently one trident batch can have a maximum of `#of partitions assigned * max.poll.records` number of tuple. With the changes in the PR it is possible to guarantee that one batch will not contain more than `max.poll.records`. Say if I have 1000 partitions assigned to my KafkaTridentSpout and my topology is optimised to process upto 500 records per batch and not more. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
