Reza Safi created SPARK-25233:
---------------------------------
Summary: Give the user the option of specifying a fixed minimum
message per partition per batch when using kafka direct API with backpressure
Key: SPARK-25233
URL: https://issues.apache.org/jira/browse/SPARK-25233
Project: Spark
Issue Type: Improvement
Components: DStreams
Affects Versions: 2.4.0
Reporter: Reza Safi
After SPARK-18371, it is guaranteed that there would be at least *one* message
per partition per batch using direct kafka API with backpressure when new
messages exist in the topics. It would be better if the user has the option of
setting the minimum instead of just a hard coded 1 limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
