Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9597#issuecomment-156602165 I think I will not change `offsetRanges `, I will make a simple change so you can see if it is reasonable. Yes, some partitions are empty while others have data is abnormal in Kafka (wrong partitioning scheme). But what about different topics? Also what I'm actually saying is that when all the partitions are empty (no data injected at that time period), is it better not to issue tasks, to let dynamic allocation ramp down resources? Besides dynamic allocation, basically issuing an empty task (we already know it is empty) without doing anything, from my understanding it can be avoided, and have no impact to most of the user cases (you could always find out use cases that beyond normal expectation).
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org