Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/9597#issuecomment-156602165
I think I will not change `offsetRanges `, I will make a simple change so
you can see if it is reasonable.
Yes, some partitions are empty while others have data is abnormal in Kafka
(wrong partitioning scheme). But what about different topics? Also what I'm
actually saying is that when all the partitions are empty (no data injected at
that time period), is it better not to issue tasks, to let dynamic allocation
ramp down resources?
Besides dynamic allocation, basically issuing an empty task (we already
know it is empty) without doing anything, from my understanding it can be
avoided, and have no impact to most of the user cases (you could always find
out use cases that beyond normal expectation).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
