Ngone51 commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-845771355
> Let me know if I misunderstand it. So I think the idea is, for example, we go to set a flag at the task set of first micro batch. The flag tells Spark scheduler that we want this task set to be evenly distributed. Is it? Yes. > Actually it sounds hacky to me. But as I'm not used to work in core module, so if you and others working in core frequently agree with it. I'm okay for this as a solution to the first micro-batch issue. It's true it's kind of hacky..but it's a compromise way that allows us to move forward (assuming people still hold the negative willingness towards the plugin API). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
