srowen commented on issue #23926: [SPARK-26872][STREAMING] Use a configurable value for final termination in the JobScheduler.stop() method URL: https://github.com/apache/spark/pull/23926#issuecomment-469274306 I think the only change I'd entertain here is some multiple of the batch interval, like 2. I don't think this warrants a whole new config. I don't see why data is not available at the first batch in your example, but, whatever it is I am not sure Spark should accommodate it with a new config.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
