HeartSaVioR edited a comment on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766278704
Also, when they can do arbitrary aggregation in forked process per partition, the aggregation is only bound to the micro-batch, even only bound to the single partition, which is no longer matched the concept of aggregation in SS which should be across the micro-batches. (That said, the output can be different between batch and streaming even if the input is same.) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
