HeartSaVioR commented on pull request #34220: URL: https://github.com/apache/spark/pull/34220#issuecomment-941792480
Sorry for the late. Technically, the result between batch and streaming should be same if the query makes the perfect watermark. We should try to fix if there is a case not respecting this and the operation is supported on both batch and streaming. Batch assumes perfect watermark because all inputs are available when running the query, so it's legitimate to consider withWatermark as no-op in the batch query. And as @zsxwing said, we are trying to make ease of switching the query between batch and streaming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
