cloud-fan commented on issue #23859: [SPARK-26956][SQL] remove streaming output mode from data source v2 APIs URL: https://github.com/apache/spark/pull/23859#issuecomment-466278082 Sorry I didn't quote the right comment: https://docs.google.com/document/d/1vI26UEuDpVuOjWw4WPoH2T6y8WAekwtI7qoowhOFnI4/edit?disco=AAAACdk332A Let me just copy the comment here: > because we can just implement complete mode with overwrite (or alternatively truncate + append). The append mode is just the normal write append. We can figure out how to support the delta mode later when we add them, since there is no source implementation for it now. > I'd understand this semantic as the sink always receives entire result table at every batch, and then your explanation of complete mode sounds to work differently. What is the difference? The sink always receives entire result table at every batch, and is asked to remove all the old data. That's the expected end-to-end behavior of complete mode.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
