HeartSaVioR edited a comment on pull request #30521: URL: https://github.com/apache/spark/pull/30521#issuecomment-737515816
I'll clarify the initial rationalization of the API `saveAsTable` again: Even we have added DSv2 which supports streaming write, there has been no way to do streaming write by the table name. That is more concerning on data source with supporting catalog - even we have tables in catalog, we couldn't do the streaming write and have to rely on the old way (format & path) which is nonsense, so I had to add the API. That was initially proposed to `writeStreamTo` with DataStreamWriterV2, not something like `saveAsTable`. That said, the API was proposed to cover DSv2 case, specifically for the data source which supports catalog. Focusing on v1 table doesn't match the initial rationalization. If it's going to support creating table, it should cover DSv2 as the first class. Further addition as well. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
