cloud-fan commented on issue #23208: [SPARK-25530][SQL] data source v2 API refactor (batch write) URL: https://github.com/apache/spark/pull/23208#issuecomment-450009721 Hi @rdblue , thanks for looking at it! > deviations between the read and write side structures should be thought through and justified in the doc's text It has been added to the doc, please take another look when you have time. > Same with differences between the batch write and streaming write Isn't it the queryId discussion? queryId has been there for years and I think it's reasonable to make the design assuming it's still there. If there is a proposal to remove queryId, it should be done in parallel and we can adjust the v2 API accordingly after we decide to remove. Technically we should not block a PR just because some specific persons haven't reviewed it. I'd appreciate it if you give some suggestions to the doc/PR and I'll update them ASAP. But if you are busy, shall we let other people review/merge and move forward? We can always have followup PRs to address new comments, and even revert it if we find serious problems later on.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
