zsxwing commented on pull request #30521: URL: https://github.com/apache/spark/pull/30521#issuecomment-738150124
> Unless we get consensus about this or the PR reflects the comment, please don't consider this as ready to merge. @HeartSaVioR could you please make it clear what we should to get consensus? If I understand correctly, we were discussing whether creating the table if not exist is a good behavior for users or not. For this, I think Kafka (this was raised by you actually) and FileStreamSink are two supportive data points that creating the table if not exist is a good behavior. At least, it shows more users would like Spark to create tables for them. Of cause, it's obvious that it's impossible to make a default behavior that all users would be happen. We need to make a tradeoff between: - More users will notice nothing as they would like to create the table if not exist. Less users will find Spark create a table that doesn't meet their requirements, and need to delete the table and re-create it manually. - More users will see an error, and have to update the codes to create the table first. And they may question why the behavior is different when using a non exist path. Less users that always create tables before the query will be happy. IMO, the first one is better as more users can get benefit from this behavior. Do you agree that this is a better default behavior? > especially providing objection of the base assumption `table is a view of path` which leads the result that stream query should create table by default. Regarding this, this is not an assumption. Sorry, I was not clear. I meant `in most existing of cases, table can be viewed as an alias of a path`. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
