cloud-fan commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-620582149
> ... and be questioned if it behaves differently. That's already the case. Spark is not 100% compatible with Hive, and `EXTERNAL` for example is not exactly the same with Hive's. Spark never promises to be 100% Hive compatible, and the `EXTERNAL` keyword has well-defined behavior in Spark. I don't see a problem with keeping it. We can drop `EXTERNAL` completely to make the syntax more clear, but I don't think it worths such a breaking change. The risk of passing `EXTERNAL` through to the catalog is: the behavior becomes not well-defined and is catalog-dependent. This can be a good feature if we want to 100% support Hive SQL. But I'm not sure if there is any plan to do it. Another concern is: It's much easier to add a new feature after the release, than dropping a feature. If we pass `EXTERNAL` through to the catalog, we are not able to remove it later if we decide to. On the other hand, it's always possible to do it in 3.1 for example if many people ask for it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
