Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14086
Shouldn't truncate function like overwrite in instances where the
distinction doesn't matter -- rather than be ignored?
I think the point of the issue was that DROP/CREATE _isn't_ the best
solution in all cases, specifically, where the table schema has not changed.
It's at best a little slower and at worst destroys metadata.
I believe everyone sees there are use cases for both TRUNCATE and
DROP/CREATE; I'm more specifically asking if we need to make the caller figure
this out or whether it's easy and simple to use TRUNCATE when possible in
Overwrite mode. Maybe it's not simple, or maybe we want to let people control
this anyway, but it does have this cost of adding another setting to the API,
and it's still possible to surprise yourself with Overwrite mode in this
context if you're not aware of what Truncate does differently.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
