AndrewKL commented on issue #26740: [SPARK-30053][SQL] Add the ability for v2 datasource so specify a vacuum action on the table URL: https://github.com/apache/spark/pull/26740#issuecomment-560939075 > * I read through the JIRA issue and see VACUUM is being supported for some systems. But do you have any custom data source which requires this, and if you have one could you please elaborate the plan? Without actual use case I'm not sure it's being accepted. > * We have a custom Datasource where users can "DELETE" records from the table. Internal these records are tomb stoned, instead of actually deleted. This is a common design pattern in many relation table storage formats. https://en.wikipedia.org/wiki/Tombstone_(data_store) For GDPR compliance users would like to be able to force the cleanup process instead of waiting on an automated system to clean things up.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
