satishkotha commented on pull request #2196: URL: https://github.com/apache/hudi/pull/2196#issuecomment-722937587
> @satishkotha > A. Thanks so much. This pr need to solved the issue with better approach. > Now I am more clear about overwrite semantic between table.overwrite and spark sql overwrite for hudi. > > B. Also spark sql for hudi overwrite should have the ability just like spark sql 、hive 、 delta lake. > these engine have three mode for overwrite about partition: > > 1. Dynamic Partition : delete all partition data ,and the insert the new data for different > 2. Static partition: just overwrite the partition which is user specified > 3. Mixed partition: mixed of 1 and 2 > more detail in : > https://spark.apache.org/docs/3.0.0-preview/sql-ref-syntax-dml-insert-overwrite-table.html > https://www.programmersought.com/article/47155360487/ > > C. our plan > > 1. Now spark sql for hudi overwrite is Dynamic Partition. I will resolved it in this issue HUDI-1349, first support delete all partition in [HUDI-1350](https://issues.apache.org/jira/browse/HUDI-1350) , then land this issue. (Just like @satishkotha 's Suggestion) > 2. Now spark sql for hudi does not support "Static partition" mode, will then land it in [HUDI-1374](https://issues.apache.org/jira/browse/HUDI-1374) > 3. future support "Mixed partition" mode > > cc @n3nash @vinothchandar please help to review if the plan about spark sql for hudi overwirte is suitable. > It is also possible that my understanding is inappropriate Just fyi, in the [RFC](https://cwiki.apache.org/confluence/display/HUDI/RFC+-+18+Insert+Overwrite+API#RFC18InsertOverwriteAPI-API) we discussed having 'insert_overwrite_table' operation to support dynamic partitioning. static partitioning is supported by 'insert_overwrite'. But, agree with getting approval from Nishith/Vinoth before starting implementation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
