deniskuzZ commented on PR #6924:
URL: https://github.com/apache/hive/pull/6924#issuecomment-6918556145

   > [@deniskuzZ](https://github.com/deniskuzZ) I have two questions regarding 
this point:
   > 
   > > MERGE with UPDATE and DELETE clauses on V3: one delete branch 
(SplitMergeRewriter ORs the clauses' conditions); the update's insert branch is 
not counted, rows affected unchanged
   > 
   > 1. Can merging into native ACID and Iceberg v2 tables with both delete and 
delete of update branches work the same way?
   >    The delete branches (one for update and one for delete) never overlap. 
The combined branch could have a predicate like A OR B, so it would delete the 
exact same rows as the two separate branches. On disk, native ACID would write 
one delete delta per statement instead of two, and Iceberg v2 would write one 
set of position-delete files instead of two. This would mean fewer small files 
and a less complex `SplitMergeRewriter`, since only the single-delete-branch 
implementation would be needed.
   > 2. If `1.` is a no-go, can the special code for handling update-deletes 
and delete-deletes in the same branch be extracted into another 
`SplitMergeRewriter`?
   >    The decision about which one to instantiate could be made in 
`MergeRewriterFactory`:
   > 
   > ```
   >   } else if (splitUpdate) {
   >     if (hasDistributeKeys(table)) {  // storageHandler != null && 
!acidDistributeColumns(table, DELETE).isEmpty()
   >         return new SingleDeleteBranchMergeRewriter(db, conf, 
sqlGeneratorFactory)
   >     } else {
   >         return new SplitMergeRewriter(db, conf, sqlGeneratorFactory);
   >     }
   >   }
   > ```
   
   Thanks!                                                                      
                                                                                
                                                  
     1. Yes, it would work for native ACID and V2 too: the update's delete 
predicate and the delete clause's predicate never overlap, so a single branch 
with (A) OR (B) deletes the same rows, and writes one  delete delta (or one set 
of position delete files) instead of two. I kept this PR to V3, where one 
delete branch is needed for correctness (one DV per data file), because the 
same change rewrites the plan of every native ACID and Iceberg V2 MERGE that 
has both clauses (`hive.split.update` is on by default). 
    I'd propose doing it in a follow-up: make the single delete branch the 
default for every table, which also drops the V3-only condition from 
SplitMergeRewriter. I can open the JIRA if you agree.                           
                                                                                
              
     2. With that follow-up there would be one implementation, so a second 
rewriter wouldn't be needed.             


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to