jackjlli commented on pull request #8240:
URL: https://github.com/apache/pinot/pull/8240#issuecomment-1048333326


   I think what you're trying to do here is to support consistent push for 
append use cases, e.g. for daily data push. If that's the case, the logic 
should handle the following scenarios:
   * For refresh table, it's always not allowed to have two concurrent push 
happening at the same time, regardless of the emptiness of the existing table. 
In this case, the previous `IN_PROGRESS` segment lineage should be cleaned up.
   * For append table, it's okay to have more than one batch push at the same 
time, as long as they're not pushing for the same set of data (i.e. they're not 
overwriting each other).
   
   So going back to the code, I think it'd be good to include the validation 
above instead of just releasing the `forceCleanup` to false. cc: @snleee 
@jtao15 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to