loquisgon edited a comment on pull request #12137: URL: https://github.com/apache/druid/pull/12137#issuecomment-1018821742
@JulianJaffePinterest This feature is about replacing time chunks that happen to fall in a given interval with data provided in that interval, re-ingesting with `dropExisting` set to `true`. It is not intended to be used to replace whole tables but I guess that if the interval in the re-ingestion is large enough it will do that. It is also not intended to have transactional semantics for multiple time chunk replacement. The intention is that the feature will integrate nicely with the current transactional semantics where all segments in a given timechunk are replaced all at once but with no guarantees that segments in different time chunks will be replaced atomically (as described [here](https://druid.apache.org/docs/latest/design/segments.html)). So since tombstones are inserted and manipulated most of the time the same as other `DataSegments` they are intended to work the same from that perspective. The other feature that I mentioned previously, the issue was that a race between dropping old segments and loading new segments sometimes caused data unavailability. This new replace implementation using tombstones, since it does not drop segments, rather it replaces segments with new versions for time chunks, does not have that race condition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
