walterddr opened a new issue #8072: URL: https://github.com/apache/pinot/issues/8072
In our current `SegmentDeletionManager`, deleting a segment or all segments in deep store result in a move from active segment file to deleted_ prefix segment files; then `RetentionManager` checks and deletes these files after a RETENTION PERIOD default to 7 days. In cloud object stored backed PinotFS like S3, `move` is essentially a copy-and-delete operation and thus could cause huge overhead during the delete operation if the entire table contains too many segments. The proposal here is to support instant delete without going through RetentionManager (see #8069 as implementation). The question are: 1. whether we want to support instant delete other than a total-table-deletion scenario? 2. should we provide the user interface as a tableConfig, a REST API parameter, or simply a default only when deleting the entire table (without changing the API). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
