cbalci opened a new issue, #12424: URL: https://github.com/apache/pinot/issues/12424
Proposing a somewhat cosmetic change and some related small features. Pinot already has a mechanism for keeping 'deleted' segments around for a configurable period under `<deep-store>/Deleted_segments` directory. I'm proposing to rename this feature from **Deleted_Segments** to **Archived_Segments** and provide more configuration options: - Retention period (Already exists) - Archival location (not hard coded to '<dir>/Deleted_segments') - Archival format (Capability to use different storage formats, e.g. psf, parquet, etc.) - ...(Open to more ideas) This would make the archival feature more useful for recovery purposes. Users can specify external locations with different durability and quota properties. With customized archival formats, users can optimize based on their needs, data size, query-ability etc. I'm willing to get the implementation going if I can get some alignment on the idea. Please let me know what you think. cc @chenboat @ankitsultana @Jackie-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
