noahprince22 opened a new pull request #6250: URL: https://github.com/apache/incubator-pinot/pull/6250
## Description Addresses https://github.com/apache/incubator-pinot/issues/6187 New configurations on the server include: Option | Default | Description ------------------------------ pinot.server.instance.lazyLoadSegments | false | Enables segment lazy loading from deep storage. This will keep segments from being stored on the local file system until they are needed for a query. pinot.server.instance.maxSegmentDiscUsageMb | Integer.MAX_VALUE | Sets a maximum bound before cached segments should be cleaned up. This should be set below the maximum disc space on the particular server. pinot.server.instance.purgeOnStart | false | After the server has crashed, there's no guarantee the cached segments are up to date. Rather than refresh them all, you can optionally just purge all existing segments from the local disc when the server starts. New configurations from the broker include: Option | Default | Description ----------------------------- pinot.broker.query.segment.limit | Integer.MAX_VALUE | Because we are lazy loading segment, materializing a large number of unpruned segments can cause massive resource usage and disc pressure. This acts as a safeguard against `select *` queries with no `where` clause. ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [*] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes) Does this PR fix a zero-downtime upgrade introduced earlier? * [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes) This PR will require release notes and updates to the pinot docs gitbook which appears to be a separate repo. * [*] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes) ## Release Notes See above configuration options, and below documentation. ## Documentation TODO ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
