xiangfu0 opened a new pull request, #18784:
URL: https://github.com/apache/pinot/pull/18784
## Summary
- Add server ingestion OOM protection under
`ingestionConfig.streamIngestionConfig.serverIngestionOomProtectionConfig`.
- Apply server-local backpressure to realtime consume loops when heap usage
exceeds the configured threshold; default server policy is enabled for upsert
realtime tables.
- Add server metrics, table config validation, serialization coverage, and
an upsert quickstart sample/manual note.
## User manual / sample config
Server-level defaults are documented in
`pinot-tools/src/main/resources/examples/stream/upsertMeetupRsvp/README.md`,
including:
```properties
pinot.server.instance.ingestion.oom.protection.enabled=true
pinot.server.instance.ingestion.oom.protection.table.selection.mode=UPSERT_ONLY
pinot.server.instance.ingestion.oom.protection.heap.usage.threshold=0.85
pinot.server.instance.ingestion.oom.protection.heap.recovery.threshold=0.75
pinot.server.instance.ingestion.oom.protection.check.interval.ms=1000
```
The upsert meetup realtime table config now includes this table override
sample:
```json
"serverIngestionOomProtectionConfig": {
"mode": "DEFAULT",
"heapUsageThreshold": 0.9,
"heapRecoveryThreshold": 0.8
}
```
## Testing
- `./mvnw -pl pinot-core -am -Dtest=ServerIngestionOomProtectionManagerTest
-Dsurefire.failIfNoSpecifiedTests=false test`
- `./mvnw spotless:apply -pl
pinot-spi,pinot-common,pinot-segment-local,pinot-core,pinot-tools`
- `./mvnw license:format license:check checkstyle:check -pl
pinot-spi,pinot-common,pinot-segment-local,pinot-core,pinot-tools`
- `git diff --check`
- `git diff --cached --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
