Mrxiashu commented on issue #9967:
URL: https://github.com/apache/druid/issues/9967#issuecomment-953593217
> @Mrxiashu你能分享你的摄取配置吗？想到的一种优化是调整`maxRowsInMemory`和`maxBytesInMemory`。使用
Oak，我会建议`maxRowsInMemory=1M`并更新`maxBytesInMemory`您的硬件限制。
Of course,
on-Heap Configuration in kafka-ingest json：
"tuningConfig": {
"type": "kafka",
"maxRowsInMemory": 2000000,
"maxRowsPerSegment": 8000000,
"maxTotalRows": 20000000,
"reportParseExceptions": false
}
oak Configuration in kafka-ingest json：
"tuningConfig": {
"type": "kafka",
"maxRowsInMemory": 2000000,
"maxRowsPerSegment": 8000000,
"maxTotalRows": 20000000,
"reportParseExceptions": false,
"appendableIndexSpec": {
"type": "oak",
"oakMaxMemoryCapacity": 34359738368,
"oakBlockSize": 33554432,
"oakChunkMaxItems": 512
}
Here, I tried to adjust the values of oakBlockSize and oakChunkMaxItems in
the oak tree. After checking the source code, I still cannot understand the
relationship between these parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
