FrankChen021 commented on issue #10579: URL: https://github.com/apache/druid/issues/10579#issuecomment-727147307
Hi @gianm , Current implementation holds value of `druid.processing.buffer.sizeBytes` in integer which means the limit for this configuration is INT.MAX. And the limitation is not mentioned in the doc except the default value. Do you know what's the consideration behind the implementation ? If this is what we expect, I think the doc needs to be updated to describe this limitation. https://github.com/apache/druid/blame/master/docs/configuration/index.md#L1201 > |`druid.processing.buffer.sizeBytes`|This` specifies a buffer size for the storage of intermediate results. The computation engine in both the Historical and Realtime processes will use a scratch buffer of this size to do all of their intermediate computations off-heap. Larger values allow for more aggregations in a single pass over the data while smaller values can require more passes depending on the query that is being executed. Human-readable format is supported, see [here](human-readable-byte.md). |auto (max 1 GiB)| ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
