jerrypeng commented on PR #56878: URL: https://github.com/apache/spark/pull/56878#issuecomment-4898586973
> intConf for byte-size configs. networkBufferSize and writerMaxMemory are byte-size configs but take raw integer bytes with no range check. Spark's convention for byte sizes is .bytesConf(ByteUnit.BYTE) with a string default (e.g. "32k", "32m"), which is more readable/configurable and validates the value. One caveat: bytesConf yields a Long, whereas BUFFER_SIZE is currently an Int (used for the semaphore permit count and Unpooled.directBuffer(int)), so it's not a pure rename — you'd need to clamp/adjust at the boundary. (networkBufferMaxWaitTimeMs is a duration, so longConf is fine there.) At minimum, add a checkValue(_ > 0) so a zero/negative value can't wedge the allocator and semaphore. Will add checkValue(_ > 0) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
