ezhou413 commented on code in PR #23483:
URL: https://github.com/apache/kafka/pull/23483#discussion_r4066630823
##########
clients/src/main/java/org/apache/kafka/clients/producer/internals/ChunkedByteBufferOutputStream.java:
##########
@@ -241,11 +262,28 @@ private void releaseUnusedChunks() {
for (ByteBuffer chunk : unused)
pool.deallocate(chunk);
}
+ for (ByteBuffer chunk : unused)
Review Comment:
Yep you're right, updated this section
##########
clients/src/main/java/org/apache/kafka/clients/producer/internals/ChunkedByteBufferOutputStream.java:
##########
@@ -158,8 +161,25 @@ private void advanceWhileCurrentChunkFull() {
*/
private void advanceToNextChunk() {
if (currentChunkIndex + 1 >= chunks.size()) {
- // TODO: KAFKA-20579. With compression support, grow here instead
of throwing.
- throw new IllegalStateException("write exceeded the stream's
remaining chunk capacity");
+ ByteBuffer next = null;
+ try {
+ List<ByteBuffer> chunk = pool.allocateChunks(chunkSize, 0);
+ next = chunk.get(0);
+ } catch (BufferExhaustedException e) {
+ // pool out of memory — fall through to heap
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ // fall through to heap
Review Comment:
As I understand it, `interrupt()` doesn't immediately stop the thread, so we
still are able to get to the else section below (since `next = null`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
