LuciferYang opened a new issue, #9522: URL: https://github.com/apache/paimon/issues/9522
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, `9c7deebbd` (2.1-SNAPSHOT) ### Compute Engine Java API (`paimon-common`). `MemorySliceOutput` backs the SST block writer, the sorted-index footers and the global-index key serializers. ### Minimal reproduce step `MemorySliceOutput.ensureSize` grows the buffer by doubling, starting from the current segment size: ```java int newCapacity = segment.size(); int minNewCapacity = segment.size() + minWritableBytes; while (newCapacity < minNewCapacity) { newCapacity <<= 1; } ``` Doubling never leaves zero, so with an empty segment the loop condition stays true forever: ```java MemorySliceOutput out = new MemorySliceOutput(0); out.writeByte(5); // spins on the CPU and never returns ``` ### What doesn't meet your expectations? The first write should either allocate a buffer or fail. Instead the thread burns a core with no exception, no log line and no progress, which in a Flink or Spark task looks like a hang rather than a bug: the stack trace shows a live thread inside `ensureSize` and nothing else. Every construction site in the repository passes a positive size (the smallest is 2, in the global-index key serializer), so no current Paimon code path reaches this. `MemorySliceOutput` is public in paimon-common and takes the capacity from its caller, so a size computed at runtime that happens to be zero is enough. ### Anything else? Two related things I found while checking this, both older than the zero case and neither fixed here. `newCapacity <<= 1` overflows for a buffer above 2^30: the value walks through negative into 0 and the same loop hangs again. And when `segment.size() + minWritableBytes` itself overflows, the loop is skipped entirely, `ensureSize` allocates a same-sized array and returns as though it had grown, after which the caller runs `MemorySegment.put`, which is an unchecked `UNSAFE.copyMemory`. `HeapBytesVector.calculateNewBytesCapacity` in the same module already shows the shape that avoids both, computing in `long` and clamping against a maximum array size. `AbstractHeapVector.reserveDictionaryIds` has the identical doubling-from-zero loop, reachable if a zero-row batch is followed by a non-empty one on the same reused vector. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
