aqni opened a new issue, #41648:
URL: https://github.com/apache/arrow/issues/41648
### Describe the bug, including details regarding any error messages,
version, and platform.
version: 16.0.0
I am currently working on a project using Apache Arrow in Java and I have
encountered a potential memory leak issue.
run the showMemoryLeak method
```java
@Test
public void showMemoryLeak() {
RootAllocator allocator = new RootAllocator(Long.MAX_VALUE);
ValueVector slice;
try (BufferAllocator child = allocator.newChildAllocator("child", 0,
Long.MAX_VALUE)) {
try (IntVector vector = new IntVector("vector", child)) {
vector.setSafe(0, 1);
vector.setValueCount(1);
TransferPair transferPair = vector.getTransferPair(allocator);
transferPair.splitAndTransfer(0, 1);
slice = transferPair.getTo();
}
}
slice.close();
}
```
then a exception is thrown
```
java.lang.IllegalStateException: Attempting operation on allocator when
allocator is closed.
Allocator(child) 0/0/16384/9223372036854775807 (res/actual/peak/limit)
at
org.apache.arrow.memory.BaseAllocator.assertOpen(BaseAllocator.java:172)
at org.apache.arrow.memory.BufferLedger.decrement(BufferLedger.java:149)
at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:125)
at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:105)
at
org.apache.arrow.vector.BaseValueVector.releaseBuffer(BaseValueVector.java:114)
at
org.apache.arrow.vector.BaseFixedWidthVector.clear(BaseFixedWidthVector.java:247)
at
org.apache.arrow.vector.BaseFixedWidthVector.close(BaseFixedWidthVector.java:238)
at
cn.edu.tsinghua.iginx.parquet.util.arrow.ArrowVectorsTest.showMemoryLeak(ArrowVectorsTest.java:30)
...
```
If I continue to execute transfer() explicitly, there will be no memory
leak, as shown below
```java
@Test
public void showMemoryLeak() {
RootAllocator allocator = new RootAllocator(Long.MAX_VALUE);
ValueVector slice;
try (BufferAllocator child = allocator.newChildAllocator("child", 0,
Long.MAX_VALUE)) {
try (IntVector vector = new IntVector("vector", child)) {
vector.setSafe(0, 1);
vector.setValueCount(1);
TransferPair transferPair = vector.getTransferPair(allocator);
transferPair.splitAndTransfer(0, 1);
try(ValueVector to = transferPair.getTo()) {
TransferPair transferPair2 = to.getTransferPair(allocator);
transferPair2.transfer();
slice = transferPair2.getTo();
}
}
}
slice.close();
}
```
This problem is caused by the fact that when startIndex == 0,
splitAndTransfer() does not transfer the validity buffer.
### Component(s)
Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
