steveloughran commented on issue #3237:
URL: https://github.com/apache/parquet-java/issues/3237#issuecomment-2944470573
can be seen on master with hadoop 3.3.5+ i.e. when vector IO went in.
full stacks on that branch
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.parquet.hadoop.TestParquetReader
[ERROR] Tests run: 12, Failures: 0, Errors: 12, Skipped: 0, Time elapsed:
1.378 s <<< FAILURE! -- in org.apache.parquet.hadoop.TestParquetReader
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testRangeFiltering[0] --
Time elapsed: 0.123 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
24 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testNoFiltering[0] --
Time elapsed: 0.068 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
80 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testSimpleFiltering[0]
-- Time elapsed: 0.064 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
40 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testCurrentRowIndex[0]
-- Time elapsed: 0.019 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
20 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testRangeFiltering[1] --
Time elapsed: 0.047 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
14 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testNoFiltering[1] --
Time elapsed: 0.044 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
48 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testSimpleFiltering[1]
-- Time elapsed: 0.055 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
28 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testCurrentRowIndex[1]
-- Time elapsed: 0.016 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
12 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testRangeFiltering[2] --
Time elapsed: 0.029 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
4 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testNoFiltering[2] --
Time elapsed: 0.029 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
8 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testSimpleFiltering[2]
-- Time elapsed: 0.019 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
4 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testCurrentRowIndex[2]
-- Time elapsed: 0.011 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
2 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:160)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Set org.apache.parquet.bytes.TrackingByteBufferAllocator.DEBUG = true for more
info
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 20
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 12
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 2
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 80
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 48
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 8
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 24
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 14
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 4
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 40
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 28
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[ERROR] TestParquetReader.closeAllocator:175 » LeakedByteBuffer 4
ByteBuffer object(s) is/are remained unreleased after closing this allocator.
[INFO]
[ERROR] Tests run: 12, Failures: 0, Errors: 12, Skipped: 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
