steveloughran opened a new issue, #3237:
URL: https://github.com/apache/parquet-java/issues/3237
### Describe the bug, including details regarding any error messages,
version, and platform.
this surfaces when running `TestParquetReader` with hadoop >= 3.4.0
```
mvn -Dtest=TestParquetReader -Dhadoop.version=3.4.1
```
This is disappointing as it means
1. the hadoop release process hasn't been doing the parquet build and test
(it's an optional action)
2. this got through our own large scale QE testing.
The good news: this hasn't surfaced in production, presumably because
nothing is trying to read into direct buffers -something parquet blocks
automatically.
```
[ERROR] org.apache.parquet.hadoop.TestParquetReader.testCurrentRowIndex[2]
-- Time elapsed: 0.013 s <<< ERROR!
org.apache.parquet.bytes.TrackingByteBufferAllocator$LeakedByteBufferException:
2 ByteBuffer object(s) is/are remained unreleased after closing this allocator.
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.close(TrackingByteBufferAllocator.java:171)
at
org.apache.parquet.hadoop.TestParquetReader.closeAllocator(TestParquetReader.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Caused by:
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException:
Allocation stacktrace of the first ByteBuffer:
at
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException.create(TrackingByteBufferAllocator.java:100)
at
org.apache.parquet.bytes.TrackingByteBufferAllocator$ByteBufferAllocationStacktraceException.access$300(TrackingByteBufferAllocator.java:94)
at
org.apache.parquet.bytes.TrackingByteBufferAllocator.allocate(TrackingByteBufferAllocator.java:140)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.readVectored(RawLocalFileSystem.java:336)
at
org.apache.hadoop.fs.BufferedFSInputStream.readVectored(BufferedFSInputStream.java:182)
at
org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:307)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readVectored(ChecksumFileSystem.java:444)
at
org.apache.hadoop.fs.FSDataInputStream.readVectored(FSDataInputStream.java:307)
```
### Component(s)
Core
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
