Roelof Naude created PARQUET-2496:
-------------------------------------
Summary: ParquetWriter::close sometimes fail
Key: PARQUET-2496
URL: https://issues.apache.org/jira/browse/PARQUET-2496
Project: Parquet
Issue Type: Bug
Components: parquet-java
Affects Versions: 1.14.0
Reporter: Roelof Naude
we sometimes run into an exception when closing a ParquetWriter instance:
{code:java}
2024-06-10 10:44:01.398
org.apache.parquet.util.AutoCloseables$ParquetCloseResourceException: Unable to
close resource
2024-06-10 10:44:01.398 at
org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:85)
2024-06-10 10:44:01.398 at
org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:94)
2024-06-10 10:44:01.398 at
org.apache.parquet.hadoop.InternalParquetRecordWriter.close(InternalParquetRecordWriter.java:144)
2024-06-10 10:44:01.398 at
org.apache.parquet.hadoop.ParquetWriter.close(ParquetWriter.java:437)
2024-06-10 10:44:01.398 Caused by: java.nio.channels.ClosedChannelException:
null
2024-06-10 10:44:01.398 at
org.apache.hadoop.hdfs.ExceptionLastSeen.throwException4Close(ExceptionLastSeen.java:73)
2024-06-10 10:44:01.398 at
org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:158)
2024-06-10 10:44:01.398 at
org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.java:639)
2024-06-10 10:44:01.398 at
org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:594)
2024-06-10 10:44:01.398 at
org.apache.hadoop.fs.FSDataOutputStream.hflush(FSDataOutputStream.java:136)
2024-06-10 10:44:01.398 at
org.apache.parquet.hadoop.util.HadoopPositionOutputStream.close(HadoopPositionOutputStream.java:65)
2024-06-10 10:44:01.398 at
org.apache.parquet.hadoop.ParquetFileWriter.close(ParquetFileWriter.java:1663)
2024-06-10 10:44:01.398 at
org.apache.parquet.util.AutoCloseables.close(AutoCloseables.java:49)
2024-06-10 10:44:01.398 at
org.apache.parquet.util.AutoCloseables.uncheckedClose(AutoCloseables.java:83){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
