1fanwang opened a new pull request, #3805: URL: https://github.com/apache/parquet-java/pull/3805
### Rationale for this change Reading the writer's size after successfully closing a Parquet file throws a null-pointer exception because its column buffers have already been released. Callers cannot report the completed file size through the writer. Closes https://github.com/apache/parquet-java/issues/2037. ### What changes are included in this PR? Save the final byte position after writing the footer, before closing the output stream. The size accessor then uses that position without accessing released buffers or a closed channel. ### Are these changes tested? #### Testing Done On Java 17, this standalone program writes one row, closes the file, and compares the reported size with the real file length. Before, on 2df8d02678dab4bb8b926a0d3221cc652984c7ab: ```text java.lang.NullPointerException: Cannot invoke "org.apache.parquet.column.ColumnWriteStore.getBufferedSize()" because "this.columnStore" is null ``` After: ```text reported=312, file=312 ``` Run the following from the base checkout and the PR checkout, saving the source below after the build: ```bash mvn -q -pl parquet-hadoop -am -DskipTests package dependency:build-classpath -Dmdep.outputFile=target/runtime-classpath java -cp "parquet-common/target/classes:parquet-hadoop/target/classes:$(cat parquet-hadoop/target/runtime-classpath)" WriterSizeProbe.java size.parquet ``` <details> <summary>Reproducer source</summary> Save as WriterSizeProbe.java: ```java import java.nio.file.Files; import java.nio.file.Path; import org.apache.parquet.example.data.Group; import org.apache.parquet.example.data.simple.SimpleGroup; import org.apache.parquet.hadoop.ParquetWriter; import org.apache.parquet.hadoop.example.ExampleParquetWriter; import org.apache.parquet.io.LocalOutputFile; import org.apache.parquet.schema.MessageType; import org.apache.parquet.schema.MessageTypeParser; class WriterSizeProbe { public static void main(String[] args) throws Exception { Path path = Path.of(args[0]); MessageType schema = MessageTypeParser.parseMessageType("message test { required int32 value; }"); ParquetWriter<Group> writer = ExampleParquetWriter.builder(new LocalOutputFile(path)) .withType(schema).build(); writer.write(new SimpleGroup(schema).append("value", 7)); writer.close(); try { long reported = writer.getDataSize(); long actual = Files.size(path); System.out.printf("reported=%d, file=%d%n", reported, actual); if (reported != actual) { throw new AssertionError("Reported size differs from the completed file"); } } catch (NullPointerException failure) { System.out.println(failure); } } } ``` </details> The regressions also read back empty, single-row and multiple-row-group files through both local and FileChannel-backed outputs, including a second close. ### Are there any user-facing changes? After a successful close, the size accessor returns the completed file size, including the footer. Its behavior while the writer is open is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
