[
https://issues.apache.org/jira/browse/PARQUET-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318607#comment-17318607
]
ASF GitHub Bot commented on PARQUET-1851:
-----------------------------------------
vdiravka commented on a change in pull request #852:
URL: https://github.com/apache/parquet-mr/pull/852#discussion_r611094854
##########
File path:
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java
##########
@@ -860,6 +860,10 @@ public void endColumn() throws IOException {
* @throws IOException if there is an error while writing
*/
public void endBlock() throws IOException {
+ if (currentRecordCount == 0) {
+ throw new IOException("End block with zero record");
Review comment:
The empty parquet files was used in Drill earlier and now this change
breaks the possibility to store empty tables (with header schema only) in
parquet files. Any suggestions to bypass this and integrate Parquet-1.12.0 to
Drill?
@gszadovszky @cenjunjiedada
##########
File path:
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java
##########
@@ -860,6 +860,10 @@ public void endColumn() throws IOException {
* @throws IOException if there is an error while writing
*/
public void endBlock() throws IOException {
+ if (currentRecordCount == 0) {
+ throw new IOException("End block with zero record");
Review comment:
The empty parquet files was used in Drill earlier and now this change
breaks the possibility to store empty tables (with header schema only) in
parquet files. Any suggestions to bypass this and integrate Parquet-1.12.0 to
Drill?
@gszadovszky @chenjunjiedada
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> ParquetMetadataConveter throws NPE in an Iceberg unit test
> ----------------------------------------------------------
>
> Key: PARQUET-1851
> URL: https://issues.apache.org/jira/browse/PARQUET-1851
> Project: Parquet
> Issue Type: Bug
> Affects Versions: 1.11.0
> Reporter: Junjie Chen
> Assignee: Junjie Chen
> Priority: Major
> Fix For: 1.12.0
>
>
> When writing data to parquet in an Iceberg unit test, it throws NPE as below
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> org.apache.parquet.format.converter.ParquetMetadataConverter.addRowGroup(ParquetMetadataConverter.java:476)
> at
> org.apache.parquet.format.converter.ParquetMetadataConverter.toParquetMetadata(ParquetMetadataConverter.java:177)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.serializeFooter(ParquetFileWriter.java:914)
> at
> org.apache.parquet.hadoop.ParquetFileWriter.end(ParquetFileWriter.java:864)
> at org.apache.iceberg.parquet.ParquetWriter.close(ParquetWriter.java:206) at
> org.apache.iceberg.data.TestLocalScan.writeFile(TestLocalScan.java:429)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
