[
https://issues.apache.org/jira/browse/PARQUET-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Rushton updated PARQUET-1408:
--------------------------------------
Summary: parquet-tools SimpleRecord does not display empty fields (was:
parquet-tools SimpleRecord does display null columns)
> parquet-tools SimpleRecord does not display empty fields
> --------------------------------------------------------
>
> Key: PARQUET-1408
> URL: https://issues.apache.org/jira/browse/PARQUET-1408
> Project: Parquet
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Nicholas Rushton
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.10.1
>
>
> When using parquet-tools on a parquet file with null records the null columns
> are omitted from the output.
>
> Example:
> {code:java}
> scala> case class Foo(a: Int, b: String)
> defined class Foo
> scala> org.apache.spark.sql.SparkSession.builder.getOrCreate.createDataset((0
> to 1000).map(x => Foo(1,null))).write.parquet("/tmp/foobar/"){code}
> Actual:
> {code:java}
> ☁ parquet-tools [master] ⚡ java -jar
> target/parquet-tools-1.10.1-SNAPSHOT.jar cat -j
> /tmp/foobar/part-00000-436a4d37-d82a-4771-8e7e-e4d428464675-c000.snappy.parquet
> | head -n5
> {"a":1}
> {"a":1}
> {"a":1}
> {"a":1}
> {"a":1}{code}
> Expected:
> {code:java}
> ☁ parquet-tools [master] ⚡ java -jar
> target/parquet-tools-1.10.1-SNAPSHOT.jar cat -j
> /tmp/foobar/part-00000-436a4d37-d82a-4771-8e7e-e4d428464675-c000.snappy.parquet
> | head -n5
> {"a":1,"b":null}
> {"a":1,"b":null}
> {"a":1,"b":null}
> {"a":1,"b":null}
> {"a":1,"b":null}{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
