Kousuke Saruta created SPARK-35104:
--------------------------------------
Summary: Fix ugly indentation of multiple JSON records in a single
split file generated by JacksonGenerator when pretty option is true
Key: SPARK-35104
URL: https://issues.apache.org/jira/browse/SPARK-35104
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1, 3.0.2, 3.2.0
Reporter: Kousuke Saruta
Assignee: Kousuke Saruta
When writing multiple JSON records into a single split file with pretty option
true, indentation will be broken except for the first JSON record.
{code}
// Run in the Spark Shell.
// Set spark.sql.leafNodeDefaultParallelism to 1 for the current master.
// Or set spark.default.parallelism for the previous releases.
spark.conf.set("spark.sql.leafNodeDefaultParallelism", 1)
val df = Seq("a", "b", "c").toDF
df.write.option("pretty", "true").json("/path/to/output")
# Run in the Shell
$ cat /path/to/output/*.json
{
"value" : "a"
}
{
"value" : "b"
}
{
"value" : "c"
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
