[
https://issues.apache.org/jira/browse/SPARK-26581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738948#comment-16738948
]
Hyukjin Kwon commented on SPARK-26581:
--------------------------------------
{{multiline}} is not supported in write option. You can easily do it via manual
conversion with DataFrame APIs. For instance,
{code}
ds.toJSON.mapPartitions { iter => // write [ for the first line, and ] for the
last line }.write.text("...")
{code}
> Spark Dataset write JSON with Multiline
> ---------------------------------------
>
> Key: SPARK-26581
> URL: https://issues.apache.org/jira/browse/SPARK-26581
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Anil
> Priority: Major
>
> Hi,
> Spark currently can only write JSON file for single node, if i have multiple
> lines or nodes, spark writes nodes with curly braces " \{ }" without comma
> "," in between both the nodes and there is no square brackets at start and
> end of the file. How to achive this. i am trying to write the JSON file like:.
> ds.write().format("JSON").option("multiline","true").save(path);
> please help on this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
