Maxim Gekk created SPARK-24190:
----------------------------------
Summary: lineSep shouldn't be required in JSON write
Key: SPARK-24190
URL: https://issues.apache.org/jira/browse/SPARK-24190
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.3.0
Reporter: Maxim Gekk
Currently, the lineSep option is required by JSON datasource in write if
encoding is different from UTF-8. For example, the code:
{code:scala}
df.write.option("encoding", "UTF-32BE").json(file)
{code}
throws the exception:
{code}
requirement failed: The lineSep option must be specified for the UTF-32BE
encoding
java.lang.IllegalArgumentException: requirement failed: The lineSep option must
be specified for the UTF-32BE encoding
at scala.Predef$.require(Predef.scala:224)
at
org.apache.spark.sql.catalyst.json.JSONOptions$$anonfun$32.apply(JSONOptions.scala:118)
at
org.apache.spark.sql.catalyst.json.JSONOptions$$anonfun$32.apply(JSONOptions.scala:103)
at scala.Option.map(Option.scala:146)
{code}
The restriction should NOT be applied to writing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
