[
https://issues.apache.org/jira/browse/SPARK-14034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16492078#comment-16492078
]
Maxim Gekk commented on SPARK-14034:
------------------------------------
I checked on Spark 2.3:
{code:scala}
case class Y(c: Long, b: Long, a: Long)
defined class X
spark.read.json("test.json").as[X].collect().head.arr.head.c
{code}
{code}
res0: Long = 1
{code}
Changing order of parameters in class Y doesn't impact on the result. It seems
the issue doesn't exist any more.
> Converting to Dataset causes wrong order and values in nested array of
> documents
> --------------------------------------------------------------------------------
>
> Key: SPARK-14034
> URL: https://issues.apache.org/jira/browse/SPARK-14034
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.1
> Reporter: Steven She
> Priority: Major
>
> I'm deserializing the following JSON document into a Dataset with Spark 1.6.1
> in the console:
> {noformat}
> {"arr": [{"c": 1, "b": 2, "a": 3}]}
> {noformat}
> I have the following case classes:
> {noformat}
> case class X(arr: Seq[Y])
> case class Y(c: Int, b: Int, a: Int)
> {noformat}
> I run the following in the console to retrieve the value of `c` in the array,
> which should have a value of 1 in the data file, but I get the value 3
> instead:
> {noformat}
> scala> sqlContext.read.json("../test.json").as[X].collect().head.arr.head.c
> res19: Int = 3
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
