[
https://issues.apache.org/jira/browse/SPARK-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105221#comment-14105221
]
Hanwei Jin commented on SPARK-2096:
-----------------------------------
I think i almost have solved the issue.
I have passed the test case in JsonSuite ("Complex field and type inferring
(Ignored)") which is ignored, by a little modified.
modified test part :
checkAnswer(
sql("select arrayOfStruct.field1, arrayOfStruct.field2 from jsonTable"),
(Seq(true, false, null), Seq("str1", null, null)) :: Nil
)
However, another question is repeated nested structure is a problem, like
arrayOfStruct.field1.arrayOfStruct.field1 or
arrayOfStruct[0].field1.arrayOfStruct[0].field1
I plan to ignore this problem and try to add "select arrayOfStruct.field1,
arrayOfStruct.field2 from jsonTable where arrayOfStruct.field1==true "
Besides, my friend anyweil (Wei Li) solved the problem of arrayOfStruct.field1
and its Filter part( means where parsing).
I am fresh here but will continue working on spark :)
> Correctly parse dot notations for accessing an array of structs
> ---------------------------------------------------------------
>
> Key: SPARK-2096
> URL: https://issues.apache.org/jira/browse/SPARK-2096
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.0.0
> Reporter: Yin Huai
> Priority: Minor
> Labels: starter
>
> For example, "arrayOfStruct" is an array of structs and every element of this
> array has a field called "field1". "arrayOfStruct[0].field1" means to access
> the value of "field1" for the first element of "arrayOfStruct", but the SQL
> parser (in sql-core) treats "field1" as an alias. Also,
> "arrayOfStruct.field1" means to access all values of "field1" in this array
> of structs and the returns those values as an array. But, the SQL parser
> cannot resolve it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
