Brad Willard created SPARK-5092:
-----------------------------------
Summary: Selecting from a nested structure with SparkSQL should
return a nested structure
Key: SPARK-5092
URL: https://issues.apache.org/jira/browse/SPARK-5092
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 1.2.0
Reporter: Brad Willard
Priority: Minor
When running a sparksql query like this (at least on a json dataset)
select
rid,
meta_data.name
from
a_table
The rows returned lose the nested structure. I receive a row like
Row(rid='123', name='delete')
instead of
Row(rid='123', meta_data=Row(name='data'))
I personally think this is confusing especially when programmatically building
and executing queries and then parsing it to find your data in a new structure.
I could understand how that's less desirable in some situations, but you could
get around it by supporting 'as'. If you wanted to skip the nested structure
simply write.
select
rid,
meta_data.name as name
from
a_table
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
