[
https://issues.apache.org/jira/browse/SPARK-7133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512968#comment-14512968
]
Wenchen Fan commented on SPARK-7133:
------------------------------------
Hi [~rxin] , the reason why we didn't generalize UnresolvedGetField to support
all map, struct, and array is because we didn't need to. With SQL, we can
distinguish GetField and GetItem during parse time as they have different
grammar("[]" and "."), so we only need to consider ArrayGetField and
StructGetField.
Now if we want to define a unified API for all map, struct, and array in
DataFrame, I think we should add MapGetField.
cc [~marmbrus] , what do you think?
> Implement struct, array, and map field accessor using apply in Scala and
> __getitem__ in Python
> ----------------------------------------------------------------------------------------------
>
> Key: SPARK-7133
> URL: https://issues.apache.org/jira/browse/SPARK-7133
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Reynold Xin
> Labels: starter
>
> Typing
> {code}
> df.col[1]
> {code}
> and
> {code}
> df.col['field']
> {code}
> is so much eaiser than
> {code}
> df.col.getField('field')
> df.col.getItem(1)
> {code}
> This would require us to define (in Column) an apply function in Scala, and a
> __getitem__ function in Python.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
