[
https://issues.apache.org/jira/browse/SPARK-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321792#comment-14321792
]
Yanbo Liang edited comment on SPARK-5823 at 2/15/15 2:57 AM:
-------------------------------------------------------------
Hi [~yhuai]
Actually, I have implemented reusing the mutable row for inner structures at
https://issues.apache.org/jira/browse/SPARK-5738.
However I have found that you have mentioned it will extend Spark SQL's JSON
support to handle the case where each object in the dataset might have
considerably different schema
(https://databricks.com/blog/2015/02/02/an-introduction-to-json-support-in-spark-sql.html).
In this scenario, the inner nested mutable row strategy will not take much
performance improvements, am I right?
was (Author: yanboliang):
Hi [~yhuai]
Actually, I have implemented reusing the mutable row for inner structures at
https://issues.apache.org/jira/browse/SPARK-5738.
However I have found that you have mentioned it will extend Spark SQL's JSON
support to handle the case where each object in the dataset might have
considerably different schema. In this scenario, the inner nested mutable row
strategy will not take much performance improvements, am I right?
> Reuse mutable rows for inner structures when parsing JSON objects
> -----------------------------------------------------------------
>
> Key: SPARK-5823
> URL: https://issues.apache.org/jira/browse/SPARK-5823
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Yin Huai
>
> With SPARK-5738, we will reuse a mutable row for rows when parsing JSON
> objects. We can do the same thing for inner structures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
