[
https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
holdenk updated SPARK-15369:
----------------------------
Description: Transferring data from the JVM to the Python executor can be a
substantial bottleneck. While Jython is not suitable for all UDFs or map
functions, it may be suitable for some simple ones. We should investigate the
option of using Jython to accelerate these small functions. (was: Transfering
data from the JVM to the Python executor can be a substantial bottleneck. While
JYthon is not suitable for all UDFs or map functions, it may be suitable for
some simple ones. We should investigate the option of using JYthon to
accelerate these small functions.)
> Investigate selectively using Jython for parts of PySpark
> ---------------------------------------------------------
>
> Key: SPARK-15369
> URL: https://issues.apache.org/jira/browse/SPARK-15369
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Reporter: holdenk
> Priority: Minor
>
> Transferring data from the JVM to the Python executor can be a substantial
> bottleneck. While Jython is not suitable for all UDFs or map functions, it
> may be suitable for some simple ones. We should investigate the option of
> using Jython to accelerate these small functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
