Kazuaki Ishizaki created SPARK-16213:
----------------------------------------
Summary: Reduce runtime overhead of a program that creates an
primitive array in DataFrame
Key: SPARK-16213
URL: https://issues.apache.org/jira/browse/SPARK-16213
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Kazuaki Ishizaki
Reduce runtime overhead of a program that creates an primitive array in
DataFrame
When a program creates an array in DataFrame, the code generator creates boxing
operations. If an array is for primitive type, there are some opportunities for
optimizations in generated code to reduce runtime overhead.
Here is a simple example that has generated code with boxing operation
{code}
val df = sparkContext.parallelize(Seq(0.0d, 1.0d), 1).toDF
df.selectExpr("Array(value + 1.1d, value + 2.2d)").show
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
