Kazuaki Ishizaki created SPARK-20046:
----------------------------------------
Summary: Facilitate loop optimizations in a JIT compiler regarding
sqlContext.read.parquet()
Key: SPARK-20046
URL: https://issues.apache.org/jira/browse/SPARK-20046
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Reporter: Kazuaki Ishizaki
[This
article|https://databricks.com/blog/2017/02/16/processing-trillion-rows-per-second-single-machine-can-nested-loop-joins-fast.html]
suggests that better generated code can improve performance by facilitating
compiler optimizations.
This JIRA changes the generated code for {{sqlContext.read.parquet("file")}} to
facilitate loop optimizations in a JIT compiler for achieving better
performance. In particular, [this stackoverflow
entry|http://stackoverflow.com/questions/40629435/fast-parquet-row-count-in-spark]
suggests me to improve performance of
{{sqlContext.read.parquet("file").count}}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
