Idan Zalzberg created SPARK-5318:
------------------------------------
Summary: Add ability to control partition count in SparkSql
Key: SPARK-5318
URL: https://issues.apache.org/jira/browse/SPARK-5318
Project: Spark
Issue Type: New Feature
Components: SQL
Reporter: Idan Zalzberg
When using SparkSql, e.g. sqlContext.sql("..."), spark might need to read
hadoop files.
However, unlike the hadoopFile API, there is no documented way to set the
minimal partition count when reading.
There is an undocumented way, though, using "mapred.map.tasks" in hiveConf
I suggest we make a documented way to do it, in the exact same way (possibly
with a better name)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
