[
https://issues.apache.org/jira/browse/HIVE-9342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274984#comment-14274984
]
Pierre Yin commented on HIVE-9342:
----------------------------------
Another finding is:
If I use the options together with my patch (spark.yarn.executor.memory / cores
/ memory), yarn-cluster mode also can work as what I expected.
spark.executor.instances is one properties in spark. It seems to be one issue
in spark. It seems that SparkSubmit doesn't pick up spark.executor.instances
but it can pick up --num-executors option.
> add num-executors / executor-cores / executor-memory option support for hive
> on spark in Yarn mode
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-9342
> URL: https://issues.apache.org/jira/browse/HIVE-9342
> Project: Hive
> Issue Type: Improvement
> Components: spark-branch
> Affects Versions: spark-branch
> Reporter: Pierre Yin
> Priority: Minor
> Labels: spark
> Fix For: spark-branch
>
> Attachments: HIVE-9342.1-spark.patch
>
>
> When I run hive on spark with Yarn mode, I want to control some yarn option,
> such as --num-executors, --executor-cores, --executor-memory.
> We can append these options into argv in SparkClientImpl.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
