[
https://issues.apache.org/jira/browse/HIVE-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joydeep Sen Sarma updated HIVE-1520:
------------------------------------
Attachment: 1520.1.patch
> hive.mapred.local.mem should only be used in case of local mode job
> submissions
> -------------------------------------------------------------------------------
>
> Key: HIVE-1520
> URL: https://issues.apache.org/jira/browse/HIVE-1520
> Project: Hadoop Hive
> Issue Type: Bug
> Reporter: Joydeep Sen Sarma
> Attachments: 1520.1.patch
>
>
> Currently - whenever we submit a map-reduce job via a child jvm process, hive
> sets HADOOP_HEAPSIZE to hive.mapred.local.mem (thereby limiting the max heap
> memory of the child jvm). the assumption being that we are submitting a job
> for local mode execution and different memory limits apply for that.
> however - one can submit jobs via a child jvm for non local mode execution as
> well. This is useful, for example, if hive wants to submit jobs via different
> hadoop clients (for sending jobs to different hadoop clusters). in such case,
> we can use the 'hive.exec.submitviachild' and 'hadoop.bin.path' to dispatch
> job via an alternate hadoop client install point. however in such case, we
> don't need to set HADOOP_HEAPSIZE. all we are using the child jvm is to run
> the small bit of hive code that submits the job (and not for local mode
> execution).
> in this case - we shouldn't be setting the child jvm's memory limit and
> should leave it to what the parent's value is.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
