[
https://issues.apache.org/jira/browse/HIVE-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021535#comment-16021535
]
Laurel Hale commented on HIVE-16735:
[~leftylev] Thanks, but I think I must be missing something? I checked both
places and neither mentions that the value should be set in MBs. I cleared my
browser cache. This is what I see for the first link above:
--
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.mapred.local.mem
hive.mapred.local.mem
Default Value: 0
Added In: Hive 0.3.0
For local mode, memory of the mappers/reducers.
-
And here is what I see for the second link listed above:
-
https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Hive,Map-ReduceandLocal-Mode
Note that there may be differences in the runtime environment of Hadoop server
nodes and the machine running the Hive client (because of different jvm
versions or different software libraries). This can cause unexpected
behavior/errors while running in local mode. Also note that local mode
execution is done in a separate, child jvm (of the Hive client). If the user so
wishes, the maximum amount of memory for this child jvm can be controlled via
the option hive.mapred.local.mem. By default, it's set to zero, in which case
Hive lets Hadoop determine the default memory limits of the child jvm.
-
Thanks for being patient.
Best,
Laurel
> Please add that hive.mapred.local.mem must be set in MBs in the documentation
> -
>
> Key: HIVE-16735
> URL: https://issues.apache.org/jira/browse/HIVE-16735
> Project: Hive
> Issue Type: Task
> Components: Documentation
>Reporter: Laurel Hale
>Assignee: Lefty Leverenz
>Priority: Minor
>
> Hi Lefty,
> One of my developers noticed this and we do not document the local mode.
> Thought I'd let you know.
> Here's a good place where you could put this:
> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution
> There's already a small subsection where hive.mapred.local.mem is documented,
> but there isn't any information about how to set the value. My developer
> discovered that when you set it in bytes, it throws the following error:
> 2017-03-12 09:24:31,835 ERROR [HiveServer2-Background-Pool: Thread-22163()]:
> mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(341)) - Exception:
> java.lang.NumberFormatException: For input string: "4294967296"
> java.lang.NumberFormatException: For input string: "4294967296"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:495)
> at java.lang.Integer.parseInt(Integer.java:527)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1259)
> at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:2415)
> at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:2424)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeInChildVM(MapredLocalTask.java:228)
> Instead, this value should be set in MBs for success.
> Thanks,
> Laurel
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
