chuanjie.duan created HIVE-16515: ------------------------------------ Summary: Hive MapRedTask should has its own OPTS environment variable Key: HIVE-16515 URL: https://issues.apache.org/jira/browse/HIVE-16515 Project: Hive Issue Type: Improvement Components: HiveServer2 Affects Versions: 1.2.1 Reporter: chuanjie.duan Priority: Minor
Hiveserver2 submit a mapred job using Hiveserver's variable HADOOP_OPTS and HADOOP_CLIENT_OPTS, it's weird. eg: hiveserver started with HADOOP_CLIENT_OPTS="-Xms12000m -Xmx12000m", hive mr sub process would need 12g memory allocate. I think maybe beeline should have another environment variable like HIVE_HEAPSIZE to overwrite HADOOP_CLIENT_OPTS when hive mr job submit MapRedTask.java public int execute(DriverContext driverContext) { …… if (ShimLoader.getHadoopShims().isLocalMode(conf)) { ...... } else { // nothing to do - we are not running in local mode - only submitting // the job via a child process. in this case it's appropriate that the // child jvm use the same memory as the parent jvm } …… String jarCmd = hiveJar + " " + ExecDriver.class.getName() + libJarsOption; String cmdLine = hadoopExec + " jar " + jarCmd + " -plan " + planPath.toString() + " " + isSilent + " " + hiveConfArgs; …… } // Run ExecDriver in another JVM executor = Runtime.getRuntime().exec(cmdLine, env, new File(workDir)); ${HADOOP_HOME}/bin/hadoop ...... # Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS" ...... export CLASSPATH=$CLASSPATH exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@" -- This message was sent by Atlassian JIRA (v6.3.15#6346)