[
https://issues.apache.org/jira/browse/HIVE-8373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15769284#comment-15769284
]
liyunzhang_intel commented on HIVE-8373:
----------------------------------------
[~xuefuz] and [~lirui]: i have run spark.master=local successfully to avoid
"OOM" exception by appending following in conf/hive-env.sh
{code}
export HADOOP_OPTS="$HADOOP_OPTS -XX:MaxPermSize=128m"
{code}
Refer the
[answer|http://stackoverflow.com/questions/34476195/why-does-creating-hivecontext-fail-with-java-lang-outofmemoryerror-permgen-spa]
on stackoverflow to solve the OOM exception.
Do we need to increase the value of "MaxPermSize" to make hos run successfully
on local mode? if yes, we should add this on wiki.
> OOM for a simple query with spark.master=local [Spark Branch]
> -------------------------------------------------------------
>
> Key: HIVE-8373
> URL: https://issues.apache.org/jira/browse/HIVE-8373
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Xuefu Zhang
>
> I have a straigh forward query to run in Spark local mode, but get an OOM
> even though the data volumn is tiny:
> {code}
> Exception in thread "Spark Context Cleaner"
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Spark Context Cleaner"
> Exception in thread "Executor task launch worker-1"
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Executor task launch worker-1"
> Exception in thread "Keep-Alive-Timer"
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Keep-Alive-Timer"
> Exception in thread "Driver Heartbeater"
> Exception: java.lang.OutOfMemoryError thrown from the
> UncaughtExceptionHandler in thread "Driver Heartbeater"
> {code}
> The query is:
> {code}
> select product_name, avg(item_price) as avg_price from product join item on
> item.product_pk=product.product_pk group by product_name order by avg_price;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
