une 11, 2015 8:43 AM
Subject: spark-sql from CLI --->EXCEPTION: java.lang.OutOfMemoryError: Java
heap space
hey guys
Using Hive and Impala daily intensively.Want to transition to spark-sql in CLI
mode
Currently in my sandbox I am using the Spark (standalone mode) in the CDH
distribution (sta
have spark on YARN but could never get it to
>> run fairly complex queries and I have no answers from this group of the CDH
>> groups.
>>
>> So my assumption is that its possibly not solved , else I have always got
>> very quick answers and responses :-) to m
ption is that its possibly not solved , else I have always got
> very quick answers and responses :-) to my questions on all CDH groups,
> Spark, Hive
>
> best regards
>
> sanjay
>
>
>
> --------------
> *From:* Josh Rosen
> *To:* Sanjay Subra
:-) to my questions on all CDH groups, Spark, Hive
best regards
sanjay
From: Josh Rosen
To: Sanjay Subramanian
Cc: "user@spark.apache.org"
Sent: Friday, June 12, 2015 7:15 AM
Subject: Re: spark-sql from CLI --->EXCEPTION: java.lang.OutOfMemoryError:
Java heap space
It sounds like this might be caused by a memory configuration problem. In
addition to looking at the executor memory, I'd also bump up the driver memory,
since it appears that your shell is running out of memory when collecting a
large query result.
Sent from my phone
> On Jun 11, 2015, at 8:
Sent from my phone
> On Jun 11, 2015, at 8:43 AM, Sanjay Subramanian
> wrote:
>
> hey guys
>
> Using Hive and Impala daily intensively.
> Want to transition to spark-sql in CLI mode
>
> Currently in my sandbox I am using the Spark (standalone mode) in the CDH
> distribution (starving devel
hey guys
Using Hive and Impala daily intensively.Want to transition to spark-sql in CLI
mode
Currently in my sandbox I am using the Spark (standalone mode) in the CDH
distribution (starving developer version 5.3.3)
3 datanode hadoop cluster32GB RAM per node8 cores per node
| spark | 1.2.0+cdh5.3
