ssage in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> below:
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
> To unsubscribe from spark sql left join gives KryoException: Buffer overflow,
> click here.
> NAML
--
View this message in context:
ht
.
Am I correct in my expectations?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-left-join-gives-KryoException-Buffer-overflow-tp10157p11432.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
>
> When SPARK-2211 is done, will spark sql automatically choose join
> algorithms?
> Is there some way to manually hint the optimizer?
>
Ideally we will select the best algorithm for you. We are also considering
ways to allow the user to hint.
Hi Michael,
Thanks for the suggestion. In my query, both table are too large to use
broadcast join.
When SPARK-2211 is done, will spark sql automatically choose join
algorithms?
Is there some way to manually hint the optimizer?
2014-07-19 5:23 GMT+08:00 Michael Armbrust :
> Unfortunately, this
Unfortunately, this is a query where we just don't have an efficiently
implementation yet. You might try switching the table order.
Here is the JIRA for doing something more efficient:
https://issues.apache.org/jira/browse/SPARK-2212
On Fri, Jul 18, 2014 at 7:05 AM, Pei-Lun Lee wrote:
> Hi,
>
Hi,
We have a query with left joining and got this error:
Caused by: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 1.0:0 failed 4 times, most recent failure: Exception failure
in TID 5 on host ip-10-33-132-101.us-west-2.compute.internal:
com.esotericsoftware.kryo.KryoExc
