[
https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276743#comment-15276743
]
Davies Liu commented on SPARK-14946:
------------------------------------
[[email protected]] It seems that the second job (scan the bigger
table) did not get started, could you try to disable the broadcast join by set
spark.sql.autoBroadcastJoinThreshold to 0?
> Spark 2.0 vs 1.6.1 Query Time(out)
> ----------------------------------
>
> Key: SPARK-14946
> URL: https://issues.apache.org/jira/browse/SPARK-14946
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Raymond Honderdors
> Priority: Critical
> Attachments: Query Plan 1.6.1.png, screenshot-spark_2.0.png,
> spark-defaults.conf, spark-env.sh, version 1.6.1 screen 1 - thrift collect =
> true.png, version 1.6.1 screen 1 thrift collect = false.png, version 1.6.1
> screen 2 thrift collect =false.png, version 2.0 -screen 1 thrift collect =
> false.png, version 2.0 screen 2 thrift collect = true.png, versiuon 2.0
> screen 1 thrift collect = true.png
>
>
> I run a query using JDBC driver running it on version 1.6.1 it return after 5
> – 6 min , the same query against version 2.0 fails after 2h (due to timeout)
> for details on how to reproduce (also see comments below)
> here is what I tried
> I run the following query: select * from pe_servingdata sd inner join
> pe_campaigns_gzip c on sd.campaignid = c.campaign_id ;
> (with and without a counter and group by on campaigne_id)
> I run spark 1.6.1 and Thriftserver
> then running the sql from beeline or squirrel, after a few min I get answer
> (0 row) it is correct due to the fact my data did not have matching campaign
> ids in both tables
> when I run spark 2.0 and Thriftserver, I once again run the sql statement and
> after 2:30 min it gives up, bit already after 30/60 sec I stop seeing
> activity on the spark ui
> (sorry for the delay in competing the description of the bug, I was on and
> off work due to national holidays)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
