[
https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297783#comment-15297783
]
Raymond Honderdors commented on SPARK-14946:
--------------------------------------------
both tables (parquet based) compressed 1.7 MB ~ 1500 files
> Spark 2.0 vs 1.6.1 Query Time(out)
> ----------------------------------
>
> Key: SPARK-14946
> URL: https://issues.apache.org/jira/browse/SPARK-14946
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Raymond Honderdors
> Priority: Critical
> Attachments: Query Plan 1.6.1.png, resolved timeout long
> duration.png, screenshot-spark_2.0.png, spark-defaults.conf, spark-env.sh,
> version 1.6.1 screen 1 - thrift collect = true.png, version 1.6.1 screen 1
> thrift collect = false.png, version 1.6.1 screen 2 thrift collect =false.png,
> version 2.0 -screen 1 thrift collect = false.png, version 2.0 screen 2 thrift
> collect = true.png, versiuon 2.0 screen 1 thrift collect = true.png
>
>
> I run a query using JDBC driver running it on version 1.6.1 it return after 5
> – 6 min , the same query against version 2.0 fails after 2h (due to timeout)
> for details on how to reproduce (also see comments below)
> here is what I tried
> I run the following query: select * from pe_servingdata sd inner join
> pe_campaigns_gzip c on sd.campaignid = c.campaign_id ;
> (with and without a counter and group by on campaigne_id)
> I run spark 1.6.1 and Thriftserver
> then running the sql from beeline or squirrel, after a few min I get answer
> (0 row) it is correct due to the fact my data did not have matching campaign
> ids in both tables
> when I run spark 2.0 and Thriftserver, I once again run the sql statement and
> after 2:30 min it gives up, bit already after 30/60 sec I stop seeing
> activity on the spark ui
> (sorry for the delay in competing the description of the bug, I was on and
> off work due to national holidays)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
