[
https://issues.apache.org/jira/browse/SPARK-16357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zeweichen updated SPARK-16357:
------------------------------
Environment:
Apache Spark 1.6.1
TPCx-BB 1.0.1
jdk 1.7
was:
Apache Spark 1.6.1
TPCx-BB 1.0.1
jdk 1.8.0_73
> After enabling Spark shuffle RPC encryption using 3DES, Sparksql query has
> poor performance when running in parallel.
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-16357
> URL: https://issues.apache.org/jira/browse/SPARK-16357
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.1
> Environment: Apache Spark 1.6.1
> TPCx-BB 1.0.1
> jdk 1.7
> Reporter: zeweichen
> Priority: Minor
>
> We use TPCx-BB(BigBench) to evaluate the performance of Spark shuffle RPC
> encryption.
> During our performance test of Spark shuffle RPC encryption using 3DES on
> Sparksql, we found that throughput test of 2 stream lasts 2.68X time than
> power test, which is much larger than it should be. Q30 cost 0.5 hour in
> stream1, but q30 last 2.5 hours in stream 0 which is much longer than it
> should be. This caused that queries in stream1 go ahead much faster than
> stream0. So it works like single stream and can not fully use CPU resource.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
