zeweichen created SPARK-16357:
---------------------------------
Summary: For Spark shuffle RPC encryption using 3DES on Sparksql,
throughput test of 2 stream lasts 2.68X time than power test.
Key: SPARK-16357
URL: https://issues.apache.org/jira/browse/SPARK-16357
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.1
Environment: Apache Spark 1.6.1 with local patch for supporting Q7
subquery feature
Apache Spark 1.6.1 with Spark shuffle RPC encryption
BigBench 1.0.1
Reporter: zeweichen
Priority: Minor
Fix For: 2.1.0
During our performance test of Spark shuffle RPC encryption using 3DES on
Sparksql, we found that throughput test of 2 stream lasts 2.68X time than power
test, which is much larger than it should be. Reason is that q30 cost 0.5 hour
in stream1, but q30 last 2.5 hours in stream 0 which is much longer than it
should be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
