[
https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000148#comment-16000148
]
Takeshi Yamamuro commented on SPARK-19112:
------------------------------------------
I also put the result here:
{code}
scaleFactor: 4
AWS instance: c4.4xlarge
-- zstd
Running execution q4-v1.4 iteration: 1, StandardRun=true
Execution time: 53.315878375s
Running execution q4-v1.4 iteration: 2, StandardRun=true
Execution time: 53.468174668s
Running execution q4-v1.4 iteration: 3, StandardRun=true
Execution time: 57.282403146s
-- lz4
Running execution q4-v1.4 iteration: 1, StandardRun=true
Execution time: 20.779643053s
Running execution q4-v1.4 iteration: 2, StandardRun=true
Execution time: 16.520911319s
Running execution q4-v1.4 iteration: 3, StandardRun=true
Execution time: 15.897124967s
-- snappy
Running execution q4-v1.4 iteration: 1, StandardRun=true
Execution time: 21.132412036999998s
Running execution q4-v1.4 iteration: 2, StandardRun=true
Execution time: 15.908867743999998s
Running execution q4-v1.4 iteration: 3, StandardRun=true
Execution time: 15.789648712s
-- lzf
Running execution q4-v1.4 iteration: 1, StandardRun=true
Execution time: 21.339518781s
Running execution q4-v1.4 iteration: 2, StandardRun=true
Execution time: 16.881225328s
Running execution q4-v1.4 iteration: 3, StandardRun=true
Execution time: 15.813455479s
{code}
ISTM it's okay to close the current pr for now. But we should close this ticket
now? IMHO the performance depends on environments, configurations, code
structure, and so on. So, we could keep this open for collecting other's
performance results?
> add codec for ZStandard
> -----------------------
>
> Key: SPARK-19112
> URL: https://issues.apache.org/jira/browse/SPARK-19112
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Thomas Graves
> Priority: Minor
>
> ZStandard: https://github.com/facebook/zstd and
> http://facebook.github.io/zstd/ has been in use for a while now. v1.0 was
> recently released. Hadoop
> (https://issues.apache.org/jira/browse/HADOOP-13578) and others
> (https://issues.apache.org/jira/browse/KAFKA-4514) are adopting it.
> Zstd seems to give great results => Gzip level Compression with Lz4 level CPU.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
