[
https://issues.apache.org/jira/browse/AURORA-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14148382#comment-14148382
]
Kevin Sweeney commented on AURORA-722:
--------------------------------------
Microbenchmark results:
{noformat}
# Deflater.BEST_SPEED, No buffer
deflate: 11745ms
total:12734ms
compression ratio: 647270930/121616834 = 532.22%
# Deflater.BEST_SPEED, buf = 512KiB
deflate: 11015ms
total:12001ms
compression ratio: 647270930/121616834 = 532.22%
# Deflater.BEST_SPEED, BufferedOutputStream
deflate: 6885ms
total:7694ms
compression ratio: 647270930/121616834 = 532.22%
# Deflater.BEST_SPEED, BufferedOutputStream at 512KiB
deflate: 6752ms
total:7585ms
compression ratio: 647270930/121616834 = 532.22%
# Deflater.DEFAULT_COMPRESSION
deflate: 16664ms
total:17219ms
compression ratio: 647270930/92690409 = 698.31%
# Deflater.DEFAULT_COMPRESSION, buf size = 512KiB
deflate: 16516ms
total:17103ms
compression ratio: 647270930/92690409 = 698.31%
# Deflater.DEFAULT_COMPRESSION, BufferedOutputStream
deflate: 12241ms
total:12788ms
compression ratio: 647270930/92690409 = 698.31%
# Deflater.DEFAULT_COMPRESSION, BufferedOutputStream at 512KiB
deflate: 11548ms
total:12108ms
compression ratio: 647270930/92690409 = 698.31%
{noformat}
> snapshot performance issues
> ---------------------------
>
> Key: AURORA-722
> URL: https://issues.apache.org/jira/browse/AURORA-722
> Project: Aurora
> Issue Type: Bug
> Components: Scheduler
> Reporter: Kevin Sweeney
> Assignee: Kevin Sweeney
> Fix For: 0.6.0
>
>
> In one of our larger production clusters we're seeing issues with snapshot
> performance that cause the scheduler to failover before completing a snapshot.
> For background, the scheduler writes a compressed (when -deflate_snapshots is
> enabled), binary-encoded Snapshot (from api.thrift) to the mesos replicated
> log every hour (or -dlog_snapshot_interval). This snapshot represents most of
> the scheduler's heap usage, including the configuration for all tasks running
> in the cluster.
> Add appropriate instrumentation to the snapshot routine and patch any obvious
> performance bottlenecks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
