[
https://issues.apache.org/jira/browse/KUDU-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239015#comment-17239015
]
ASF subversion and git services commented on KUDU-3214:
-------------------------------------------------------
Commit 64592274ac4ee5bff4c01807602e6e39714a0f80 in kudu's branch
refs/heads/master from ningw
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=6459227 ]
KUDU-3214 Parallelize DownloadWALs in tablet copy operation
Make DownloadWALs operation in parallel.
Add tests:
- Correctly collect status of parallelized DownloadWALs.
- Random inject latency and heavy insert.
- Random inject latency and heavy update.
Perf: Timing download a tablet without any block data.
WAL size (In MB) 575 1149 2183
WAL count 87 174 348
In seq: 3.417s 6.404s 12.371s
In parallel(4threads): 2.543s 4.858s 8.761s
About 25% promotion.
Change-Id: Ib513118d233efdf4be7cbb0b790354c4c3e1bd84
Reviewed-on: http://gerrit.cloudera.org:8080/16726
Tested-by: Andrew Wong <[email protected]>
Reviewed-by: Andrew Wong <[email protected]>
> Parallelize DownloadWAL in tablet copy operations
> -------------------------------------------------
>
> Key: KUDU-3214
> URL: https://issues.apache.org/jira/browse/KUDU-3214
> Project: Kudu
> Issue Type: New Feature
> Components: consensus, tablet
> Reporter: wangningito
> Assignee: wangningito
> Priority: Minor
>
> When downloading tablet data via tablet copy operation, it is necessary to
> download blocks data and wal data.
> In [KUDU-1728|https://issues.apache.org/jira/browse/KUDU-1728] the
> DownloadBlocks were parallelized.
> However, to intuition, it may make senses to make DownloadWALs parallelized
> as well. It requires very strict performance tests and correctness guarantee
> to determine whether to have parallelized version DownloadWALs or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
