Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/7952
to look at the new patch set (#2).
Change subject: KUDU-2131: switch to LIFO log container retrieval
......................................................................
KUDU-2131: switch to LIFO log container retrieval
Currently, LBM selects available log container from available container
queue using FIFO, given the intention to spread the load across all
available containers. However, this may cause inefficiency especially
in the case of tablet copy if there is a long available containers
queue. One failed test case is copying tablet of size 26.81GB on a
tablet server with 14 disks and 19199 containers (only 97 of them are
full). In this case, there are too many containers involved, so that
DownloadBlocks took longer that 3m, the default idle timeout for a
tablet copy session, to commit the new blocks such that the session is
terminated before DownloadWALs.
This patch switches to LIFO for log container selection, so that
finalized blocks will exhaust one container at a time. Thus, when
batching disk synchronization for a groups of created blocks, the
number of fsync() operations will be significant lower for tablet
servers with plenty of available log containers.
It also marks '--tablet_copy_idle_timeout_ms', the flag that controls
the amount of time without activity before a tablet copy session
expires, as a user facing flag. And renames it to
'--tablet_copy_idle_timeout_sec'.
Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
M src/kudu/tserver/tablet_copy_service-test.cc
M src/kudu/tserver/tablet_copy_service.cc
5 files changed, 52 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/7952/2
--
To view, visit http://gerrit.cloudera.org:8080/7952
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Hao Hao <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <[email protected]>
