Hao Hao has uploaded a new change for review. http://gerrit.cloudera.org:8080/7952
Change subject: KUDU-2131: switch to FILO log container retrieval ...................................................................... KUDU-2131: switch to FILO log container retrieval Currently, LBM selects available log container from available container queue using FIFO, given the intention to spread the load across all available containers. However, this may cause inefficiency especially in the case for tablet copy if there is a long available containers queue. One failed test case is copying tablet of size 26.81GB on a tablet server with 14 disks and 19199 containers(only 97 of them are full). In this case, there are too many containers involved, so that DownloadBlocks took >3m (default idle timeout) to commit the new blocks such that the session is terminated before DownloadWALs. This patch switches to FILO for log container selection, so that finalized blocks will exhaust one container at a time. Thus, when batching disk synchronization for a groups of created blocks, the amount of fsync() will be significant lower for tablet servers with plenty available log containers. It also marks '--tablet_copy_idle_timeout_ms', the flag that controls the amount of time without activity before a tablet copy session expires, to be a user facing flag. And rename it to '--tablet_copy_idle_timeout_sec'. Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687 --- M src/kudu/fs/log_block_manager-test.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/tserver/tablet_copy_service-test.cc M src/kudu/tserver/tablet_copy_service.cc 5 files changed, 52 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/7952/1 -- To view, visit http://gerrit.cloudera.org:8080/7952 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Hao Hao <[email protected]>
