Adar Dembo has submitted this change and it was merged. Change subject: KUDU-2131: switch to LIFO log container retrieval ......................................................................
KUDU-2131: switch to LIFO log container retrieval Currently, LBM selects available log container from available container queue using FIFO, given the intention to spread the load across all available containers. However, this may cause inefficiency especially in the case of tablet copy if there is a long available containers queue. One failed test case is copying tablet of size 26.81GB on a tablet server with 14 disks and 19199 containers (only 97 of them are full). In this case, there are too many containers involved, so that DownloadBlocks took longer that 3m, the default idle timeout for a tablet copy session, to commit the new blocks such that the session is terminated before DownloadWALs. This patch switches to LIFO for log container selection, so that finalized blocks will exhaust one container at a time. Thus, when batching disk synchronization for a groups of created blocks, the number of fsync() operations will be significant lower for tablet servers with plenty of available log containers. It also marks '--tablet_copy_idle_timeout_ms', the flag that controls the amount of time without activity before a tablet copy session expires, as a user facing flag. And renames it to '--tablet_copy_idle_timeout_sec'. Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687 Reviewed-on: http://gerrit.cloudera.org:8080/7956 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <[email protected]> --- M src/kudu/fs/log_block_manager-test.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/tserver/tablet_copy_service-test.cc M src/kudu/tserver/tablet_copy_service.cc 5 files changed, 52 insertions(+), 9 deletions(-) Approvals: Adar Dembo: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/7956 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6eb0bc99422c293417b64e0b6e86c87a3d55e687 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: branch-1.5.x Gerrit-Owner: Adar Dembo <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Hao Hao <[email protected]> Gerrit-Reviewer: Kudu Jenkins
