Jean-Daniel Cryans has submitted this change and it was merged. Change subject: block_manager: better preallocation in log block manager ......................................................................
block_manager: better preallocation in log block manager As has been discussed to death, the LBM's preallocation strategy isn't great because it yields one filesystem extent per created block. While multiple Append() calls will write to the same extent, small blocks (i.e. those with just a single Append()) suffer. We can do much better, and in this patch, we do: a rolling preallocation window is maintained for each container. Implementation-wise, I did what I could to restrict this knowledge to the containers. But it did mean a small semantic change to UpdateBytesWritten() to accomodate for the fact that Append() now updates total_bytes_written_. As a quick test, I ran block_manager_stress-test before and after, using filefrag to count the number of extents. The number of containers was the same in both cases (64). Before, I counted 446 extents totaling 2.1Gb and after I counted 72 extents totaling 1.4Gb. Change-Id: Ie7dcba01aff125a68817ca91b3c306977c7104e6 Reviewed-on: http://gerrit.cloudera.org:8080/4848 Tested-by: Adar Dembo <a...@cloudera.com> Reviewed-by: Jean-Daniel Cryans <jdcry...@apache.org> --- M src/kudu/fs/block_manager-test.cc M src/kudu/fs/log_block_manager.cc 2 files changed, 124 insertions(+), 63 deletions(-) Approvals: Jean-Daniel Cryans: Looks good to me, approved Adar Dembo: Verified -- To view, visit http://gerrit.cloudera.org:8080/4848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ie7dcba01aff125a68817ca91b3c306977c7104e6 Gerrit-PatchSet: 5 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Jean-Daniel Cryans <jdcry...@apache.org> Gerrit-Reviewer: Mike Percy <mpe...@apache.org> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
