Hello Alexey Serbin, Ashwani Raina, Yingchun Lai, Yifan Zhang, Attila Bukor,
Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20680
to look at the new patch set (#4).
Change subject: KUDU-3523 Use f_bsize as Kudu block size instead of st_blksize
......................................................................
KUDU-3523 Use f_bsize as Kudu block size instead of st_blksize
Currently, Kudu uses st_blksize to decide Kudu block size. But
st_blksize is preferred I/O block size, not the real filesystem
block size. In some systems, st_blksize is 64K, and at the same
time the file system block size is 4K. That will cause some unit
tests failed. The following are the failed unit tests:
EncryptionEnabled/LogBlockManagerTest.ContainerPreallocationTest/1
EncryptionEnabled/LogBlockManagerTest.TestPreallocationAndTruncation/1
EncryptionEnabled/LogBlockManagerTest.TestRepairPreallocateExcessSpace/0
EncryptionEnabled/LogBlockManagerTest.TestRepairPreallocateExcessSpace/1
EncryptionEnabled/LogBlockManagerTest.TestRepairUnpunchedBlocks/1
EncryptionEnabled/LogBlockManagerTest.TestDetectMalformedRecords/0
EncryptionEnabled/LogBlockManagerTest.TestDetectMalformedRecords/1
EncryptionEnabled/LogBlockManagerTest.TestDeleteDeadContainersByDeletionTransaction/1
Take EncryptionEnabled/LogBlockManagerTest.ContainerPreallocationTest/1
as an example. It creates the first block and expects the on-disk size
of the container data file is FLAGS_log_container_preallocate_byte.
But actually, it is less than FLAGS_log_container_preallocate_byte.
Because Kudu takes st_blksize as the basic unit for every data block.
When writing the encryption head into the file, Kudu expects it
occupies st_blksize bytes on disk. But it actually occupies f_bsize,
bytes which is less than st_blksize bytes. After that, Kudu
preallocates (FLAGS_log_container_preallocate_byte - st_blksize)
for the container data file from the offset st_blksize. Therefore,
the on-disk file size of the container data file is less than
FLAGS_log_container_preallocate_byte actually. To avoid this problem,
it is better to use f_bsize (file system block size) as Kudu's block
size.
To be compitable with case in which Kudu has already ran on a system
where f_bsize is not equal to st_blksize, --enable_use_f_bsize is
used to control whether or not enable it.
Change-Id: I04343caf5fe377a4fa9b57e6e450307e6b995928
---
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/util/env_posix.cc
2 files changed, 30 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/80/20680/4
--
To view, visit http://gerrit.cloudera.org:8080/20680
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I04343caf5fe377a4fa9b57e6e450307e6b995928
Gerrit-Change-Number: 20680
Gerrit-PatchSet: 4
Gerrit-Owner: Wang Xixu <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Ashwani Raina <[email protected]>
Gerrit-Reviewer: Attila Bukor <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Wang Xixu <[email protected]>
Gerrit-Reviewer: Yifan Zhang <[email protected]>
Gerrit-Reviewer: Yingchun Lai <[email protected]>
