Wang Xixu has posted comments on this change. ( http://gerrit.cloudera.org:8080/20680 )
Change subject: KUDU-3523 Use f_bsize as Kudu block size instead of st_blksize ...................................................................... Patch Set 4: > Patch Set 3: > > Generally, both st_blksize and f_bsize are same when we are working on a > native filesystems on linux e.g. ext3/ext4. > > The use case that you have mentioned here seems to be unusual. I noticed the > filesystem type that you are using is of type overlayfs which IIUC is a > unionfs/layeredfs that probably had layers of different filesystem type. I > can't say for sure, but there is a possibility that in layeredfs, the > st_blksize may differ from the f_bsize depending on which filesystem driver > is used for get that information. I am planning to test this out on local > setup. Will try to share the details once I complete the test or you can > update here if you are already aware of it. > > I am very much interested in knowing more about the setup you are using, what > all actual filesystems are constituting this setup, purpose of having a > overlayfs setup, directory structure used in overlayfs setup, etc. > > LBM needs hole punching support from underlying filesystem (ext4 does provide > that currently). However, I am not quite clear on what it would mean if Kudu > data is residing on overlayfs that has multiple different filesystems > underneath it and there is one filesystem that doesn't support hole-punching. > Maybe it would be worth an exercise to evaluate behaviour of POSIX APIs (used > in Kudu) on overlayfs mount-point. > > IIUC, overlayfs is just an abstraction layer and all the requests go to > targeted underlying file-system so it shouldn't matter much but it is worth a > try though because what all underlying filesystems are supported for Kudu is > something that is not clearly visible when it is an overlayfs setup. > > TL;DR: You could provide more information on the following points: > 1. Provide some more information about the directory setup and how it uses > overlayfs. > 2. Check if all the underlying filesystems support hole punching (if Kudu is > using LBM). > 3. Purpose of using overlayfs. > 4. Can you try reproduction with same steps on a regular underlying > filesystem e.g. ext4 (just to rule out other possibilities)? I reproduction it on a XFS filesystem, st_blkszie is 65536 and f_bsize is 4096. It seems that this case is not unique for overlay filesystem. Please see the issue: #KUDU-3523. -- To view, visit http://gerrit.cloudera.org:8080/20680 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04343caf5fe377a4fa9b57e6e450307e6b995928 Gerrit-Change-Number: 20680 Gerrit-PatchSet: 4 Gerrit-Owner: Wang Xixu <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Wang Xixu <[email protected]> Gerrit-Reviewer: Yifan Zhang <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Thu, 07 Dec 2023 02:49:11 +0000 Gerrit-HasComments: No
