[Gluster-devel] Fwd: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #4631
Can we check the following failure? 1 test(s) failed ./tests/00-geo-rep/georep-basic-dr-rsync-arbiter.t -- Forwarded message - From: Date: Mon, Jul 1, 2019 at 11:48 PM Subject: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #4631 To: See < https://build.gluster.org/job/regression-test-burn-in/4631/display/redirect?page=changes > Changes: [Amar Tumballi] rfc.sh: Improve bug identification [Amar Tumballi] glusterd: fix clang scan defects [Amar Tumballi] core: use multiple servers while mounting a volume using ipv6 -- [...truncated 3.99 MB...] ./tests/bugs/replicate/bug-1561129-enospc.t - 9 second ./tests/bugs/replicate/bug-1221481-allow-fops-on-dir-split-brain.t - 9 second ./tests/bugs/quota/bug-1250582-volume-reset-should-not-remove-quota-quota-deem-statfs.t - 9 second ./tests/bugs/protocol/bug-1321578.t - 9 second ./tests/bugs/posix/bug-1122028.t - 9 second ./tests/bugs/glusterfs/bug-861015-log.t - 9 second ./tests/bugs/glusterfs/bug-848251.t - 9 second ./tests/bugs/glusterd/bug-949930.t - 9 second ./tests/bugs/gfapi/bug-1032894.t - 9 second ./tests/bugs/fuse/bug-983477.t - 9 second ./tests/bugs/ec/bug-1227869.t - 9 second ./tests/bugs/distribute/bug-1088231.t - 9 second ./tests/bugs/cli/bug-1022905.t - 9 second ./tests/bugs/changelog/bug-1321955.t - 9 second ./tests/bugs/changelog/bug-1208470.t - 9 second ./tests/bugs/bug-1371806_2.t - 9 second ./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t - 9 second ./tests/basic/quota-nfs.t - 9 second ./tests/basic/md-cache/bug-1317785.t - 9 second ./tests/basic/glusterd/thin-arbiter-volume-probe.t - 9 second ./tests/basic/fop-sampling.t - 9 second ./tests/basic/ctime/ctime-noatime.t - 9 second ./tests/basic/changelog/changelog-rename.t - 9 second ./tests/basic/afr/stale-file-lookup.t - 9 second ./tests/basic/afr/root-squash-self-heal.t - 9 second ./tests/basic/afr/afr-read-hash-mode.t - 9 second ./tests/basic/afr/add-brick-self-heal.t - 9 second ./tests/bugs/upcall/bug-1369430.t - 8 second ./tests/bugs/transport/bug-873367.t - 8 second ./tests/bugs/snapshot/bug-1260848.t - 8 second ./tests/bugs/snapshot/bug-1064768.t - 8 second ./tests/bugs/shard/shard-inode-refcount-test.t - 8 second ./tests/bugs/shard/bug-1258334.t - 8 second ./tests/bugs/replicate/bug-986905.t - 8 second ./tests/bugs/replicate/bug-1686568-send-truncate-on-arbiter-from-shd.t - 8 second ./tests/bugs/replicate/bug-1626994-info-split-brain.t - 8 second ./tests/bugs/replicate/bug-1132102.t - 8 second ./tests/bugs/replicate/bug-1037501.t - 8 second ./tests/bugs/quota/bug-1104692.t - 8 second ./tests/bugs/posix/bug-1175711.t - 8 second ./tests/bugs/md-cache/afr-stale-read.t - 8 second ./tests/bugs/glusterfs/bug-902610.t - 8 second ./tests/bugs/glusterfs/bug-872923.t - 8 second ./tests/bugs/glusterd/bug-1242875-do-not-pass-volinfo-quota.t - 8 second ./tests/bugs/distribute/bug-1086228.t - 8 second ./tests/bugs/cli/bug-1087487.t - 8 second ./tests/bugs/bug-1258069.t - 8 second ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t - 8 second ./tests/basic/xlator-pass-through-sanity.t - 8 second ./tests/basic/glusterd/arbiter-volume-probe.t - 8 second ./tests/basic/gfapi/libgfapi-fini-hang.t - 8 second ./tests/basic/fencing/fencing-crash-conistency.t - 8 second ./tests/basic/ec/statedump.t - 8 second ./tests/basic/distribute/file-create.t - 8 second ./tests/basic/afr/ta-write-on-bad-brick.t - 8 second ./tests/basic/afr/tarissue.t - 8 second ./tests/basic/afr/ta-read.t - 8 second ./tests/basic/afr/granular-esh/add-brick.t - 8 second ./tests/gfid2path/gfid2path_fuse.t - 7 second ./tests/bugs/shard/bug-1259651.t - 7 second ./tests/bugs/replicate/bug-767585-gfid.t - 7 second ./tests/bugs/replicate/bug-1250170-fsync.t - 7 second ./tests/bugs/replicate/bug-1101647.t - 7 second ./tests/bugs/quota/bug-1287996.t - 7 second ./tests/bugs/quota/bug-1243798.t - 7 second ./tests/bugs/nfs/bug-915280.t - 7 second ./tests/bugs/io-cache/bug-858242.t - 7 second ./tests/bugs/glusterd/quorum-value-check.t - 7 second ./tests/bugs/glusterd/bug-948729/bug-948729-force.t - 7 second ./tests/bugs/distribute/bug-884597.t - 7 second ./tests/bugs/distribute/bug-1368012.t - 7 second ./tests/bugs/core/bug-1699025-brick-mux-detach-brick-fd-issue.t - 7 second ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t - 7 second ./tests/bugs/bug-1702299.t - 7 second ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t - 7 second ./tests/bitrot/bug-1221914.t - 7 second ./tests/bitrot/br-stub.t - 7 second ./tests/basic/ec/ec-read-policy.t - 7 second ./tests/basic/ec/ec-anonymous-fd.t - 7 second ./tests/basic/afr/ta.t - 7 second ./tests/basic/afr/ta-shd.t - 7 second ./tests/basic/afr/gfid-heal.t - 7 second ./tests/gfid2path/get-gfid-to-path.t - 6 second
[Gluster-devel] Fwd: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #4632
Can we check these failures please? 2 test(s) failed ./tests/bugs/glusterd/bug-1699339.t ./tests/bugs/glusterd/bug-857330/normal.t -- Forwarded message - From: Date: Wed, Jul 3, 2019 at 12:08 AM Subject: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #4632 To: See < https://build.gluster.org/job/regression-test-burn-in/4632/display/redirect?page=changes > Changes: [Amar Tumballi] Removing one top command from gluster v help [Amar Tumballi] glusterfs-fops: fix the modularity [Sheetal Pamecha] cli: Remove Wformat-truncation compiler warning [Nithya Balachandran] cluster/dht: Fixed a memleak in dht_rename_cbk -- [...truncated 4.02 MB...] ./tests/bugs/ec/bug-1227869.t - 9 second ./tests/bugs/distribute/bug-1122443.t - 9 second ./tests/bugs/cli/bug-1022905.t - 9 second ./tests/bugs/changelog/bug-1321955.t - 9 second ./tests/bugs/changelog/bug-1208470.t - 9 second ./tests/bugs/bug-1258069.t - 9 second ./tests/bugs/bitrot/1209751-bitrot-scrub-tunable-reset.t - 9 second ./tests/bitrot/bug-1207627-bitrot-scrub-status.t - 9 second ./tests/basic/xlator-pass-through-sanity.t - 9 second ./tests/basic/md-cache/bug-1317785.t - 9 second ./tests/basic/glusterd/thin-arbiter-volume-probe.t - 9 second ./tests/basic/afr/stale-file-lookup.t - 9 second ./tests/basic/afr/root-squash-self-heal.t - 9 second ./tests/basic/afr/add-brick-self-heal.t - 9 second ./tests/features/readdir-ahead.t - 8 second ./tests/bugs/snapshot/bug-1260848.t - 8 second ./tests/bugs/snapshot/bug-1064768.t - 8 second ./tests/bugs/shard/shard-inode-refcount-test.t - 8 second ./tests/bugs/shard/bug-1260637.t - 8 second ./tests/bugs/replicate/bug-986905.t - 8 second ./tests/bugs/replicate/bug-1686568-send-truncate-on-arbiter-from-shd.t - 8 second ./tests/bugs/replicate/bug-1132102.t - 8 second ./tests/bugs/replicate/bug-1101647.t - 8 second ./tests/bugs/replicate/bug-1037501.t - 8 second ./tests/bugs/protocol/bug-1321578.t - 8 second ./tests/bugs/posix/bug-1175711.t - 8 second ./tests/bugs/nfs/bug-915280.t - 8 second ./tests/bugs/io-cache/bug-858242.t - 8 second ./tests/bugs/glusterfs/bug-872923.t - 8 second ./tests/bugs/glusterfs/bug-848251.t - 8 second ./tests/bugs/glusterd/bug-1696046.t - 8 second ./tests/bugs/glusterd/bug-1242875-do-not-pass-volinfo-quota.t - 8 second ./tests/bugs/fuse/bug-983477.t - 8 second ./tests/bugs/distribute/bug-1088231.t - 8 second ./tests/bugs/distribute/bug-1086228.t - 8 second ./tests/bugs/cli/bug-1087487.t - 8 second ./tests/bugs/bug-1371806_2.t - 8 second ./tests/bugs/bitrot/1209818-vol-info-show-scrub-process-properly.t - 8 second ./tests/bitrot/br-stub.t - 8 second ./tests/basic/glusterd/arbiter-volume-probe.t - 8 second ./tests/basic/gfapi/libgfapi-fini-hang.t - 8 second ./tests/basic/fop-sampling.t - 8 second ./tests/basic/fencing/fencing-crash-conistency.t - 8 second ./tests/basic/ec/statedump.t - 8 second ./tests/basic/distribute/file-create.t - 8 second ./tests/basic/ctime/ctime-noatime.t - 8 second ./tests/basic/changelog/changelog-rename.t - 8 second ./tests/basic/afr/tarissue.t - 8 second ./tests/basic/afr/ta-read.t - 8 second ./tests/basic/afr/granular-esh/add-brick.t - 8 second ./tests/basic/afr/afr-read-hash-mode.t - 8 second ./tests/bugs/upcall/bug-1369430.t - 7 second ./tests/bugs/shard/bug-1342298.t - 7 second ./tests/bugs/shard/bug-1259651.t - 7 second ./tests/bugs/replicate/bug-1626994-info-split-brain.t - 7 second ./tests/bugs/replicate/bug-1250170-fsync.t - 7 second ./tests/bugs/quota/bug-1243798.t - 7 second ./tests/bugs/quota/bug-1104692.t - 7 second ./tests/bugs/nfs/bug-1116503.t - 7 second ./tests/bugs/md-cache/afr-stale-read.t - 7 second ./tests/bugs/glusterd/quorum-value-check.t - 7 second ./tests/bugs/glusterd/bug-948729/bug-948729-mode-script.t - 7 second ./tests/bugs/glusterd/bug-948729/bug-948729-force.t - 7 second ./tests/bugs/glusterd/bug-1482906-peer-file-blank-line.t - 7 second ./tests/bugs/distribute/bug-884597.t - 7 second ./tests/bugs/distribute/bug-1368012.t - 7 second ./tests/bugs/core/bug-1699025-brick-mux-detach-brick-fd-issue.t - 7 second ./tests/bugs/core/bug-1168803-snapd-option-validation-fix.t - 7 second ./tests/bugs/bug-1702299.t - 7 second ./tests/bugs/bitrot/1207029-bitrot-daemon-should-start-on-valid-node.t - 7 second ./tests/basic/ec/ec-read-policy.t - 7 second ./tests/basic/afr/ta-write-on-bad-brick.t - 7 second ./tests/basic/afr/ta.t - 7 second ./tests/basic/afr/ta-shd.t - 7 second ./tests/basic/afr/gfid-heal.t - 7 second ./tests/basic/afr/arbiter-remove-brick.t - 7 second ./tests/gfid2path/gfid2path_nfs.t - 6 second ./tests/gfid2path/get-gfid-to-path.t - 6 second ./tests/bugs/snapshot/bug-1178079.t - 6 second ./tests/bugs/shard/bug-1272986.t - 6 second ./tests/bugs/shard/bug-1258334.t - 6 second
[Gluster-devel] Release 7 Branch Created
Hi Team, Release 7 branch has been created in upstream. ## Schedule Curretnly the plan working backwards on the schedule, here's what we have: - Announcement: Week of Aug 4th, 2019 - GA tagging: Aug-02-2019 - RC1: On demand before GA - RC0: July-03-2019 - Late features cut-off: Week of June-24th, 2018 - Branching (feature cutoff date): June-17-2018 Regards Rinku ___ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/836554017 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/486278655 Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] fallocate behavior in glusterfs
On 02/07/19 8:52 PM, FNU Raghavendra Manjunath wrote: Hi All, In glusterfs, there is an issue regarding the fallocate behavior. In short, if someone does fallocate from the mount point with some size that is greater than the available size in the backend filesystem where the file is present, then fallocate can fail with a subset of the required number of blocks allocated and then failing in the backend filesystem with ENOSPC error. The behavior of fallocate in itself is simlar to how it would have been on a disk filesystem (atleast xfs where it was checked). i.e. allocates subset of the required number of blocks and then fail with ENOSPC. And the file in itself would show the number of blocks in stat to be whatever was allocated as part of fallocate. Please refer [1] where the issue is explained. Now, there is one small difference between how the behavior is between glusterfs and xfs. In xfs after fallocate fails, doing 'stat' on the file shows the number of blocks that have been allocated. Whereas in glusterfs, the number of blocks is shown as zero which makes tools like "du" show zero consumption. This difference in behavior in glusterfs is because of libglusterfs on how it handles sparse files etc for calculating number of blocks (mentioned in [1]) At this point I can think of 3 things on how to handle this. 1) Except for how many blocks are shown in the stat output for the file from the mount point (on which fallocate was done), the remaining behavior of attempting to allocate the requested size and failing when the filesystem becomes full is similar to that of XFS. Hence, what is required is to come up with a solution on how libglusterfs calculate blocks for sparse files etc (without breaking any of the existing components and features). This makes the behavior similar to that of backend filesystem. This might require its own time to fix libglusterfs logic without impacting anything else. I think we should just revert the commit b1a5fa55695f497952264e35a9c8eb2bbf1ec4c3 (BZ 817343) and see if it really breaks anything (or check whatever it breaks is something that we can live with). XFS speculative preallocation is not permanent and the extra space is freed up eventually. It can be sped up via procfs tunable: http://xfs.org/index.php/XFS_FAQ#Q:_How_can_I_speed_up_or_avoid_delayed_removal_of_speculative_preallocation.3F. We could also tune the allocsize option to a low value like 4k so that glusterfs quota is not affected. FWIW, ENOSPC is not the only fallocate problem in gluster because of 'iatt->ia_block' tweaking. It also breaks the --keep-size option (i.e. the FALLOC_FL_KEEP_SIZE flag in fallocate(2)) and reports incorrect du size. Regards, Ravi OR 2) Once the fallocate fails in the backend filesystem, make posix xlator in the brick truncate the file to the previous size of the file before attempting fallocate. A patch [2] has been sent for this. But there is an issue with this when there are parallel writes and fallocate operations happening on the same file. It can lead to a data loss. a) statpre is obtained ===> before fallocate is attempted, get the stat hence the size of the file b) A parrallel Write fop on the same file that extends the file is successful c) Fallocate fails d) ftruncate truncates it to size given by statpre (i.e. the previous stat and the size obtained in step a) OR 3) Make posix check for available disk size before doing fallocate. i.e. in fallocate once posix gets the number of bytes to be allocated for the file from a particular offset, it checks whether so many bytes are available or not in the disk. If not, fail the fallocate fop with ENOSPC (without attempting it on the backend filesystem). There still is a probability of a parallel write happening while this fallocate is happening and by the time falllocate system call is attempted on the disk, the available space might have been less than what was calculated before fallocate. i.e. following things can happen a) statfs ===> get the available space of the backend filesystem b) a parallel write succeeds and extends the file c) fallocate is attempted assuming there is sufficient space in the backend While the above situation can arise, I think we are still fine. Because fallocate is attempted from the offset received in the fop. So, irrespective of whether write extended the file or not, the fallocate itself will be attempted for so many bytes from the offset which we found to be available by getting statfs information. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1724754#c3 [2] https://review.gluster.org/#/c/glusterfs/+/22969/ Please provide feedback. Regards, Raghavendra ___ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/836554017 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge:
[Gluster-devel] fallocate behavior in glusterfs
Hi All, In glusterfs, there is an issue regarding the fallocate behavior. In short, if someone does fallocate from the mount point with some size that is greater than the available size in the backend filesystem where the file is present, then fallocate can fail with a subset of the required number of blocks allocated and then failing in the backend filesystem with ENOSPC error. The behavior of fallocate in itself is simlar to how it would have been on a disk filesystem (atleast xfs where it was checked). i.e. allocates subset of the required number of blocks and then fail with ENOSPC. And the file in itself would show the number of blocks in stat to be whatever was allocated as part of fallocate. Please refer [1] where the issue is explained. Now, there is one small difference between how the behavior is between glusterfs and xfs. In xfs after fallocate fails, doing 'stat' on the file shows the number of blocks that have been allocated. Whereas in glusterfs, the number of blocks is shown as zero which makes tools like "du" show zero consumption. This difference in behavior in glusterfs is because of libglusterfs on how it handles sparse files etc for calculating number of blocks (mentioned in [1]) At this point I can think of 3 things on how to handle this. 1) Except for how many blocks are shown in the stat output for the file from the mount point (on which fallocate was done), the remaining behavior of attempting to allocate the requested size and failing when the filesystem becomes full is similar to that of XFS. Hence, what is required is to come up with a solution on how libglusterfs calculate blocks for sparse files etc (without breaking any of the existing components and features). This makes the behavior similar to that of backend filesystem. This might require its own time to fix libglusterfs logic without impacting anything else. OR 2) Once the fallocate fails in the backend filesystem, make posix xlator in the brick truncate the file to the previous size of the file before attempting fallocate. A patch [2] has been sent for this. But there is an issue with this when there are parallel writes and fallocate operations happening on the same file. It can lead to a data loss. a) statpre is obtained ===> before fallocate is attempted, get the stat hence the size of the file b) A parrallel Write fop on the same file that extends the file is successful c) Fallocate fails d) ftruncate truncates it to size given by statpre (i.e. the previous stat and the size obtained in step a) OR 3) Make posix check for available disk size before doing fallocate. i.e. in fallocate once posix gets the number of bytes to be allocated for the file from a particular offset, it checks whether so many bytes are available or not in the disk. If not, fail the fallocate fop with ENOSPC (without attempting it on the backend filesystem). There still is a probability of a parallel write happening while this fallocate is happening and by the time falllocate system call is attempted on the disk, the available space might have been less than what was calculated before fallocate. i.e. following things can happen a) statfs ===> get the available space of the backend filesystem b) a parallel write succeeds and extends the file c) fallocate is attempted assuming there is sufficient space in the backend While the above situation can arise, I think we are still fine. Because fallocate is attempted from the offset received in the fop. So, irrespective of whether write extended the file or not, the fallocate itself will be attempted for so many bytes from the offset which we found to be available by getting statfs information. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1724754#c3 [2] https://review.gluster.org/#/c/glusterfs/+/22969/ Please provide feedback. Regards, Raghavendra ___ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/836554017 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/486278655 Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
