Re: [Users] ploop on ext4 without journal - bug?
Dima, could you help? On 02:08 Wed 30 Sep , Nick Knutov wrote: > Hello all, > > I have an ext4 partition without journal (I need it so): > > mount | grep vz2 > /dev/sde1 on /vz2 type ext4 > (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) > > debugfs -R features /dev/sde1 > debugfs 1.41.12 (17-May-2010) > Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg > sparse_super large_file huge_file uninit_bg dir_nlink extra_isize > > When I'm trying to create CT with ploop layout - I've got > > Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K > Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 size=20971520 > sectors v2 > Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml > WARNING: /vz2 is mounted with data=writeback not recommended for ploop; > please use data=ordered instead > Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd > Adding delta dev=/dev/ploop58376 img=/vz2/private/2008.tmp/root.hdd/root.hdd > (rw) > > and now it freezes. (btw, vzctl says it's data=writeback, but it's > data=journal and journal is removed - is it ok?) > > > When ctrl+c I've got: > > ^C > Cancelling... > Cancelling... > Destroying container private area: /vz2/private/2008.tmp > ^C > Cancelling... > Cancelling... > > so I have to log in other ssh session and kill -9 it. > > Kernel: 042stab108.8 > > Is it a bug or I'm doing something wrong? > > -- > Best Regards, > Nick Knutov > http://knutov.com > ICQ: 272873706 > Voice: +7-904-84-23-130 > > ___ > Users mailing list > Users@openvz.org > https://lists.openvz.org/mailman/listinfo/users ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] ploop on ext4 without journal - bug?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thanks, yes, I'm using SSDs. Partition was tune2fs -O ^has_journal /dev/sdX so I thought the journal was removed completely and data= section is not important at all. Ok, what is the right way to fix it for me now? Will remount with data=ordered (and still tune2fs -O ^has_journal) be fine? Was that fixed bug already compiled and sent to yum repository (package ploop I suppose) ? 07.10.2015 17:03, Dmitry Monakhov пишет: > Sergey Bronnikovwrites: > >> Dima, could you help? >> >> On 02:08 Wed 30 Sep , Nick Knutov wrote: >>> Hello all, >>> >>> I have an ext4 partition without journal (I need it so): > First of all. The subject you mentioned is incorrect. This is not > nojournal mode. Configuration you want to create is external journal with data=journal. > > data=journal is full data journaling mode. Such mode assumes that it > will pass through journal all data, but ploop directly issues bios to > lower-fs(i.e. baypass journal). This done for performance reasons. That > is why ploop is faster that any other solutions. > All this means that full journaling for lower(/vz/private) fs is not > compatible with ploop. So please do not use it, otherwise you'll get > undefined behavior (most likely silent corruptions in guest-fs) > > The glitch you have mentioned most likely happen due to the fact that > you use SSD. Recently we have found a bug in mm reclaim code which > result in deadlock (swap on ssd in our case) https://jira.sw.ru/browse/PSBM-39335 > > Bug was fixed here: > *diff-ms-mm-vmscan-do-not-wait-for-page-writeback-for-GFP_NOFS-allocations > Added to 042stab112_3 > > mm, vmscan: Do not wait for page writeback for GFP_NOFS > Backport of mainline patch ecf5fc6e9654 > >>> >>> mount | grep vz2 >>> /dev/sde1 on /vz2 type ext4 (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) >>> >>> debugfs -R features /dev/sde1 >>> debugfs 1.41.12 (17-May-2010) >>> Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize >>> >>> When I'm trying to create CT with ploop layout - I've got >>> >>> Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K >>> Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 size=20971520 sectors v2 >>> Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml >>> WARNING: /vz2 is mounted with data=writeback not recommended for ploop; please use data=ordered instead >>> Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd >>> Adding delta dev=/dev/ploop58376 img=/vz2/private/2008.tmp/root.hdd/root.hdd (rw) >>> >>> and now it freezes. (btw, vzctl says it's data=writeback, but it's >>> data=journal and journal is removed - is it ok?) >>> >>> >>> When ctrl+c I've got: >>> >>> ^C >>> Cancelling... >>> Cancelling... >>> Destroying container private area: /vz2/private/2008.tmp >>> ^C >>> Cancelling... >>> Cancelling... >>> >>> so I have to log in other ssh session and kill -9 it. >>> >>> Kernel: 042stab108.8 >>> >>> Is it a bug or I'm doing something wrong? >>> >>> -- >>> Best Regards, >>> Nick Knutov >>> http://knutov.com >>> ICQ: 272873706 >>> Voice: +7-904-84-23-130 >>> >> >>> ___ >>> Users mailing list >>> Users@openvz.org >>> https://lists.openvz.org/mailman/listinfo/users - -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJWFT2CAAoJELne4KEUgITt1GMH/2Xys5rse+SK1+vH/NbP6Lbs UbiLBMpti3btEKJh8UkUb3QTnTvHeSQT43m6o27jmG4ZuSG0m8Phf+DSlcl7FsCc OuTU4rY6lFQOdsDibsFputyNf1cb0y7pKZoTQZMg/UWouVN8+n7y24FHnq7mWgQl unwGhMq0fi/MGBjakZ3QRJ5NO5VchSLtKajVIBNXC40TCICL+0mxIU0IblcBJIXH PvjB7w1bXsWRFXmm3poK5AZj880ULR0qw11gS9GBhCKOtiyFmKlMsMEknEPbbFS+ vd/ehyD/4DHoEju6KEQDfPt+XAbG8CxffgSqoMkvfit9eFC8GYTwan3xWTtdVMk= =5DRi -END PGP SIGNATURE- ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] ploop on ext4 without journal - bug?
Thanks, yes, I'm using SSDs. Partition was tune2fs -O ^has_journal /dev/sdX so I thought the journal was removed completely and data= section is not important at all. Ok, what is the right way to fix it for me now? Will remount with data=ordered (and still tune2fs -O ^has_journal) be fine? Was that fixed bug already compiled and sent to yum repository (package ploop I suppose) ? 07.10.2015 17:03, Dmitry Monakhov пишет: > Sergey Bronnikovwrites: > >> Dima, could you help? >> >> On 02:08 Wed 30 Sep , Nick Knutov wrote: >>> Hello all, >>> >>> I have an ext4 partition without journal (I need it so): > First of all. The subject you mentioned is incorrect. This is not > nojournal mode. Configuration you want to create is external journal with > data=journal. > > data=journal is full data journaling mode. Such mode assumes that it > will pass through journal all data, but ploop directly issues bios to > lower-fs(i.e. baypass journal). This done for performance reasons. That > is why ploop is faster that any other solutions. > All this means that full journaling for lower(/vz/private) fs is not > compatible with ploop. So please do not use it, otherwise you'll get > undefined behavior (most likely silent corruptions in guest-fs) > > The glitch you have mentioned most likely happen due to the fact that > you use SSD. Recently we have found a bug in mm reclaim code which > result in deadlock (swap on ssd in our case) > https://jira.sw.ru/browse/PSBM-39335 > > Bug was fixed here: > *diff-ms-mm-vmscan-do-not-wait-for-page-writeback-for-GFP_NOFS-allocations > Added to 042stab112_3 > > mm, vmscan: Do not wait for page writeback for GFP_NOFS > Backport of mainline patch ecf5fc6e9654 > >>> mount | grep vz2 >>> /dev/sde1 on /vz2 type ext4 >>> (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) >>> >>> debugfs -R features /dev/sde1 >>> debugfs 1.41.12 (17-May-2010) >>> Filesystem features: ext_attr resize_inode dir_index filetype extent >>> flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize >>> >>> When I'm trying to create CT with ploop layout - I've got >>> >>> Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K >>> Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 >>> size=20971520 sectors v2 >>> Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml >>> WARNING: /vz2 is mounted with data=writeback not recommended for ploop; >>> please use data=ordered instead >>> Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd >>> Adding delta dev=/dev/ploop58376 >>> img=/vz2/private/2008.tmp/root.hdd/root.hdd (rw) >>> >>> and now it freezes. (btw, vzctl says it's data=writeback, but it's >>> data=journal and journal is removed - is it ok?) >>> >>> >>> When ctrl+c I've got: >>> >>> ^C >>> Cancelling... >>> Cancelling... >>> Destroying container private area: /vz2/private/2008.tmp >>> ^C >>> Cancelling... >>> Cancelling... >>> >>> so I have to log in other ssh session and kill -9 it. >>> >>> Kernel: 042stab108.8 >>> >>> Is it a bug or I'm doing something wrong? >>> >>> -- >>> Best Regards, >>> Nick Knutov >>> http://knutov.com >>> ICQ: 272873706 >>> Voice: +7-904-84-23-130 >>> >>> ___ >>> Users mailing list >>> Users@openvz.org >>> https://lists.openvz.org/mailman/listinfo/users -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] ploop on ext4 without journal - bug?
Do I understand right - after all patches (in a little bit future) ploop should work fine on ext4 without journal with any data= mount option? And if I have journal - I should use data=ordered to avoid silent bugs? If ext4 has journal - there is known "feature" with mysql - mysql is much faster (near two times in my tests) if mount options are "data=journal,journal_async_commit" (and may be with O_DIRECT flag) compared to data=ordered. How to configure (if it's possible) such setup in case when mysql is inside CT with ploop on ext4 with journal and data=ordered? 07.10.2015 21:05, Dmitry Monakhov пишет: Nick Knutovwrites: yes, I'm using SSDs. Partition was tune2fs -O ^has_journal /dev/sdX so I thought the journal was removed completely and data= section is not important at all. WOW.. This is hilarious. Indeed even w/o journal ext4 show journal related options in /proc/mounts. This is bug(minor, but still). I'll prepare patch for mainstream. Ok, what is the right way to fix it for me now? Ok. If you want to run your host w/o journal than it is ok. We do not test such configuration, but it does not contradict to any assumptions. Will remount with data=ordered (and still tune2fs -O ^has_journal) be fine? No you do not have to modify /etc/fstab Was that fixed bug already compiled and sent to yum repository (package ploop I suppose) ? This was kernel's issue. Just update your kernel to most recent one (042stab112_3 or higher) #yum update vzkernel 07.10.2015 17:03, Dmitry Monakhov пишет: Sergey Bronnikov writes: Dima, could you help? On 02:08 Wed 30 Sep , Nick Knutov wrote: Hello all, I have an ext4 partition without journal (I need it so): First of all. The subject you mentioned is incorrect. This is not nojournal mode. Configuration you want to create is external journal with data=journal. data=journal is full data journaling mode. Such mode assumes that it will pass through journal all data, but ploop directly issues bios to lower-fs(i.e. baypass journal). This done for performance reasons. That is why ploop is faster that any other solutions. All this means that full journaling for lower(/vz/private) fs is not compatible with ploop. So please do not use it, otherwise you'll get undefined behavior (most likely silent corruptions in guest-fs) The glitch you have mentioned most likely happen due to the fact that you use SSD. Recently we have found a bug in mm reclaim code which result in deadlock (swap on ssd in our case) https://jira.sw.ru/browse/PSBM-39335 Bug was fixed here: *diff-ms-mm-vmscan-do-not-wait-for-page-writeback-for-GFP_NOFS-allocations Added to 042stab112_3 mm, vmscan: Do not wait for page writeback for GFP_NOFS Backport of mainline patch ecf5fc6e9654 mount | grep vz2 /dev/sde1 on /vz2 type ext4 (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) debugfs -R features /dev/sde1 debugfs 1.41.12 (17-May-2010) Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize When I'm trying to create CT with ploop layout - I've got Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 size=20971520 sectors v2 Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml WARNING: /vz2 is mounted with data=writeback not recommended for ploop; please use data=ordered instead Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd Adding delta dev=/dev/ploop58376 img=/vz2/private/2008.tmp/root.hdd/root.hdd (rw) and now it freezes. (btw, vzctl says it's data=writeback, but it's data=journal and journal is removed - is it ok?) When ctrl+c I've got: ^C Cancelling... Cancelling... Destroying container private area: /vz2/private/2008.tmp ^C Cancelling... Cancelling... so I have to log in other ssh session and kill -9 it. Kernel: 042stab108.8 Is it a bug or I'm doing something wrong? -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users - -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (MingW32) iQEcBAEBAgAGBQJWFT2CAAoJELne4KEUgITt1GMH/2Xys5rse+SK1+vH/NbP6Lbs UbiLBMpti3btEKJh8UkUb3QTnTvHeSQT43m6o27jmG4ZuSG0m8Phf+DSlcl7FsCc OuTU4rY6lFQOdsDibsFputyNf1cb0y7pKZoTQZMg/UWouVN8+n7y24FHnq7mWgQl unwGhMq0fi/MGBjakZ3QRJ5NO5VchSLtKajVIBNXC40TCICL+0mxIU0IblcBJIXH PvjB7w1bXsWRFXmm3poK5AZj880ULR0qw11gS9GBhCKOtiyFmKlMsMEknEPbbFS+ vd/ehyD/4DHoEju6KEQDfPt+XAbG8CxffgSqoMkvfit9eFC8GYTwan3xWTtdVMk= =5DRi -END PGP SIGNATURE- -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 ___ Users mailing list
Re: [Users] ploop on ext4 without journal - bug?
Nick Knutovwrites: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Thanks, > > yes, I'm using SSDs. > > Partition was > tune2fs -O ^has_journal /dev/sdX > so I thought the journal was removed completely and data= section is not > important at all. WOW.. This is hilarious. Indeed even w/o journal ext4 show journal related options in /proc/mounts. This is bug(minor, but still). I'll prepare patch for mainstream. > > Ok, what is the right way to fix it for me now? Ok. If you want to run your host w/o journal than it is ok. We do not test such configuration, but it does not contradict to any assumptions. > Will > remount with data=ordered (and still tune2fs -O ^has_journal) > be fine? No you do not have to modify /etc/fstab > > Was that fixed bug already compiled and sent to yum repository (package > ploop I suppose) ? This was kernel's issue. Just update your kernel to most recent one (042stab112_3 or higher) #yum update vzkernel > > > 07.10.2015 17:03, Dmitry Monakhov пишет: >> Sergey Bronnikov writes: >> >>> Dima, could you help? >>> >>> On 02:08 Wed 30 Sep , Nick Knutov wrote: Hello all, I have an ext4 partition without journal (I need it so): >> First of all. The subject you mentioned is incorrect. This is not >> nojournal mode. Configuration you want to create is external journal > with data=journal. >> >> data=journal is full data journaling mode. Such mode assumes that it >> will pass through journal all data, but ploop directly issues bios to >> lower-fs(i.e. baypass journal). This done for performance reasons. That >> is why ploop is faster that any other solutions. >> All this means that full journaling for lower(/vz/private) fs is not >> compatible with ploop. So please do not use it, otherwise you'll get >> undefined behavior (most likely silent corruptions in guest-fs) >> >> The glitch you have mentioned most likely happen due to the fact that >> you use SSD. Recently we have found a bug in mm reclaim code which >> result in deadlock (swap on ssd in our case) > https://jira.sw.ru/browse/PSBM-39335 >> >> Bug was fixed here: >> *diff-ms-mm-vmscan-do-not-wait-for-page-writeback-for-GFP_NOFS-allocations >> Added to 042stab112_3 >> >> mm, vmscan: Do not wait for page writeback for GFP_NOFS >> Backport of mainline patch ecf5fc6e9654 >> mount | grep vz2 /dev/sde1 on /vz2 type ext4 > (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) debugfs -R features /dev/sde1 debugfs 1.41.12 (17-May-2010) Filesystem features: ext_attr resize_inode dir_index filetype extent > flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize When I'm trying to create CT with ploop layout - I've got Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 > size=20971520 sectors v2 Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml WARNING: /vz2 is mounted with data=writeback not recommended for > ploop; please use data=ordered instead Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd Adding delta dev=/dev/ploop58376 > img=/vz2/private/2008.tmp/root.hdd/root.hdd (rw) and now it freezes. (btw, vzctl says it's data=writeback, but it's data=journal and journal is removed - is it ok?) When ctrl+c I've got: ^C Cancelling... Cancelling... Destroying container private area: /vz2/private/2008.tmp ^C Cancelling... Cancelling... so I have to log in other ssh session and kill -9 it. Kernel: 042stab108.8 Is it a bug or I'm doing something wrong? -- Best Regards, Nick Knutov http://knutov.com ICQ: 272873706 Voice: +7-904-84-23-130 >>> ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users > > - -- > Best Regards, > Nick Knutov > http://knutov.com > ICQ: 272873706 > Voice: +7-904-84-23-130 > -BEGIN PGP SIGNATURE- > Version: GnuPG v2.0.22 (MingW32) > > iQEcBAEBAgAGBQJWFT2CAAoJELne4KEUgITt1GMH/2Xys5rse+SK1+vH/NbP6Lbs > UbiLBMpti3btEKJh8UkUb3QTnTvHeSQT43m6o27jmG4ZuSG0m8Phf+DSlcl7FsCc > OuTU4rY6lFQOdsDibsFputyNf1cb0y7pKZoTQZMg/UWouVN8+n7y24FHnq7mWgQl > unwGhMq0fi/MGBjakZ3QRJ5NO5VchSLtKajVIBNXC40TCICL+0mxIU0IblcBJIXH > PvjB7w1bXsWRFXmm3poK5AZj880ULR0qw11gS9GBhCKOtiyFmKlMsMEknEPbbFS+ > vd/ehyD/4DHoEju6KEQDfPt+XAbG8CxffgSqoMkvfit9eFC8GYTwan3xWTtdVMk= > =5DRi > -END PGP SIGNATURE- signature.asc Description: PGP signature ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users
Re: [Users] ploop on ext4 without journal - bug?
Sergey Bronnikovwrites: > Dima, could you help? > > On 02:08 Wed 30 Sep , Nick Knutov wrote: >> Hello all, >> >> I have an ext4 partition without journal (I need it so): First of all. The subject you mentioned is incorrect. This is not nojournal mode. Configuration you want to create is external journal with data=journal. data=journal is full data journaling mode. Such mode assumes that it will pass through journal all data, but ploop directly issues bios to lower-fs(i.e. baypass journal). This done for performance reasons. That is why ploop is faster that any other solutions. All this means that full journaling for lower(/vz/private) fs is not compatible with ploop. So please do not use it, otherwise you'll get undefined behavior (most likely silent corruptions in guest-fs) The glitch you have mentioned most likely happen due to the fact that you use SSD. Recently we have found a bug in mm reclaim code which result in deadlock (swap on ssd in our case) https://jira.sw.ru/browse/PSBM-39335 Bug was fixed here: *diff-ms-mm-vmscan-do-not-wait-for-page-writeback-for-GFP_NOFS-allocations Added to 042stab112_3 mm, vmscan: Do not wait for page writeback for GFP_NOFS Backport of mainline patch ecf5fc6e9654 >> >> mount | grep vz2 >> /dev/sde1 on /vz2 type ext4 >> (rw,relatime,discard,errors=remount-ro,commit=20,data=journal,journal_async_commit) >> >> debugfs -R features /dev/sde1 >> debugfs 1.41.12 (17-May-2010) >> Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg >> sparse_super large_file huge_file uninit_bg dir_nlink extra_isize >> >> When I'm trying to create CT with ploop layout - I've got >> >> Creating image: /vz2/private/2008.tmp/root.hdd/root.hdd size=10485760K >> Creating delta /vz2/private/2008.tmp/root.hdd/root.hdd bs=2048 size=20971520 >> sectors v2 >> Storing /vz2/private/2008.tmp/root.hdd/DiskDescriptor.xml >> WARNING: /vz2 is mounted with data=writeback not recommended for ploop; >> please use data=ordered instead >> Opening delta /vz2/private/2008.tmp/root.hdd/root.hdd >> Adding delta dev=/dev/ploop58376 img=/vz2/private/2008.tmp/root.hdd/root.hdd >> (rw) >> >> and now it freezes. (btw, vzctl says it's data=writeback, but it's >> data=journal and journal is removed - is it ok?) >> >> >> When ctrl+c I've got: >> >> ^C >> Cancelling... >> Cancelling... >> Destroying container private area: /vz2/private/2008.tmp >> ^C >> Cancelling... >> Cancelling... >> >> so I have to log in other ssh session and kill -9 it. >> >> Kernel: 042stab108.8 >> >> Is it a bug or I'm doing something wrong? >> >> -- >> Best Regards, >> Nick Knutov >> http://knutov.com >> ICQ: 272873706 >> Voice: +7-904-84-23-130 >> > >> ___ >> Users mailing list >> Users@openvz.org >> https://lists.openvz.org/mailman/listinfo/users signature.asc Description: PGP signature ___ Users mailing list Users@openvz.org https://lists.openvz.org/mailman/listinfo/users