Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
On Fri, Oct 14, 2022 at 10:28:28PM +0200, Roberto Fastec wrote: > TIP and HINT > forget SSDs with LVM unless of enterprise level > especially if you are going to use/implement the thin provisioning As a user and developer of Qubes OS, this makes me nervous. Qubes OS uses LVM heavily in the default configuration, and is designed for end-user systems. -- Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab signature.asc Description: PGP signature ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
TIP and HINT forget SSDs with LVM unless of enterprise level especially if you are going to use/implement the thin provisioning How to identify an SSD of enterprise level: it costs from 1,00 euro per gigabyte up to 1,50 euro per gigabyte Kind regards Roberto Gini Technical Manager @ www.RecuperoDatiRAIDFAsTec.it Ottieni BlueMail per Android Il giorno 14 ott 2022, 21:50, alle ore 21:50, Roger Heflin ha scritto: >What is the underlying disk hardware you are running this on? >virtual, spinning, ssd, nvme? > >On Thu, Oct 13, 2022 at 2:01 AM Pawan Sharma > wrote: >> >> adding this to lvm-devel mailing list also. >> >> Regards, >> Pawan >> >> From: Pawan Sharma >> Sent: Wednesday, October 12, 2022 10:42 PM >> To: linux-lvm@redhat.com >> Cc: Mitta Sai Chaithanya ; Kapil Upadhayay > >> Subject: LVM2 : performance drop even after deleting the snapshot >> >> Hi Everyone, >> >> >> We are evaluating lvm2 snapshots and doing performance testing on it. >This is what we are doing : >> >> dump some data to lvm2 volume (using fio) >> take the snapshot >> delete the snapshot (no IOs anywhere after creating the snapshot) >> run the fio on lvm2 volume >> >> Here as you can see, we are just creating the snapshot and >immediately deleting it. There are no IOs to the main volume or >anywhere. When we run the fio after this (step 4) and we see around 50% >drop in performance with reference to the number we get in step 1. >> >> It is expected to see a performance drop if there is a snapshot >because of the COW. But here we deleted the snapshot, and it is not >referring to any data also. We should not see any performance drop >here. >> >> Could someone please help me understand this behavior. Why are we >seeing the performance drop in this case? It seems like we deleted the >snapshot but still it is not deleted, and we are paying the COW >penalty. >> >> System Info: >> >> OS : ubuntu 18.04 >> Kernel : 5.4.0 >> >> # lvm version >> LVM version: 2.02.176(2) (2017-11-03) >> Library version: 1.02.145 (2017-11-03) >> Driver version: 4.41.0 >> >> We also tried on latest ubuntu with newer version of LVM. We got the >same behavior. >> >> Any help/pointers would be appreciated. Thanks in advance. >> >> Regards, >> Pawan >> ___ >> linux-lvm mailing list >> linux-lvm@redhat.com >> https://listman.redhat.com/mailman/listinfo/linux-lvm >> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > >___ >linux-lvm mailing list >linux-lvm@redhat.com >https://listman.redhat.com/mailman/listinfo/linux-lvm >read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
The thin volume was created on a block disk provided by Microsoft Azure Managed disk provisioned on premium SSD which is ready to be used in production environment. The disk can provide sustained Max IOPs at 5000. Thanks, Kapil Upadhayay. From: Roberto Fastec Sent: Saturday, October 15, 2022 1:58 AM To: LVM general discussion and development Cc: Kapil Upadhayay ; Mitta Sai Chaithanya ; lvm-de...@redhat.com Subject: [EXTERNAL] Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot Some people who received this message don't often get email from roberto.fas...@gmail.com<mailto:roberto.fas...@gmail.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> TIP and HINT forget SSDs with LVM unless of enterprise level especially if you are going to use/implement the thin provisioning How to identify an SSD of enterprise level: it costs from 1,00 euro per gigabyte up to 1,50 euro per gigabyte Kind regards Roberto Gini Technical Manager @ www.RecuperoDatiRAIDFAsTec.it<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.recuperodatiraidfastec.it%2F=05%7C01%7Ckupadhayay%40microsoft.com%7Ccd9e45aadb0c4dd0a03508daae22aa91%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013761207553033%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=9i73rMTiPlvD6jlcxLti7Pvrku7jE%2BLciLP57umpwpA%3D=0> Ottieni BlueMail per Android<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbluemail.me%2F=05%7C01%7Ckupadhayay%40microsoft.com%7Ccd9e45aadb0c4dd0a03508daae22aa91%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013761207553033%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=hGhews%2FSF8d7zCJvRzIOs24CE0xLHH1NqyWjL3IqthI%3D=0> Il giorno 14 ott 2022, alle ore 21:50, Roger Heflin mailto:rogerhef...@gmail.com>> ha scritto: What is the underlying disk hardware you are running this on? virtual, spinning, ssd, nvme? On Thu, Oct 13, 2022 at 2:01 AM Pawan Sharma mailto:sharmapa...@microsoft.com>> wrote: adding this to lvm-devel mailing list also. Regards, Pawan From: Pawan Sharma Sent: Wednesday, October 12, 2022 10:42 PM To: linux-lvm@redhat.com<mailto:linux-lvm@redhat.com> mailto:linux-lvm@redhat.com>> Cc: Mitta Sai Chaithanya mailto:mit...@microsoft.com>>; Kapil Upadhayay mailto:kupadha...@microsoft.com>> Subject: LVM2 : performance drop even after deleting the snapshot Hi Everyone, We are evaluating lvm2 snapshots and doing performance testing on it. This is what we are doing : dump some data to lvm2 volume (using fio) take the snapshot delete the snapshot (no IOs anywhere after creating the snapshot) run the fio on lvm2 volume Here as you can see, we are just creating the snapshot and immediately deleting it. There are no IOs to the main volume or anywhere. When we run the fio after this (step 4) and we see around 50% drop in performance with reference to the number we get in step 1. It is expected to see a performance drop if there is a snapshot because of the COW. But here we deleted the snapshot, and it is not referring to any data also. We should not see any performance drop here. Could someone please help me understand this behavior. Why are we seeing the performance drop in this case? It seems like we deleted the snapshot but still it is not deleted, and we are paying the COW penalty. System Info: OS : ubuntu 18.04 Kernel : 5.4.0 # lvm version LVM version: 2.02.176(2) (2017-11-03) Library version: 1.02.145 (2017-11-03) Driver version: 4.41.0 We also tried on latest ubuntu with newer version of LVM. We got the same behavior. Any help/pointers would be appreciated. Thanks in advance. Regards, Pawan linux-lvm mailing list linux-lvm@redhat.com<mailto:linux-lvm@redhat.com> https://listman.redhat.com/mailman/listinfo/linux-lvm<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistman.redhat.com%2Fmailman%2Flistinfo%2Flinux-lvm=05%7C01%7Ckupadhayay%40microsoft.com%7Ccd9e45aadb0c4dd0a03508daae22aa91%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013761207553033%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=2HZwpc4MH0w7bCVwNuohJ7jG5naZaCNGdAJHwsIzsZg%3D=0> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftldp.org%2FHOWTO%2FLVM-HOWTO=05%7C01%7Ckupadhayay%40microsoft.com%7Ccd9e45aadb0c4dd0a03508daae22aa91%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638013761207553033%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=nMH03VV84Qe%2FkZ5g0hoBsZi53QKOYs%2B9CiTnPb6xu4s%3D=0>/
Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
What is the underlying disk hardware you are running this on? virtual, spinning, ssd, nvme? On Thu, Oct 13, 2022 at 2:01 AM Pawan Sharma wrote: > > adding this to lvm-devel mailing list also. > > Regards, > Pawan > > From: Pawan Sharma > Sent: Wednesday, October 12, 2022 10:42 PM > To: linux-lvm@redhat.com > Cc: Mitta Sai Chaithanya ; Kapil Upadhayay > > Subject: LVM2 : performance drop even after deleting the snapshot > > Hi Everyone, > > > We are evaluating lvm2 snapshots and doing performance testing on it. This is > what we are doing : > > dump some data to lvm2 volume (using fio) > take the snapshot > delete the snapshot (no IOs anywhere after creating the snapshot) > run the fio on lvm2 volume > > Here as you can see, we are just creating the snapshot and immediately > deleting it. There are no IOs to the main volume or anywhere. When we run the > fio after this (step 4) and we see around 50% drop in performance with > reference to the number we get in step 1. > > It is expected to see a performance drop if there is a snapshot because of > the COW. But here we deleted the snapshot, and it is not referring to any > data also. We should not see any performance drop here. > > Could someone please help me understand this behavior. Why are we seeing the > performance drop in this case? It seems like we deleted the snapshot but > still it is not deleted, and we are paying the COW penalty. > > System Info: > > OS : ubuntu 18.04 > Kernel : 5.4.0 > > # lvm version > LVM version: 2.02.176(2) (2017-11-03) > Library version: 1.02.145 (2017-11-03) > Driver version: 4.41.0 > > We also tried on latest ubuntu with newer version of LVM. We got the same > behavior. > > Any help/pointers would be appreciated. Thanks in advance. > > Regards, > Pawan > ___ > linux-lvm mailing list > linux-lvm@redhat.com > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
Dne 13. 10. 22 v 8:53 Pawan Sharma napsal(a): adding this to lvm-devel mailing list also. Regards, Pawan -- *From:* Pawan Sharma *Sent:* Wednesday, October 12, 2022 10:42 PM *To:* linux-lvm@redhat.com *Cc:* Mitta Sai Chaithanya ; Kapil Upadhayay *Subject:* LVM2 : performance drop even after deleting the snapshot Hi Everyone, We are evaluating lvm2 snapshots and doing performance testing on it. This is what we are doing : 1. dump some data to lvm2 volume (using fio) 2. take the snapshot 3. delete the snapshot (no IOs anywhere after creating the snapshot) 4. run the fio on lvm2 volume Here as you can see, we are just creating the snapshot and immediately deleting it. There are no IOs to the main volume or anywhere. When we run the fio after this (step 4) and we see around 50% drop in performance with reference to the number we get in step 1. It is expected to see a performance drop if there is a snapshot because of the COW. But here we deleted the snapshot, and it is not referring to any data also. We should not see any performance drop here. Could someone please help me understand this behavior. Why are we seeing the performance drop in this case? It seems like we deleted the snapshot but still it is not deleted, and we are paying the COW penalty. System Info: OS : ubuntu 18.04 Kernel : 5.4.0 # lvm version LVM version:2.02.176(2) (2017-11-03) Library version: 1.02.145 (2017-11-03) Driver version:4.41.0 We also tried on latest ubuntu with newer version of LVM. We got the same behavior. Hi Debugging 5 year old software is likely not going to get lot of attention from upstream. So please: a) reproduce the issue with some recent kernel & lvm2 b) take 'dmsetup table && dmsetup status' before you run every 'fio' test and present here your result in some form - otherwise we can hardly see what is the problem. What should be expected - if you use old/thick snapshots - when you 'drop' snapshot - you have your original intact LV - so results should mostly match results before you take the snapshot - but you clearly have to take into account if you use some 'SSD/NVMe' discarding and other things - so always run series of tests and average your results. If you use thin snapshot - that you can get various results depending on your settings of thin chunks, discard usage. Also maybe try your benchmark with different filesystems... Regards Zdenek ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
[linux-lvm] LVM2 : performance drop even after deleting the snapshot
Hi Everyone, We are evaluating lvm2 snapshots and doing performance testing on it. This is what we are doing : 1. dump some data to lvm2 volume (using fio) 2. take the snapshot 3. delete the snapshot (no IOs anywhere after creating the snapshot) 4. run the fio on lvm2 volume Here as you can see, we are just creating the snapshot and immediately deleting it. There are no IOs to the main volume or anywhere. When we run the fio after this (step 4) and we see around 50% drop in performance with reference to the number we get in step 1. It is expected to see a performance drop if there is a snapshot because of the COW. But here we deleted the snapshot, and it is not referring to any data also. We should not see any performance drop here. Could someone please help me understand this behavior. Why are we seeing the performance drop in this case? It seems like we deleted the snapshot but still it is not deleted, and we are paying the COW penalty. System Info: OS : ubuntu 18.04 Kernel : 5.4.0 # lvm version LVM version: 2.02.176(2) (2017-11-03) Library version: 1.02.145 (2017-11-03) Driver version: 4.41.0 We also tried on latest ubuntu with newer version of LVM. We got the same behavior. Any help/pointers would be appreciated. Thanks in advance. Regards, Pawan ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM2 : performance drop even after deleting the snapshot
adding this to lvm-devel mailing list also. Regards, Pawan From: Pawan Sharma Sent: Wednesday, October 12, 2022 10:42 PM To: linux-lvm@redhat.com Cc: Mitta Sai Chaithanya ; Kapil Upadhayay Subject: LVM2 : performance drop even after deleting the snapshot Hi Everyone, We are evaluating lvm2 snapshots and doing performance testing on it. This is what we are doing : 1. dump some data to lvm2 volume (using fio) 2. take the snapshot 3. delete the snapshot (no IOs anywhere after creating the snapshot) 4. run the fio on lvm2 volume Here as you can see, we are just creating the snapshot and immediately deleting it. There are no IOs to the main volume or anywhere. When we run the fio after this (step 4) and we see around 50% drop in performance with reference to the number we get in step 1. It is expected to see a performance drop if there is a snapshot because of the COW. But here we deleted the snapshot, and it is not referring to any data also. We should not see any performance drop here. Could someone please help me understand this behavior. Why are we seeing the performance drop in this case? It seems like we deleted the snapshot but still it is not deleted, and we are paying the COW penalty. System Info: OS : ubuntu 18.04 Kernel : 5.4.0 # lvm version LVM version: 2.02.176(2) (2017-11-03) Library version: 1.02.145 (2017-11-03) Driver version: 4.41.0 We also tried on latest ubuntu with newer version of LVM. We got the same behavior. Any help/pointers would be appreciated. Thanks in advance. Regards, Pawan ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
