Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

> Anybody? Did I not submit this properly? You never sent the final patch (using 4K blocks)? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

Anybody? Did I not submit this properly? On Tue, Dec 13, 2016 at 10:13 PM, James Lavoy wrote: > Sorry to resurrect an old thread here, but I was expecting this to get > implemented by now. Have I not done something I need to to get some form of > this patch merged? LVM

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

Sorry to resurrect an old thread here, but I was expecting this to get implemented by now. Have I not done something I need to to get some form of this patch merged? LVM Migrations continue to be unnecessarily slow. On Wed, Sep 7, 2016 at 1:32 AM, James Lavoy wrote: > I'm

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

I'm okay with 4K. Again, anything other than default is fine by me. On Thu, Aug 25, 2016 at 3:55 AM, Wolfgang Bumiller wrote: > On Wed, Aug 24, 2016 at 07:19:04PM +0200, Dietmar Maurer wrote: > > I am afraid that increasing block size is bad for sparse file detection. >

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

On Wed, Aug 24, 2016 at 07:19:04PM +0200, Dietmar Maurer wrote: > I am afraid that increasing block size is bad for sparse file detection. > > Seems dd simply use "bs" for for memory buffer and zero block detection? The speed values for 4k look acceptable though? Might be a good compromise?

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

I am afraid that increasing block size is bad for sparse file detection. Seems dd simply use "bs" for for memory buffer and zero block detection? ___ pve-devel mailing list pve-devel@pve.proxmox.com

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

On Wed, 24 Aug 2016 18:59:29 +0200 (CEST) Dietmar Maurer wrote: > > Seems 64k is a good value. > > What value do you get with 4K? > mir@esx1:~$ time dd if=/tmp/big.file bs=4K |ssh esx2 "dd of=big.file bs=4k" 262144+0 records in 262144+0 records out 1073741824 bytes (1.1

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

> Seems 64k is a good value. What value do you get with 4K? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

On Wed, 24 Aug 2016 09:45:39 +0200 Michael Rasmussen wrote: > I can test on infiniband when I get home today. > mir@esx1:~$ time dd if=/tmp/big.file bs=32K |ssh esx2 "dd of=1GB.file bs=32k"32768+0 records in 32768+0 records out 1073741824 bytes (1.1 GB) copied, 3.75128 s, 286

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

I can test on infiniband when I get home today. On August 24, 2016 9:35:35 AM GMT+02:00, Alexandre DERUMIER wrote: >>>64k got similar speeds. > >Do you use 1gigabit network link ? (as it seem to be around 100MB/S >max) > >- Mail original - >De: "jlavoy"

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

>>64k got similar speeds. Do you use 1gigabit network link ? (as it seem to be around 100MB/S max) - Mail original - De: "jlavoy" À: "pve-devel" Envoyé: Mardi 23 Août 2016 19:54:31 Objet: Re: [pve-devel] [PATCH] Increases the dd read/write

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

I don't feel super strongly about what bs is used, from where I'm sitting, whatever the community wants is fine, provided it's higher than default. 64k got similar speeds. If we wanna go that low that's fine by me: abrams Code # time dd if=1GB.file bs=64K |ssh mary "dd of=/root/1GB.file bs=64K"

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

Default block equals logical block size so either 512B or 4k. On August 23, 2016 12:16:49 PM GMT+02:00, Alexandre DERUMIER wrote: >>>I think a value which can be handle by disk cache is ok. (1M should >be large enough, if the disk can do 200iops, it's around 200MB/S) >

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

>>I think a value which can be handle by disk cache is ok. (1M should be large >>enough, if the disk can do 200iops, it's around 200MB/S) Another thing to check , is the sparse option. If the block size is too high, the are more chance that they are datas inside the block, so it'll be full

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

>>IMHO a block size of 64MB is a bit too large. I guess you can see the same >>speed improvements with much smaller sizes - mabe 1M is large enough? I think a value which can be handle by disk cache is ok. (1M should be large enough, if the disk can do 200iops, it's around 200MB/S) - Mail

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

Totally up to you guys. Anything higher than default is a marked improvement. I got similar speed results doing the same test with 1M. abrams Code # time dd if=1GB.file bs=1M |ssh mary "dd of=/root/1GB.file bs=1M" 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 10.002 s,

Re: [pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

> + run_command([["dd", "if=$src", "bs=64M"],["/usr/bin/ssh", > "root\@${target_host}", > + "dd", 'conv=sparse', "of=$dst", "bs=64M"]]); IMHO a block size of 64MB is a bit too large. I guess you can see the same speed improvements with much smaller sizes

[pve-devel] [PATCH] Increases the dd read/write speed when migrating LVM disks. This should bring LVM migration speed up to match that of rsyncing qcows.

For example: ``` abrams Code # time dd if=1GB.file |ssh mary "dd of=/root/1GB.file" 2097152+0 records in 2097152+0 records out 1073741824 bytes (1.1 GB) copied, 34.0764 s, 31.5 MB/s 2097152+0 records in 2097152+0 records out 1073741824 bytes (1.1 GB) copied, 33.9548 s, 31.6 MB/s real0m34.079s