Re: New experimental btrfs branch ready for testing
Chris Mason wrote: On Sat, Jun 06, 2009 at 11:38:45AM -0500, Steven Pratt wrote: No problem. Raid results are uploading to http://btrfs.boxacle.net/repository/raid/history/History.html now. There were massive improvements in the random write workloads, especially with cow enabled!! MailServer had moderate perf gains, but dramatic decrease in CPU utilization, so this is very good as well. The only regression I see is on large file creates, CPU is up 200% or more while performance is fairly flat. btrfs_tree_lock now dominates the profile. I'm not able to reproduce the btrfs_tree_lock usage that you're seeing. Could you please use the callgraph option to oprofile? Ok, back from vacation and have re-engaged my brain :-) Was thinking I would have to re-run this for you, but we already have callgraph data for all the runs. For the 128 thread create workload it is here: http://btrfs.boxacle.net/repository/raid//2-6-30-rc7-newformat/btrfs-6-2-newformat/btrfs1.ffsb.large_file_creates__threads_0128.09-06-04_01.23.30/analysis/oprofile.breakout.001/oprofile-callgraph Steve -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Sat, Jun 06, 2009 at 11:38:45AM -0500, Steven Pratt wrote: No problem. Raid results are uploading to http://btrfs.boxacle.net/repository/raid/history/History.html now. There were massive improvements in the random write workloads, especially with cow enabled!! MailServer had moderate perf gains, but dramatic decrease in CPU utilization, so this is very good as well. The only regression I see is on large file creates, CPU is up 200% or more while performance is fairly flat. btrfs_tree_lock now dominates the profile. I'm not able to reproduce the btrfs_tree_lock usage that you're seeing. Could you please use the callgraph option to oprofile? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On 1. juni. 2009, at 23.04, Chris Mason wrote: I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- unstable.git newformat # git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- unstable.git newformat fatal: Not a git repository For the progs: git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- progs-unstable.git newformat # git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- progs-unstable.git newformat fatal: Not a git repository Have this code been removed, or is it me doing something funny? roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Sun, Jun 07, 2009 at 01:50:27PM +0200, Roy Sigurd Karlsbakk wrote: On 1. juni. 2009, at 23.04, Chris Mason wrote: I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- unstable.git newformat # git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- unstable.git newformat fatal: Not a git repository For the progs: git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- progs-unstable.git newformat # git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs- progs-unstable.git newformat fatal: Not a git repository Have this code been removed, or is it me doing something funny? You're doing something funny. I'm guessing you don't already have a copy of the btrfs repositories, so you should be using clone instead of pull. If you do have them, cd into them. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
Chris Mason wrote: On Fri, Jun 05, 2009 at 04:27:55PM -0500, Steven Pratt wrote: Steven Pratt wrote: Chris Mason wrote: On Thu, Jun 04, 2009 at 02:02:20PM -0500, Steven Pratt wrote: Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. This is a large change, and I'm hoping to have it stable in time for the 2.6.31 merge window. I've been testing it for about a week now, and haven't been able to cause major problems yet. But, testing the compatibility with old format filesystems is the hard part, and everyone that pulls the new code should backup their data first. I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat So I started the performance runs on this. The base tests completed fine on the raid system and I will post results as soon as I can finish postprocessing, but when I tried to do nodatacow that machine it crashed pretty early. Here is console log: Hi Steve, Thanks again for hammering on these. Yan Zheng and I have both been trying to reproduce problems with nodatacow and with the database random write run. So now that the raid machine is actually up, I discovered it got further than I thought on nodatacow. It did all the read tests, but appeared to died on 16 thread random write(not odirect). There were no messages logged to var/log/messages at all. Last I saw was : Jun 4 03:14:24 btrfs1 kernel: [65856.065491] btrfs: setting nodatacow Jun 4 15:24:45 btrfs1 syslogd 1.4.1: restart. Just dead until we rebooted machine later that day. So the raid system complete the re-run of the nodatacow runs without error. So still no idea what happened on this box the first time around. As for the single disk system, it died during the random write test again, but it now looks like we might have a real HW failure. This time we see SCSI error messages. I have replaced the test disks and will try one more time. The net is, I would hold off digging too much into this as even I don't have any repeatable errors. Thanks for rerunning all of this, appreciate the update. No problem. Raid results are uploading to http://btrfs.boxacle.net/repository/raid/history/History.html now. There were massive improvements in the random write workloads, especially with cow enabled!! MailServer had moderate perf gains, but dramatic decrease in CPU utilization, so this is very good as well. The only regression I see is on large file creates, CPU is up 200% or more while performance is fairly flat. btrfs_tree_lock now dominates the profile. I am still having issues on the single disk system, which I am still not sure if it is btrfs or HW, but I am off on a family vacation tomorrow so it will have to wait for a week or so. Steve -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Thu, Jun 04, 2009 at 02:02:20PM -0500, Steven Pratt wrote: Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. This is a large change, and I'm hoping to have it stable in time for the 2.6.31 merge window. I've been testing it for about a week now, and haven't been able to cause major problems yet. But, testing the compatibility with old format filesystems is the hard part, and everyone that pulls the new code should backup their data first. I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat So I started the performance runs on this. The base tests completed fine on the raid system and I will post results as soon as I can finish postprocessing, but when I tried to do nodatacow that machine it crashed pretty early. Here is console log: Hi Steve, Thanks again for hammering on these. Yan Zheng and I have both been trying to reproduce problems with nodatacow and with the database random write run. But, so far we haven't been able to trigger any crashes.Do you see anything in your config or setup that is unusual? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
Steven Pratt wrote: Chris Mason wrote: On Thu, Jun 04, 2009 at 02:02:20PM -0500, Steven Pratt wrote: Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. This is a large change, and I'm hoping to have it stable in time for the 2.6.31 merge window. I've been testing it for about a week now, and haven't been able to cause major problems yet. But, testing the compatibility with old format filesystems is the hard part, and everyone that pulls the new code should backup their data first. I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat So I started the performance runs on this. The base tests completed fine on the raid system and I will post results as soon as I can finish postprocessing, but when I tried to do nodatacow that machine it crashed pretty early. Here is console log: Hi Steve, Thanks again for hammering on these. Yan Zheng and I have both been trying to reproduce problems with nodatacow and with the database random write run. So now that the raid machine is actually up, I discovered it got further than I thought on nodatacow. It did all the read tests, but appeared to died on 16 thread random write(not odirect). There were no messages logged to var/log/messages at all. Last I saw was : Jun 4 03:14:24 btrfs1 kernel: [65856.065491] btrfs: setting nodatacow Jun 4 15:24:45 btrfs1 syslogd 1.4.1: restart. Just dead until we rebooted machine later that day. So the raid system complete the re-run of the nodatacow runs without error. So still no idea what happened on this box the first time around. As for the single disk system, it died during the random write test again, but it now looks like we might have a real HW failure. This time we see SCSI error messages. I have replaced the test disks and will try one more time. The net is, I would hold off digging too much into this as even I don't have any repeatable errors. Steve But, so far we haven't been able to trigger any crashes.Do you see anything in your config or setup that is unusual? No, other than using the old mkfs with the new format. I've kicked off new runs to see if I hit the same issues Steve -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
Chris Mason wrote: On Thu, Jun 04, 2009 at 02:02:20PM -0500, Steven Pratt wrote: Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. This is a large change, and I'm hoping to have it stable in time for the 2.6.31 merge window. I've been testing it for about a week now, and haven't been able to cause major problems yet. But, testing the compatibility with old format filesystems is the hard part, and everyone that pulls the new code should backup their data first. I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat So I started the performance runs on this. The base tests completed fine on the raid system and I will post results as soon as I can finish postprocessing, but when I tried to do nodatacow that machine it crashed pretty early. Here is console log: Thanks Steve. Just to clarify, which commit was the head of your git tree when you ran these tests? Commit: e84a00105718b4979e7fd8ae089e539168317728 Tree: fe8f97183bc849451837bcae612a398184ffa354 Parent: ca4ec9c2ba713999c9446f6f1bdcea092ffc8e12 Author: Chris Mason chris.ma...@oracle.com Mon, 18 May 2009 10:41:58 -0400 Committer: Chris Mason chris.ma...@oracle.com Wed, 03 Jun 2009 09:19:04 -0400 Btrfs: fix metadata dirty throttling limits Steve -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Thu, Jun 04, 2009 at 02:19:00PM -0500, Steven Pratt wrote: I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat So I started the performance runs on this. The base tests completed fine on the raid system and I will post results as soon as I can finish postprocessing, but when I tried to do nodatacow that machine it crashed pretty early. Here is console log: Thanks Steve. Just to clarify, which commit was the head of your git tree when you ran these tests? Commit: e84a00105718b4979e7fd8ae089e539168317728 Tree: Ok, this is the right one, thanks. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Tue, Jun 02, 2009 at 09:28:30AM -0400, Chris Mason wrote: On Mon, Jun 01, 2009 at 05:04:47PM -0400, Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. Just a quick note that I'm having some issues with the backward compatibility code on 32 bit kernels. It can still read all the old items but it is having problems with creating new backrefs. 32bit is working fine on an entirely new format FS, and my 64 bit box can read and write the old format FS just fine. I'm hoping to track this one down today, but it would be a good idea to wait if you want to try the new code on old filesystems on 32 bit machines. If you do hit crashes, please don't immediately reformat your FS if you can avoid it. We should be able to fix most problems people hit. Looks like Yan Zheng tracked this down yesterday, Jens Axboe bravely tested out 32bit old format compat again with his laptop. At this point I think the new format code is looking pretty stable and it is generally ready for more testing. I've rebased the newformat kernel tree to fold in the corruption fixes. This way if anyone does a git bisect they won't end up on a commit that can corrupt their FS by accident. If you've already pulled the newformat tree, the new commits will conflict with the old. So, something like this will fix things if you have already pulled the newformat branch: git reset --hard v2.6.30-rc7 git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat If you've made your own commits or pulls other than just btrfs, different steps will be required. The btrfs-progs unstable tree was also rebased. Use git reset --hard ed20f5fc905145a0673097b539442d2a59491e77 on the progs tree if you've already pulled down the newformat branch. Happy testing everyone -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New experimental btrfs branch ready for testing
On Mon, Jun 01, 2009 at 05:04:47PM -0400, Chris Mason wrote: Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. Just a quick note that I'm having some issues with the backward compatibility code on 32 bit kernels. It can still read all the old items but it is having problems with creating new backrefs. 32bit is working fine on an entirely new format FS, and my 64 bit box can read and write the old format FS just fine. I'm hoping to track this one down today, but it would be a good idea to wait if you want to try the new code on old filesystems on 32 bit machines. If you do hit crashes, please don't immediately reformat your FS if you can avoid it. We should be able to fix most problems people hit. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
New experimental btrfs branch ready for testing
Hello everyone, Yan Zheng has been doing some major surgery to the back references and extent allocation code, tackling bottlenecks in the code that tracks extents. It scales better with many snapshots and performs better in the common case of no snapshots at all. THE NEW CODE IS A FORWARD ROLLING DISK FORMAT CHANGE. This means it is compatible with the current btrfs disk format, but once you mount a filesystem with the new code, it WILL NO LONGER BE MOUNTABLE FROM OLD KERNELS. Old kernels spit out an error message when you try them on new format filesystems. This is a large change, and I'm hoping to have it stable in time for the 2.6.31 merge window. I've been testing it for about a week now, and haven't been able to cause major problems yet. But, testing the compatibility with old format filesystems is the hard part, and everyone that pulls the new code should backup their data first. I've setup git branches called newformat where you can pull the new code. For the kernel (based on 2.6.30-rc7): git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git newformat For the progs: git pull git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git newformat The main benefit of the new code is that backrefs on the extent allocation tree use a fuzzier format. It basically means that we search for the key in the extent allocation tree instead of providing an exact backref to the parent block. This means we can predict how many blocks will be changed when changing the extent allocation tree, and it makes enospc much less complex. It is also significantly faster. For regular subvolume trees, a similar change is made as long as there are no snapshots against a given block. This is the common case, and it makes COW less expensive overall. Yan Zheng also worked out a way to free blocks during the transaction without needing to do an explicit snapshot deletion on the old root when the transaction was done. This gets rid of some complex caching code, and fixes worst-case problems where btrfs could take a very very long time to unmount. btrfs-vol -b is faster with the new code as well, he added caching of high levels in the tree to speed things up. (Many kudos to Yan Zheng for all of this work!) -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
