Re: Ext4 benchmarks
On Wed, 2007-04-04 at 13:21 -0600, Andreas Dilger wrote: > On Apr 04, 2007 19:06 +0200, Cordenner jean noel wrote: > > here is the first results of the round: > > http://www.bullopensource.org/ext4/20070404/ > > Jean Noel, > thank you for the test results. It is always nice to see that ext4 is > doing so well compared to ext3 and XFS. > > Ming Ming, > it should be possible to just include the mballoc+delalloc patches that > Jean Noel used into the upstream ext4 patch series. When Alex or Christoph > get a chance to do the VFS delalloc rewrite we can move to that new patch, > but until then it seems pointless to not include this functionality which > improves the performance so much. > >From the bull website it said the test is based on 2.6.21-rc4 kernel + delalloc patch. I don't think that includes the mballoc patch. > Also, if we include those patches the mballoc and delalloc features (along > with extents) should be enabled by default if INCOMPAT_EXTENTS is in the > superblock unless: > - "noextents", "nomballoc", or "nodelalloc" mount options are given I just added noextents and nodelalloc mount options in the 2.6.21-rc5 version ext4 patch queue. But we should keep delalloc with nomballoc. The current delalloc patch in ext4 tree plays well without mballoc. We still could do multiple block allocations with delayed allocation, though not as smart as Alex's mballoc. > - delalloc needs to be disabled if blocksize != PAGE_SIZE > I believe the current ext4 delalloc code turns off delalloc in this case already. > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. > > - > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
On Apr 04, 2007 19:06 +0200, Cordenner jean noel wrote: > here is the first results of the round: > http://www.bullopensource.org/ext4/20070404/ Jean Noel, thank you for the test results. It is always nice to see that ext4 is doing so well compared to ext3 and XFS. Ming Ming, it should be possible to just include the mballoc+delalloc patches that Jean Noel used into the upstream ext4 patch series. When Alex or Christoph get a chance to do the VFS delalloc rewrite we can move to that new patch, but until then it seems pointless to not include this functionality which improves the performance so much. Also, if we include those patches the mballoc and delalloc features (along with extents) should be enabled by default if INCOMPAT_EXTENTS is in the superblock unless: - "noextents", "nomballoc", or "nodelalloc" mount options are given - delalloc needs to be disabled if blocksize != PAGE_SIZE Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Hi, here is the first results of the round: http://www.bullopensource.org/ext4/20070404/ FFSB tests: http://www.bullopensource.org/ext4/20070404/ffsb-write.html Iozone: http://www.bullopensource.org/ext4/20070404/iozone.html Kernbuild: http://www.bullopensource.org/ext4/20070404/kernbuild.html regards, Jean Noel - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Hi Jose, I have to check with the team here. Will tell you. Thanks. jean-pierre Jose R. Santos wrote: Jean-Pierre Dion wrote: Hi Jose, thank you for the feedback. We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...). We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)... We will publish the results on bullopensource.org Hi Jean-Pierre, While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity. I believe that IOzone is very limited in testing this and FFSB isn't much better at it either. Eventually, I plan to add support in FFSB to create workload profile were one can select a weight balance of these types of operations. Is this something that you are already doing for this round of testing? -JRS - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Hi Ric, that may be useful. Checking with the team here and tell you. Thanks. jean-pierre Ric Wheeler wrote: Jean-Pierre Dion wrote: Hi Jose, thank you for the feedback. We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...). We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)... We will publish the results on bullopensource.org jean-pierre I also have a benchmark that stresses a heavy synchronous write workload that I can run. ric - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
On Mar 30, 2007 10:43 +0200, Johann Lombardi wrote: > On Wed, Mar 28, 2007 at 09:20:50AM -0500, Jose R. Santos wrote: > > While it may be to late for the purposes of your OLS paper, one thing > > that doesn't seem to be getting much attention is the performance of a > > file system while doing many meta-data operations or throughput testing > > during heavy journal log activity. > > Back in 2005, Alex sent a patch to monitor journal activity through procfs: > http://marc.info/?l=linux-ext4&m=113538565128617&w=2 > > We have been using this patch for more than 1 year and it is very useful > for investigating performance issues. > Does anybody know why this patch hasn't found its way into the mainline? > Maybe it could be merged in jbd2? Yes, we use this functionality failry often, so having it in jbd2 would be quite useful. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
On Wed, Mar 28, 2007 at 09:20:50AM -0500, Jose R. Santos wrote: > While it may be to late for the purposes of your OLS paper, one thing > that doesn't seem to be getting much attention is the performance of a > file system while doing many meta-data operations or throughput testing > during heavy journal log activity. Back in 2005, Alex sent a patch to monitor journal activity through procfs: http://marc.info/?l=linux-ext4&m=113538565128617&w=2 We have been using this patch for more than 1 year and it is very useful for investigating performance issues. Does anybody know why this patch hasn't found its way into the mainline? Maybe it could be merged in jbd2? Johann - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Jean-Pierre Dion wrote: Hi Jose, thank you for the feedback. We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...). We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)... We will publish the results on bullopensource.org Hi Jean-Pierre, While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity. I believe that IOzone is very limited in testing this and FFSB isn't much better at it either. Eventually, I plan to add support in FFSB to create workload profile were one can select a weight balance of these types of operations. Is this something that you are already doing for this round of testing? -JRS - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Jean-Pierre Dion wrote: Hi Jose, thank you for the feedback. We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...). We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)... We will publish the results on bullopensource.org jean-pierre I also have a benchmark that stresses a heavy synchronous write workload that I can run. ric - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Hi Jose, thank you for the feedback. We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...). We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)... We will publish the results on bullopensource.org jean-pierre Jose R. Santos wrote: Jean-Pierre Dion wrote: Hi all, we already discussed during the conf calls what benchmarks should be ran on ext4. As we have OLS paper on the table we were thinking here at Bull what bench t run and on which kernel. If we want trying to compare ext3 and ext4, I guess we should at least show that : - ext4 has equivalent perfs than ext3, Define equivalent performance. Are the workloads only going to be focused on single repetitive operations or simulation of actual desktop/server environments? How about performance on an aged filesystem? - improvements done for ext3 are still in ext4 (mb alloc, del alloc...). So we were wondering what's best to do : - run on 2.6.19 (includes del alloc and mb alloc if I am not wrong), - run on 2.6.20 (lacks mb alloc), What about system configurations? While a desktop configuration would be easy to come by, a server configuration needs a bit more thought. Will ext4 perform better than ext3 in a wide range of storage configuration that scale from a couple thousands IOPS to several hundred thousand IOPS? Having baseline data on other filesystems like XFS or JFS would be interesting as well to see how well ext4 stacks up to the competition. :) - select relevant benchs (iozone...). I haven checked IOzone in quite a bit but last time I checked FFSB had a couple of capabilities that are not available in IOzone like multi threading on a shared data set and a very customizable IO operations to attempt to simulate real IO patterns seen on workloads. Might be worth a look if your interested in compiling a very comprehensive set of results What do you think ? Thanks. jean-pierre -JRS - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Ext4 benchmarks
Jean-Pierre Dion wrote: Hi all, we already discussed during the conf calls what benchmarks should be ran on ext4. As we have OLS paper on the table we were thinking here at Bull what bench t run and on which kernel. If we want trying to compare ext3 and ext4, I guess we should at least show that : - ext4 has equivalent perfs than ext3, Define equivalent performance. Are the workloads only going to be focused on single repetitive operations or simulation of actual desktop/server environments? How about performance on an aged filesystem? - improvements done for ext3 are still in ext4 (mb alloc, del alloc...). So we were wondering what's best to do : - run on 2.6.19 (includes del alloc and mb alloc if I am not wrong), - run on 2.6.20 (lacks mb alloc), What about system configurations? While a desktop configuration would be easy to come by, a server configuration needs a bit more thought. Will ext4 perform better than ext3 in a wide range of storage configuration that scale from a couple thousands IOPS to several hundred thousand IOPS? Having baseline data on other filesystems like XFS or JFS would be interesting as well to see how well ext4 stacks up to the competition. :) - select relevant benchs (iozone...). I haven checked IOzone in quite a bit but last time I checked FFSB had a couple of capabilities that are not available in IOzone like multi threading on a shared data set and a very customizable IO operations to attempt to simulate real IO patterns seen on workloads. Might be worth a look if your interested in compiling a very comprehensive set of results What do you think ? Thanks. jean-pierre -JRS - To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
