Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 04:31 AM, Daniel Phillips wrote: Let me be the first to catch that arithmetic error Let's say our delta size is 400MB (typical under load) and we leave a nice big gap of 112 MB after flushing each one. Let's say we do two thousand of those before deciding that we have enough

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Daniel Phillips
On 05/13/2015 06:08 AM, Mike Galbraith wrote: On Wed, 2015-05-13 at 04:31 -0700, Daniel Phillips wrote: Third possibility: build from our repository, as Mike did. Sorry about that folks. I've lost all interest, it won't happen again. Thanks for your valuable contribution. Now we are seeing

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-13 Thread Martin Steigerwald
Am Mittwoch, 13. Mai 2015, 13:38:24 schrieb Daniel Phillips: On Wednesday, May 13, 2015 1:25:38 PM PDT, Martin Steigerwald wrote: Am Mittwoch, 13. Mai 2015, 12:37:41 schrieb Daniel Phillips: On 05/13/2015 12:09 PM, Martin Steigerwald wrote: ... Daniel, if you want to change the process

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On Monday, May 11, 2015 10:38:42 PM PDT, Dave Chinner wrote: I think Ted and I are on the same page here. Competitive benchmarks only matter to the people who are trying to sell something. You're trying to sell Tux3, but By same page, do you mean transparently obvious about obstructing

Re: Tux3 Report: How fast can we fsync?

2015-05-12 Thread Daniel Phillips
Tux3 Report: How fast can we fail? Tux3 now has a preliminary out of space handling algorithm. This might sound like a small thing, but in fact handling out of space reliably and efficiently is really hard, especially for Tux3. We developed an original solution with unusually low overhead in the

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread David Lang
On Mon, 11 May 2015, Daniel Phillips wrote: On Monday, May 11, 2015 10:38:42 PM PDT, Dave Chinner wrote: I think Ted and I are on the same page here. Competitive benchmarks only matter to the people who are trying to sell something. You're trying to sell Tux3, but By same page, do you

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 11:39 AM, David Lang wrote: On Mon, 11 May 2015, Daniel Phillips wrote: ...it's the mm and core kernel developers that need to review and accept that code *before* we can consider merging tux3. Please do not say we when you know that I am just as much a we as you are. Merging

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread David Lang
On Tue, 12 May 2015, Daniel Phillips wrote: On 05/12/2015 02:30 PM, David Lang wrote: On Tue, 12 May 2015, Daniel Phillips wrote: Phoronix published a headline that identifies Dave Chinner as someone who takes shots at other projects. Seems pretty much on the money to me, and it ought to be

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 03:35 PM, David Lang wrote: On Tue, 12 May 2015, Daniel Phillips wrote: On 05/12/2015 02:30 PM, David Lang wrote: You need to get out of the mindset that Ted and Dave are Enemies that you need to overcome, they are friendly competitors, not Enemies. You are wrong about Dave

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Pavel Machek
On Mon 2015-05-11 19:34:34, Daniel Phillips wrote: On 05/11/2015 04:17 PM, Theodore Ts'o wrote: On Tue, May 12, 2015 at 12:12:23AM +0200, Pavel Machek wrote: Umm, are you sure. If some areas of disk are faster than others is still true on todays harddrives, the gaps will decrease the

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Daniel Phillips
On 05/12/2015 02:03 AM, Pavel Machek wrote: On Mon 2015-05-11 19:34:34, Daniel Phillips wrote: On 05/11/2015 04:17 PM, Theodore Ts'o wrote: and another way that people doing competitive benchmarking can screw up and produce misleading numbers. If you think we screwed up or produced

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-12 Thread Howard Chu
Daniel Phillips wrote: On 05/12/2015 02:03 AM, Pavel Machek wrote: I'd call system with 65 tasks doing heavy fsync load at the some time embarrassingly misconfigured :-). It is nice if your filesystem can stay fast in that case, but... Well, Tux3 wins the fsync race now whether it is 1 task,

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread David Lang
On Mon, 11 May 2015, Daniel Phillips wrote: On 05/11/2015 03:12 PM, Pavel Machek wrote: It is a fact of life that when you change one aspect of an intimately interconnected system, something else will change as well. You have naive/nonexistent free space management now; when you design

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Theodore Ts'o
On Tue, May 12, 2015 at 12:12:23AM +0200, Pavel Machek wrote: Umm, are you sure. If some areas of disk are faster than others is still true on todays harddrives, the gaps will decrease the performance (as you'll use up the fast areas more quickly). It's still true. The difference between O.D.

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-05-11 Thread Daniel Phillips
Hi David, On 05/11/2015 05:12 PM, David Lang wrote: On Mon, 11 May 2015, Daniel Phillips wrote: On 05/11/2015 03:12 PM, Pavel Machek wrote: It is a fact of life that when you change one aspect of an intimately interconnected system, something else will change as well. You have

Re: Tux3 Report: How fast can we fsync?

2015-05-02 Thread Daniel Phillips
On Friday, May 1, 2015 6:07:48 PM PDT, David Lang wrote: On Fri, 1 May 2015, Daniel Phillips wrote: On Friday, May 1, 2015 8:38:55 AM PDT, Dave Chinner wrote: Well, yes - I never claimed XFS is a general purpose filesystem. It is a high performance filesystem. Is is also becoming more

Re: Tux3 Report: How fast can we fsync?

2015-05-01 Thread Daniel Phillips
On Friday, May 1, 2015 8:38:55 AM PDT, Dave Chinner wrote: Well, yes - I never claimed XFS is a general purpose filesystem. It is a high performance filesystem. Is is also becoming more relevant to general purpose systems as low cost storage gains capabilities that used to be considered the

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 5:20:08 PM PDT, Dave Chinner wrote: It's easy to be fast on empty filesystems. XFS does not aim to be fast in such situations - it aims to have consistent performance across the life of the filesystem. In this case, ext4, btrfs and tux3 have optimal allocation

Re: Tux3 Report: How fast can we fsync?

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 8:50:57 PM PDT, Mike Galbraith wrote: On Wed, 2015-04-29 at 13:40 -0700, Daniel Phillips wrote: That order of magnitude latency difference is striking. It sounds good, but what does it mean? I see a smaller difference here, maybe because of running under KVM.

Re: Tux3 Report: How fast can we fsync?

2015-04-30 Thread Daniel Phillips
On Wednesday, April 29, 2015 6:46:16 PM PDT, Dave Chinner wrote: I measured fsync performance using a 7200 RPM disk as a virtual drive under KVM, configured with cache=none so that asynchronous writes are cached and synchronous writes translate into direct writes to the block device. Yup, a

Re: Tux3 Report: How fast can we fsync?

2015-04-30 Thread Daniel Phillips
On Thursday, April 30, 2015 2:17:55 PM PDT, James Cloos wrote: DP == Daniel Phillips dan...@phunq.net writes: DP you build userspace tools from the hirofumi-user branch In a fresh clone there is no hirofumi-user branch, only hirofumi and master: :; cat .git/packed-refs # pack-refs with:

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Theodore Ts'o
On Thu, Apr 30, 2015 at 11:00:05AM +0200, Martin Steigerwald wrote: IOWS, XFS just hates your disk. Spend $50 and buy a cheap SSD and the problem goes away. :) I am quite surprised that a traditional filesystem that was created in the age of rotating media does not like this kind of media

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-30 Thread Howard Chu
Daniel Phillips wrote: On 04/30/2015 07:28 AM, Howard Chu wrote: You're reading into it what isn't there. Spreading over the disk isn't (just) about avoiding fragmentation - it's about delivering consistent and predictable latency. It is undeniable that if you start by only allocating from

Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

2015-04-29 Thread Mike Galbraith
On Wed, 2015-04-29 at 14:12 -0700, Daniel Phillips wrote: Btrfs appears to optimize tiny files by storing them in its big btree, the equivalent of our itree, and Tux3 doesn't do that yet, so we are a bit hobbled for a make load. That's not a build load, it's a git load. btrfs beat all others