Re: Increasing MAXPHYS

2010-03-23 Thread Poul-Henning Kamp
In message <20100322233607.gb1...@garage.freebsd.pl>, Pawel Jakub Dawidek write s: >A class is suppose to interact with other classes only via GEOM, so I >think it should be safe to choose g_up/g_down threads for each class >individually, for example: > > /dev/ad0s1a (DEV) > | >

Re: Increasing MAXPHYS

2010-03-23 Thread Matthew Dillon
:The whole point of the discussion, sans PHK's interlude, is to reduce the context switches and indirection, not to increase it. But if you can show decreased latency/higher-iops benefits of increasing it, more power to you. I would think that the results of DFly's experiment with parallelism

Re: Increasing MAXPHYS

2010-03-22 Thread Julian Elischer
Pawel Jakub Dawidek wrote: On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: In message <4ba633a0.2090...@icyb.net.ua>, Andriy Gapon writes: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads

Re: Increasing MAXPHYS

2010-03-22 Thread Scott Long
On Mar 22, 2010, at 5:36 PM, Pawel Jakub Dawidek wrote: > On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: >> In message <4ba633a0.2090...@icyb.net.ua>, Andriy Gapon writes: >>> on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: > Hmm, it looks li

Re: Increasing MAXPHYS

2010-03-22 Thread Pawel Jakub Dawidek
On Mon, Mar 22, 2010 at 08:23:43AM +, Poul-Henning Kamp wrote: > In message <4ba633a0.2090...@icyb.net.ua>, Andriy Gapon writes: > >on 21/03/2010 16:05 Alexander Motin said the following: > >> Ivan Voras wrote: > >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, > >>> barr

Re: Increasing MAXPHYS

2010-03-22 Thread Poul-Henning Kamp
In message <3c0b01821003221207p4e4eecabqb4f448813bf5a...@mail.gmail.com>, Alexa nder Sack writes: >Am I going crazy or does this sound a lot like Sun/SVR's stream based >network stack? That is a good and pertinent observation. I did investigate a number of optimizations to the g_up/g_down scheme

Re: Increasing MAXPHYS

2010-03-22 Thread Alexander Sack
On Mon, Mar 22, 2010 at 2:45 PM, M. Warner Losh wrote: > In message: >            Scott Long writes: > : I'd like to go in the opposite direction.  The queue-dispatch-queue > : model of GEOM is elegant and easy to extend, but very wasteful for > : the simple case, where the simple case is one or

Re: Increasing MAXPHYS

2010-03-22 Thread M. Warner Losh
In message: Scott Long writes: : I'd like to go in the opposite direction. The queue-dispatch-queue : model of GEOM is elegant and easy to extend, but very wasteful for : the simple case, where the simple case is one or two simple : partition transforms (mbr, bsdlabel) and/or a simpl

Re: Increasing MAXPHYS

2010-03-22 Thread Scott Long
On Mar 22, 2010, at 9:52 AM, Alexander Sack wrote: > On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin wrote: >> On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: >>> On Sun, 21 Mar 2010 19:03:56 +0200 >>> Alexander Motin wrote: >>> Scott Long wrote: > Are there non-CAM drivers that

Re: Increasing MAXPHYS

2010-03-22 Thread Alexander Sack
On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin wrote: > On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: >> On Sun, 21 Mar 2010 19:03:56 +0200 >> Alexander Motin wrote: >> >> > Scott Long wrote: >> > > Are there non-CAM drivers that look at MAXPHYS, or that silently assume > that >> > > MA

Re: Increasing MAXPHYS

2010-03-22 Thread jhell
On Mon, 22 Mar 2010 01:53, Alexander Motin wrote: In Message-Id: <4ba705cb.9090...@freebsd.org> jhell wrote: On Sun, 21 Mar 2010 20:54, jhell@ wrote: I played with it on one re-compile of a kernel and for the sake of it DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash dum

Re: Increasing MAXPHYS

2010-03-22 Thread John Baldwin
On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: > On Sun, 21 Mar 2010 19:03:56 +0200 > Alexander Motin wrote: > > > Scott Long wrote: > > > Are there non-CAM drivers that look at MAXPHYS, or that silently assume that > > > MAXPHYS will never be more than 128k? > > > > That is a questio

Re: Increasing MAXPHYS

2010-03-22 Thread Alexander Leidinger
Quoting Scott Long (from Sat, 20 Mar 2010 12:17:33 -0600): code was actually taking advantage of the larger I/O's. The improvement really depends on the workload, of course, and I wouldn't expect it to be noticeable for most people unless they're running something like a media server. I do

Re: Increasing MAXPHYS

2010-03-22 Thread Gary Jennejohn
On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin wrote: > Scott Long wrote: > > Are there non-CAM drivers that look at MAXPHYS, or that silently assume that > > MAXPHYS will never be more than 128k? > > That is a question. > I only did a quick&dirty grep looking for MAXPHYS in /sys. Some dr

Re: Increasing MAXPHYS

2010-03-22 Thread Poul-Henning Kamp
In message <4ba633a0.2090...@icyb.net.ua>, Andriy Gapon writes: >on 21/03/2010 16:05 Alexander Motin said the following: >> Ivan Voras wrote: >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>> barring specific class behaviour, it has a fair chance of working out of >>> the

Re: Increasing MAXPHYS

2010-03-21 Thread Alexander Motin
jhell wrote: > On Sun, 21 Mar 2010 20:54, jhell@ wrote: >> I played with it on one re-compile of a kernel and for the sake of it >> DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash >> dump to be performed upon request (reboot -d) due to the boundary >> being hit for DMA which i

Re: Increasing MAXPHYS

2010-03-21 Thread jhell
On Sun, 21 Mar 2010 20:54, jhell@ wrote: On Sun, 21 Mar 2010 10:04, mav@ wrote: Julian Elischer wrote: In the Fusion-io driver we find that the limiting factor is not the size of MAXPHYS, but the fact that we can not push more than 170k tps through geom. (in my test machine. I've seen more on

Re: Increasing MAXPHYS

2010-03-21 Thread jhell
On Sun, 21 Mar 2010 10:04, mav@ wrote: Julian Elischer wrote: In the Fusion-io driver we find that the limiting factor is not the size of MAXPHYS, but the fact that we can not push more than 170k tps through geom. (in my test machine. I've seen more on some beefier machines), but that is only a

Re: Increasing MAXPHYS

2010-03-21 Thread Julian Elischer
Scott Long wrote: I agree that more threads just creates many more race complications. Even if it didn't, the storage driver is a serialization point; it doesn't matter if you have a dozen g_* threads if only one of them can be in the top half of the driver at a time. No amount of fine-grained

Re: Increasing MAXPHYS

2010-03-21 Thread Scott Long
On Mar 21, 2010, at 10:53 AM, Ulrich Spörlein wrote: > [CC trimmed] > On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote: >> On Mar 21, 2010, at 10:30 AM, Ulrich Spörlein wrote: >>> On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: Windows has a MAXPHYS equivalent of 1M. Linux has a

Re: Increasing MAXPHYS

2010-03-21 Thread Alexander Motin
Scott Long wrote: > On Mar 20, 2010, at 1:26 PM, Alexander Motin wrote: >> As you should remember, we have made it in such way, that all unchecked >> drivers keep using DFLTPHYS, which is not going to be changed ever. So >> there is no problem. I would more worry about non-CAM storages and above >>

Re: Increasing MAXPHYS

2010-03-21 Thread Ulrich Spörlein
[CC trimmed] On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote: > On Mar 21, 2010, at 10:30 AM, Ulrich Spörlein wrote: > > On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: > >> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an > >> odd number less than 512k. For th

Re: Increasing MAXPHYS

2010-03-21 Thread Scott Long
On Mar 21, 2010, at 10:30 AM, Ulrich Spörlein wrote: > On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: >> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an >> odd number less than 512k. For the purpose of benchmarking against these >> OS's, having comparable capabiliti

Re: Increasing MAXPHYS

2010-03-21 Thread Scott Long
On Mar 20, 2010, at 1:26 PM, Alexander Motin wrote: > Scott Long wrote: >> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>> Diminishing returns get hit pretty quickly with larger MAXPHYS values. >>> As long as the I/O can be pipelined the reduced transaction rate >>> becomes less int

Re: Increasing MAXPHYS

2010-03-21 Thread Ulrich Spörlein
On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: > Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an > odd number less than 512k. For the purpose of benchmarking against these > OS's, having comparable capabilities is essential; Linux easily beats FreeBSD > in the silly

Re: Increasing MAXPHYS

2010-03-21 Thread Scott Long
m On Mar 21, 2010, at 8:56 AM, Andriy Gapon wrote: > on 21/03/2010 16:05 Alexander Motin said the following: >> Ivan Voras wrote: >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>> barring specific class behaviour, it has a fair chance of working out of >>> the box) but th

Re: Increasing MAXPHYS

2010-03-21 Thread Scott Long
On Mar 21, 2010, at 8:05 AM, Alexander Motin wrote: > Ivan Voras wrote: >> Julian Elischer wrote: >>> You can get better throughput by using TSC for timing because the geom >>> and devstat code does a bit of timing.. Geom can be told to turn off >>> it's timing but devstat can't. The 170 ktps is

Re: Increasing MAXPHYS

2010-03-21 Thread Julian Elischer
Andriy Gapon wrote: on 21/03/2010 16:05 Alexander Motin said the following: Ivan Voras wrote: Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken

Re: Increasing MAXPHYS

2010-03-21 Thread Julian Elischer
Alexander Motin wrote: Julian Elischer wrote: In the Fusion-io driver we find that the limiting factor is not the size of MAXPHYS, but the fact that we can not push more than 170k tps through geom. (in my test machine. I've seen more on some beefier machines), but that is only a limit on small t

Re: Increasing MAXPHYS

2010-03-21 Thread Andriy Gapon
on 21/03/2010 16:05 Alexander Motin said the following: > Ivan Voras wrote: >> Hmm, it looks like it could be easy to spawn more g_* threads (and, >> barring specific class behaviour, it has a fair chance of working out of >> the box) but the incoming queue will need to also be broken up for >> gre

Re: Increasing MAXPHYS

2010-03-21 Thread Alexander Motin
Ivan Voras wrote: > Julian Elischer wrote: >> You can get better throughput by using TSC for timing because the geom >> and devstat code does a bit of timing.. Geom can be told to turn off >> it's timing but devstat can't. The 170 ktps is with TSC as timer, >> and geom timing turned off. > > I see

Re: Increasing MAXPHYS

2010-03-21 Thread Alexander Motin
Julian Elischer wrote: > In the Fusion-io driver we find that the limiting factor is not the > size of MAXPHYS, but the fact that we can not push more than > 170k tps through geom. (in my test machine. I've seen more on some > beefier machines), but that is only a limit on small transacrtions, > or

Re: Increasing MAXPHYS

2010-03-20 Thread Julian Elischer
Ivan Voras wrote: Julian Elischer wrote: Alexander Motin wrote: Scott Long wrote: On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: Diminishing returns get hit pretty quickly with larger MAXPHYS values. As long as the I/O can be pipelined the reduced transaction rate becomes less

Re: Increasing MAXPHYS

2010-03-20 Thread Julian Elischer
Alexander Motin wrote: Scott Long wrote: On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: Diminishing returns get hit pretty quickly with larger MAXPHYS values. As long as the I/O can be pipelined the reduced transaction rate becomes less interesting when the transaction rate is les

Re: Increasing MAXPHYS

2010-03-20 Thread Alan Cox
d last months with MAXPHYS of 1MB on > i386 and amd64 platforms. > > So my questions are: > - does somebody know any issues denying increasing MAXPHYS in HEAD? > - are there any specific opinions about value? 512K, 1MB, MD? > > For now, I think it should machine-dependent. The virtua

Re: Increasing MAXPHYS

2010-03-20 Thread Alexander Motin
Scott Long wrote: > On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>Diminishing returns get hit pretty quickly with larger MAXPHYS values. >>As long as the I/O can be pipelined the reduced transaction rate >>becomes less interesting when the transaction rate is less than a >>c

Re: Increasing MAXPHYS

2010-03-20 Thread C. P. Ghost
On Sat, Mar 20, 2010 at 6:53 PM, Matthew Dillon wrote: > > :All above I have successfully tested last months with MAXPHYS of 1MB on > :i386 and amd64 platforms. > : > :So my questions are: > :- does somebody know any issues denying increasing MAXPHYS in HEAD? > :- are there

Re: Increasing MAXPHYS

2010-03-20 Thread Matthew Dillon
:Pardon my ignorance, but wouldn't so much KVM make small embedded :devices like Soekris boards with 128 MB of physical RAM totally unusable :then? On my net4801, running RELENG_8: : :vm.kmem_size: 40878080 : :hw.physmem: 125272064 :hw.usermen: 84840448 :hw.realmem: 134217728 KVM != physical m

Re: Increasing MAXPHYS

2010-03-20 Thread Scott Long
On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: > > :All above I have successfully tested last months with MAXPHYS of 1MB on > :i386 and amd64 platforms. > : > :So my questions are: > :- does somebody know any issues denying increasing MAXPHYS in HEAD? > :- are there

Re: Increasing MAXPHYS

2010-03-20 Thread Matthew Dillon
:All above I have successfully tested last months with MAXPHYS of 1MB on :i386 and amd64 platforms. : :So my questions are: :- does somebody know any issues denying increasing MAXPHYS in HEAD? :- are there any specific opinions about value? 512K, 1MB, MD? : :-- :Alexander Motin (nswbuf

Increasing MAXPHYS

2010-03-20 Thread Alexander Motin
s are: - does somebody know any issues denying increasing MAXPHYS in HEAD? - are there any specific opinions about value? 512K, 1MB, MD? -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-curre