Re: [Kiobuf-io-devel] Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-19 Thread Christoph Hellwig
On Fri, Jan 19, 2001 at 08:05:41AM +0530, [EMAIL PROTECTED] wrote: > Shouldn't we have an error / status field too ? Might make sense. Christoph -- Of course it doesn't work. We've performed a software upgrade. - To unsubscribe from this list: send the line "unsubscribe linux-kernel"

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-19 Thread Andrew Scott
On 10 Jan 2001, at 15:32, Linus Torvalds wrote: > Latin 101. Literally "about taste no argument". Or "about taste no argument there is" if you add the 'est', which still makes sense in english, in a twisted (convoluted as apposed to 'bad' or 'sick') way. Q.E.D. > I suspect that it

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-19 Thread Andrew Scott
On 10 Jan 2001, at 15:32, Linus Torvalds wrote: Latin 101. Literally "about taste no argument". Or "about taste no argument there is" if you add the 'est', which still makes sense in english, in a twisted (convoluted as apposed to 'bad' or 'sick') way. Q.E.D. I suspect that it _should_

Re: [Kiobuf-io-devel] Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-19 Thread Christoph Hellwig
On Fri, Jan 19, 2001 at 08:05:41AM +0530, [EMAIL PROTECTED] wrote: Shouldn't we have an error / status field too ? Might make sense. Christoph -- Of course it doesn't work. We've performed a software upgrade. - To unsubscribe from this list: send the line "unsubscribe linux-kernel"

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Albert D. Cahalan <[EMAIL PROTECTED]> wrote: > >What about getting rid of both that and the pointer, and just >hanging that data on the end as a variable length array? > >struct kiovec2{ > int nbufs; > /* ... */ > struct kiobuf[0]; >} If the struct ends up

Re: [Kiobuf-io-devel] Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread bsuparna
>Ok. Then we need an additional more or less generic object that is used for >passing in a rw_kiovec file operation (and we really want that for many kinds >of IO). I thould mostly be used for communicating to the high-level driver. > >/* > * the name is just plain stupid, but that shouldn't

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Albert D. Cahalan
>> struct kiovec2 { >> int nbufs; /* Kiobufs actually referenced */ >> int array_len; /* Space in the allocated lists */ >> struct kiobuf * bufs; > > Any reason for array_len? > > Why not just > > int nbufs, > struct kiobuf *bufs;

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Linus Torvalds
On Thu, 18 Jan 2001, Christoph Hellwig wrote: > > > > Remove this. I don't think it's valid to lock the pages. Who wants to use > > this anyway? > > E.g. in the block IO pathes the pages have to be locked. > It's also used by free_kiovec to see wether to do unlock_kiovec before. This is all

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Christoph Hellwig
On Wed, Jan 17, 2001 at 05:13:31PM -0800, Linus Torvalds wrote: > > > On Thu, 18 Jan 2001, Christoph Hellwig wrote: > > > > /* > > * a simple page,offset,legth tuple like Linus wants it > > */ > > struct kiobuf2 { > > struct page * page; /* The page itself */ > >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Christoph Hellwig
On Wed, Jan 17, 2001 at 05:13:31PM -0800, Linus Torvalds wrote: On Thu, 18 Jan 2001, Christoph Hellwig wrote: /* * a simple page,offset,legth tuple like Linus wants it */ struct kiobuf2 { struct page * page; /* The page itself */ u_int16_t

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Linus Torvalds
On Thu, 18 Jan 2001, Christoph Hellwig wrote: Remove this. I don't think it's valid to lock the pages. Who wants to use this anyway? E.g. in the block IO pathes the pages have to be locked. It's also used by free_kiovec to see wether to do unlock_kiovec before. This is all MUCH

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Albert D. Cahalan
struct kiovec2 { int nbufs; /* Kiobufs actually referenced */ int array_len; /* Space in the allocated lists */ struct kiobuf * bufs; Any reason for array_len? Why not just int nbufs, struct kiobuf *bufs; Remember:

Re: [Kiobuf-io-devel] Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread bsuparna
Ok. Then we need an additional more or less generic object that is used for passing in a rw_kiovec file operation (and we really want that for many kinds of IO). I thould mostly be used for communicating to the high-level driver. /* * the name is just plain stupid, but that shouldn't matter

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-18 Thread Linus Torvalds
In article [EMAIL PROTECTED], Albert D. Cahalan [EMAIL PROTECTED] wrote: What about getting rid of both that and the pointer, and just hanging that data on the end as a variable length array? struct kiovec2{ int nbufs; /* ... */ struct kiobuf[0]; } If the struct ends up having lots of

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Linus Torvalds
On Thu, 18 Jan 2001, Christoph Hellwig wrote: > > /* > * a simple page,offset,legth tuple like Linus wants it > */ > struct kiobuf2 { > struct page * page; /* The page itself */ > u_int16_t offset; /* Offset to start of valid data */ > u_int16_t

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Christoph Hellwig
On Thu, Jan 18, 2001 at 01:05:43AM +1100, Rik van Riel wrote: > On Wed, 10 Jan 2001, Christoph Hellwig wrote: > > > Simple. Because I stated before that I DON'T even want the > > networking to use kiobufs in lower layers. My whole argument is > > to pass a kiovec into the fileop instead of a

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Rik van Riel
On Wed, 10 Jan 2001, Christoph Hellwig wrote: > Simple. Because I stated before that I DON'T even want the > networking to use kiobufs in lower layers. My whole argument is > to pass a kiovec into the fileop instead of a page, because it > makes sense for other drivers to use multiple pages,

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Rik van Riel
On Wed, 10 Jan 2001, Christoph Hellwig wrote: Simple. Because I stated before that I DON'T even want the networking to use kiobufs in lower layers. My whole argument is to pass a kiovec into the fileop instead of a page, because it makes sense for other drivers to use multiple pages, Now

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Christoph Hellwig
On Thu, Jan 18, 2001 at 01:05:43AM +1100, Rik van Riel wrote: On Wed, 10 Jan 2001, Christoph Hellwig wrote: Simple. Because I stated before that I DON'T even want the networking to use kiobufs in lower layers. My whole argument is to pass a kiovec into the fileop instead of a page,

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-17 Thread Linus Torvalds
On Thu, 18 Jan 2001, Christoph Hellwig wrote: /* * a simple page,offset,legth tuple like Linus wants it */ struct kiobuf2 { struct page * page; /* The page itself */ u_int16_t offset; /* Offset to start of valid data */ u_int16_t

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-16 Thread Rik van Riel
On Tue, 9 Jan 2001, Andrea Arcangeli wrote: > BTW, I noticed what is left in blk-13B seems to be my work Yeah yeah, we'll buy you beer at the next conference... ;) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-16 Thread Rik van Riel
On Tue, 9 Jan 2001, Andrea Arcangeli wrote: BTW, I noticed what is left in blk-13B seems to be my work Yeah yeah, we'll buy you beer at the next conference... ;) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-13 Thread yodaiken
FWIW: POSIX mq_send does not promise that the buffer is safe, it only promises that the message is queued. Interesting interface. On Wed, Jan 10, 2001 at 09:41:24AM +0100, Manfred Spraul wrote: > > > In user space, how do you know when its safe to reuse the buffer that > > > was handed to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-13 Thread yodaiken
FWIW: POSIX mq_send does not promise that the buffer is safe, it only promises that the message is queued. Interesting interface. On Wed, Jan 10, 2001 at 09:41:24AM +0100, Manfred Spraul wrote: In user space, how do you know when its safe to reuse the buffer that was handed to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-11 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 11:14:54AM -0800, Linus Torvalds wrote: > In article <[EMAIL PROTECTED]>, > > kiobufs are crap. Face it. They do NOT allow proper multi-page scatter > gather, regardless of what the kiobuf PR department has said. It's not surprising, since they were designed to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-11 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 11:14:54AM -0800, Linus Torvalds wrote: In article [EMAIL PROTECTED], kiobufs are crap. Face it. They do NOT allow proper multi-page scatter gather, regardless of what the kiobuf PR department has said. It's not surprising, since they were designed to solve a

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Andrew Morton <[EMAIL PROTECTED]> wrote: >Linus Torvalds wrote: >> >> De gustibus non disputandum. > >http://cogprints.soton.ac.uk/documents/disk0/00/00/07/57/ > > "ingestion of the afterbirth during delivery" > >eh? > > >http://www.degustibus.co.uk/ > >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Jamie Lokier
Ingo Molnar wrote: > > > well, this is a performance problem if you are using threads. For normal > > > processes there is no need for a SMP cross-call, there TLB flushes are > > > local only. > > > > > But that would be ugly as hell: > > so apache 2.0 would become slower with MSG_NOCOPY, whereas

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 02:25:43PM -0800, Linus Torvalds wrote: > In article <[EMAIL PROTECTED]>, > Stephen C. Tweedie <[EMAIL PROTECTED]> wrote: > > > >Jes has also got hard numbers for the performance advantages of > >jumbograms on some of the networks he's been using, and you ain't >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Ingo Molnar
On Wed, 10 Jan 2001, Manfred Spraul wrote: > > well, this is a performance problem if you are using threads. For normal > > processes there is no need for a SMP cross-call, there TLB flushes are > > local only. > > > But that would be ugly as hell: > so apache 2.0 would become slower with

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Manfred Spraul
Ingo Molnar wrote: > > On Wed, 10 Jan 2001, Manfred Spraul wrote: > > > That means sendmsg() changes the page tables? I measures > > smp_call_function on my Dual Pentium 350, and it took around 1950 cpu > > ticks. > > well, this is a performance problem if you are using threads. For normal >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Ingo Molnar
On Wed, 10 Jan 2001, Manfred Spraul wrote: > That means sendmsg() changes the page tables? I measures > smp_call_function on my Dual Pentium 350, and it took around 1950 cpu > ticks. well, this is a performance problem if you are using threads. For normal processes there is no need for a SMP

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Trond Myklebust
> " " == David S Miller <[EMAIL PROTECTED]> writes: >Date: Tue, 9 Jan 2001 16:27:49 +0100 (CET) From: Trond >Myklebust <[EMAIL PROTECTED]> >OK, but can you eventually generalize it to non-stream >protocols (i.e. UDP)? > Sure, this is what MSG_MORE

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread David S. Miller
Date:Wed, 10 Jan 2001 09:41:24 +0100 From: Manfred Spraul <[EMAIL PROTECTED]> That means sendmsg() changes the page tables? Not in the zerocopy patch I am proposing and asking people to test. I stated in another email that MSG_NOCOPY was considered experimental and thus left

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Manfred Spraul
> > In user space, how do you know when its safe to reuse the buffer that > > was handed to sendmsg() with the MSG_NOCOPY flag? Or does sendmsg() > > with that flag block until the buffer isn't needed by the kernel any > > more? If it does block, doesn't that defeat the use of non-blocking >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Christoph Hellwig
On Wed, Jan 10, 2001 at 12:05:01AM -0800, Linus Torvalds wrote: > > > On Wed, 10 Jan 2001, Christoph Hellwig wrote: > > > > Simple. Because I stated before that I DON'T even want the networking > > to use kiobufs in lower layers. My whole argument is to pass a kiovec > > into the fileop

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Andrew Morton
Linus Torvalds wrote: > > De gustibus non disputandum. http://cogprints.soton.ac.uk/documents/disk0/00/00/07/57/ "ingestion of the afterbirth during delivery" eh? http://www.degustibus.co.uk/ "Award winning artisan breadmakers." Ah. That'll be it. - - To unsubscribe from

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Gerd Knorr
> > Please tell me what you think the right interface is that provides a hook > > on io completion and is asynchronous. > > Suggested fix to kiovec's: get rid of them. Immediately. Replace them with > kiobuf's that can handle scatter-gather pages. kiobuf's have 90% of that > support already. >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 02:25:43PM -0800, Linus Torvalds wrote: In article [EMAIL PROTECTED], Stephen C. Tweedie [EMAIL PROTECTED] wrote: Jes has also got hard numbers for the performance advantages of jumbograms on some of the networks he's been using, and you ain't going to get udp

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Jamie Lokier
Ingo Molnar wrote: well, this is a performance problem if you are using threads. For normal processes there is no need for a SMP cross-call, there TLB flushes are local only. But that would be ugly as hell: so apache 2.0 would become slower with MSG_NOCOPY, whereas samba 2.2

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Linus Torvalds
In article [EMAIL PROTECTED], Andrew Morton [EMAIL PROTECTED] wrote: Linus Torvalds wrote: De gustibus non disputandum. http://cogprints.soton.ac.uk/documents/disk0/00/00/07/57/ "ingestion of the afterbirth during delivery" eh? http://www.degustibus.co.uk/ "Award winning

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread David S. Miller
Date:Wed, 10 Jan 2001 09:41:24 +0100 From: Manfred Spraul [EMAIL PROTECTED] That means sendmsg() changes the page tables? Not in the zerocopy patch I am proposing and asking people to test. I stated in another email that MSG_NOCOPY was considered experimental and thus left out

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Gerd Knorr
Please tell me what you think the right interface is that provides a hook on io completion and is asynchronous. Suggested fix to kiovec's: get rid of them. Immediately. Replace them with kiobuf's that can handle scatter-gather pages. kiobuf's have 90% of that support already. Never

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Andrew Morton
Linus Torvalds wrote: De gustibus non disputandum. http://cogprints.soton.ac.uk/documents/disk0/00/00/07/57/ "ingestion of the afterbirth during delivery" eh? http://www.degustibus.co.uk/ "Award winning artisan breadmakers." Ah. That'll be it. - - To unsubscribe from

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Christoph Hellwig
On Wed, Jan 10, 2001 at 12:05:01AM -0800, Linus Torvalds wrote: On Wed, 10 Jan 2001, Christoph Hellwig wrote: Simple. Because I stated before that I DON'T even want the networking to use kiobufs in lower layers. My whole argument is to pass a kiovec into the fileop instead of a

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Manfred Spraul
In user space, how do you know when its safe to reuse the buffer that was handed to sendmsg() with the MSG_NOCOPY flag? Or does sendmsg() with that flag block until the buffer isn't needed by the kernel any more? If it does block, doesn't that defeat the use of non-blocking I/O?

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Trond Myklebust
" " == David S Miller [EMAIL PROTECTED] writes: Date: Tue, 9 Jan 2001 16:27:49 +0100 (CET) From: Trond Myklebust [EMAIL PROTECTED] OK, but can you eventually generalize it to non-stream protocols (i.e. UDP)? Sure, this is what MSG_MORE is meant to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Ingo Molnar
On Wed, 10 Jan 2001, Manfred Spraul wrote: That means sendmsg() changes the page tables? I measures smp_call_function on my Dual Pentium 350, and it took around 1950 cpu ticks. well, this is a performance problem if you are using threads. For normal processes there is no need for a SMP

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Manfred Spraul
Ingo Molnar wrote: On Wed, 10 Jan 2001, Manfred Spraul wrote: That means sendmsg() changes the page tables? I measures smp_call_function on my Dual Pentium 350, and it took around 1950 cpu ticks. well, this is a performance problem if you are using threads. For normal processes

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-10 Thread Ingo Molnar
On Wed, 10 Jan 2001, Manfred Spraul wrote: well, this is a performance problem if you are using threads. For normal processes there is no need for a SMP cross-call, there TLB flushes are local only. But that would be ugly as hell: so apache 2.0 would become slower with MSG_NOCOPY,

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
On Wed, 10 Jan 2001, Christoph Hellwig wrote: > > Simple. Because I stated before that I DON'T even want the networking > to use kiobufs in lower layers. My whole argument is to pass a kiovec > into the fileop instead of a page, because it makes sense for other > drivers to use multiple

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Christoph Hellwig
On Tue, Jan 09, 2001 at 01:26:44PM -0800, Linus Torvalds wrote: > > > On Tue, 9 Jan 2001, Christoph Hellwig wrote: > > > > > > Look at sendfile(). You do NOT have a "bunch" of pages. > > > > > > Sendfile() is very much a page-at-a-time thing, and expects the actual IO > > > layers to do it's

Re: storage over IP (was Re: [PLEASE-TESTME] Zerocopy networking patch,2.4.0-1)

2001-01-09 Thread David S. Miller
Date: Tue, 9 Jan 2001 18:56:33 -0800 (PST) From: dean gaudet <[EMAIL PROTECTED]> is NFS receive single copy today? With the zerocopy patches, NFS client receive is "single cpu copy" if that's what you mean. Later, David S. Miller [EMAIL PROTECTED] - To unsubscribe from this list:

storage over IP (was Re: [PLEASE-TESTME] Zerocopy networking patch,2.4.0-1)

2001-01-09 Thread dean gaudet
On Tue, 9 Jan 2001, Ingo Molnar wrote: > On Mon, 8 Jan 2001, Rik van Riel wrote: > > > Having proper kiobuf support would make it possible to, for example, > > do zerocopy network->disk data transfers and lots of other things. > > i used to think that this is useful, but these days it isnt.

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dave Zarzycki
On Tue, 9 Jan 2001, David S. Miller wrote: > Ignore Ingo's comments about the MSG_NOCOPY flag, I've not included > those parts in the zerocopy patches as they are very controversial > and require some VM layer support. Okay, I talked to some kernel engineers where I work and they were (I think)

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread David S. Miller
Date:Tue, 9 Jan 2001 17:14:33 -0800 (PST) From: Dave Zarzycki <[EMAIL PROTECTED]> On Tue, 9 Jan 2001, Ingo Molnar wrote: > then you'll love the zerocopy patch :-) Just use sendfile() or specify > MSG_NOCOPY to sendmsg(), and you'll see effective memory-to-card >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Dave Zarzycki wrote: > In user space, how do you know when its safe to reuse the buffer that > was handed to sendmsg() with the MSG_NOCOPY flag? Or does sendmsg() > with that flag block until the buffer isn't needed by the kernel any > more? If it does block, doesn't that

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dave Zarzycki
On Tue, 9 Jan 2001, Ingo Molnar wrote: > then you'll love the zerocopy patch :-) Just use sendfile() or specify > MSG_NOCOPY to sendmsg(), and you'll see effective memory-to-card > DMA-and-checksumming on cards that support it. I'm confused. In user space, how do you know when its safe to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Andrea Arcangeli
On Tue, Jan 09, 2001 at 09:10:24PM +0100, Ingo Molnar wrote: > > On Tue, 9 Jan 2001, Andrea Arcangeli wrote: > > > BTW, I noticed what is left in blk-13B seems to be my work (Jens's > > fixes for merging when the I/O queue is full are just been integrated > > in test1x). [...] > > it was

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
On Tue, 9 Jan 2001, Benjamin C.R. LaHaise wrote: > On Tue, 9 Jan 2001, Linus Torvalds wrote: > > > The _lower-level_ stuff (ie TCP and the drivers) want the "array of > > tuples", and again, they do NOT want an array of pages, because if > > somebody does two sendfile() calls that fit in one

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Andrea Arcangeli
On Wed, Jan 10, 2001 at 12:34:35AM +0100, Jens Axboe wrote: > Ah I see. It would be nice to base the QUEUE_NR_REQUEST on something else > than a static number. For example, 3000 per queue translates into 281Kb > of request slots per queue. On a typical system with a floppy, hard drive, > and

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Jens Axboe
On Wed, Jan 10 2001, Andrea Arcangeli wrote: > On Tue, Jan 09, 2001 at 09:12:04PM +0100, Jens Axboe wrote: > > I haven't heard anything beyond the raised QUEUE_NR_REQUEST, so I'd like to > > see what you have pending so we can merge :-). The tiotest seek increase was > > mainly due to the

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Andrea Arcangeli
On Tue, Jan 09, 2001 at 09:12:04PM +0100, Jens Axboe wrote: > I haven't heard anything beyond the raised QUEUE_NR_REQUEST, so I'd like to > see what you have pending so we can merge :-). The tiotest seek increase was > mainly due to the elevator having 3000 requests to juggle and thus being able

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dan Hollis
On Tue, 9 Jan 2001, Ingo Molnar wrote: > On Tue, 9 Jan 2001, Dan Hollis wrote: > > > This is not what senfile() does, it sends (to a network socket) a > > > file (from the page cache), nothing more. > > Ok in any case, it would be nice to have a generic sendfile() which works > > on any fd's -

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Benjamin C.R. LaHaise
On Tue, 9 Jan 2001, Linus Torvalds wrote: > The _lower-level_ stuff (ie TCP and the drivers) want the "array of > tuples", and again, they do NOT want an array of pages, because if > somebody does two sendfile() calls that fit in one packet, it really needs > an array of tuples. A kiobuf simply

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Dan Hollis wrote: > > This is not what senfile() does, it sends (to a network socket) a > > file (from the page cache), nothing more. > > Ok in any case, it would be nice to have a generic sendfile() which works > on any fd's - socket or otherwise. it's a bad name in that

[patch]: ac4 blk (was Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1)

2001-01-09 Thread Jens Axboe
On Tue, Jan 09 2001, Ingo Molnar wrote: > > > > but in 2.4, with the right patch from Jens, it doesnt suck anymore. ) > > > > Is this "right patch from Jens" on the radar for 2.4 inclusion? > > i do hope so! Here's a version against 2.4.0-ac4, blk-13B did not apply cleanly due to moving of i2o

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dan Hollis
On Tue, 9 Jan 2001, David S. Miller wrote: >Just extend sendfile to allow any fd to any fd. sendfile already >does file->socket and file->file. It only needs to be extended to >do socket->file. > This is not what senfile() does, it sends (to a network socket) a > file (from the page

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Stephen C. Tweedie <[EMAIL PROTECTED]> wrote: > >Jes has also got hard numbers for the performance advantages of >jumbograms on some of the networks he's been using, and you ain't >going to get udp jumbograms through a page-by-page API, ever. Wrong. The only

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread David S. Miller
Date:Tue, 9 Jan 2001 11:14:05 -0800 (PST) From: Dan Hollis <[EMAIL PROTECTED]> Just extend sendfile to allow any fd to any fd. sendfile already does file->socket and file->file. It only needs to be extended to do socket->file. This is not what senfile() does, it sends (to

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread David S. Miller
Date: Tue, 9 Jan 2001 16:27:49 +0100 (CET) From: Trond Myklebust <[EMAIL PROTECTED]> OK, but can you eventually generalize it to non-stream protocols (i.e. UDP)? Sure, this is what MSG_MORE is meant to accomodate. UDP could support it just fine. Later, David S. Miller [EMAIL

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread David S. Miller
Date: Tue, 9 Jan 2001 15:17:25 + From: "Stephen C. Tweedie" <[EMAIL PROTECTED]> Jes has also got hard numbers for the performance advantages of jumbograms on some of the networks he's been using, and you ain't going to get udp jumbograms through a page-by-page API, ever.

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread David S. Miller
Date: Tue, 9 Jan 2001 14:25:42 + From: "Stephen C. Tweedie" <[EMAIL PROTECTED]> Perhaps tcp can merge internal 4K requests, but if you're doing udp jumbograms (or STP or VIA), you do need an interface which can give the networking stack more than one page at once. All network

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
On Tue, 9 Jan 2001, Christoph Hellwig wrote: > > > > Look at sendfile(). You do NOT have a "bunch" of pages. > > > > Sendfile() is very much a page-at-a-time thing, and expects the actual IO > > layers to do it's own scatter-gather. > > > > So sendfile() doesn't want any array at all: it

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Christoph Hellwig
On Tue, Jan 09, 2001 at 12:55:51PM -0800, Linus Torvalds wrote: > > > On Tue, 9 Jan 2001, Christoph Hellwig wrote: > > > > Also the tuple argument you gave earlier isn't right in this specific case: > > > > when doing sendfile from pagecache to an fs, you have a bunch of pages, > > an offset

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
On Tue, 9 Jan 2001, Christoph Hellwig wrote: > > Also the tuple argument you gave earlier isn't right in this specific case: > > when doing sendfile from pagecache to an fs, you have a bunch of pages, > an offset in the first and a length that makes the data end before last > page's end. No.

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Christoph Hellwig
In article <[EMAIL PROTECTED]> you wrote: > On Tue, 9 Jan 2001, Ingo Molnar wrote: >> >> So i do believe that the networking >> code is properly designed in this respect, and this concept goes to the >> highest level of the networking code. > Absolutely. This is

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
On Tue, 9 Jan 2001, Ingo Molnar wrote: > >So i do believe that the networking > code is properly designed in this respect, and this concept goes to the > highest level of the networking code. Absolutely. This is why I have no conceptual problems with the

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Jens Axboe
On Tue, Jan 09 2001, Andrea Arcangeli wrote: > > > > Thats fine. Get me 128K-512K chunks nicely streaming into my raid controller > > > > and I'll be a happy man > > > > > > No problem, apply blk-13B and you'll get 512K chunks for SCSI and RAID. > > > > i cannot agree more - Jens' patch did

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Andrea Arcangeli wrote: > BTW, I noticed what is left in blk-13B seems to be my work (Jens's > fixes for merging when the I/O queue is full are just been integrated > in test1x). [...] it was Jens' [i think those were implemented by Jens entirely] batch-freeing changes

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On 9 Jan 2001, Linus Torvalds wrote: > I told David that he can fix the network zero-copy code two ways: either > he makes it _truly_ scatter-gather (an array of not just pages, but of > proper page-offset-length tuples), or he makes it just a single area and > lets the low-level TCP/whatever

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Andrea Arcangeli
On Tue, Jan 09, 2001 at 07:38:28PM +0100, Ingo Molnar wrote: > > On Tue, 9 Jan 2001, Jens Axboe wrote: > > > > > ever seen, this is why i quoted it - the talk was about block-IO > > > > performance, and Stephen said that our block IO sucks. It used to suck, > > > > but in 2.4, with the right

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread J Sloan
Alan Cox wrote: > > > it might not be important to others, but we do hold one particular > > SPECweb99 world record: on 2-way, 2 GB RAM, testing a load with a full > > And its real world value is exactly the same as the mindcraft NT values. Don't > forget that. In other words, devastating. jjs

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Christoph Hellwig wrote: > I didn't want to suggest that - I'm to clueless concerning networking > to even consider an internal design for network zero-copy IO. I'm just > talking about the VFS interface to the rest of the kernel. (well, i think you just cannot be clueless

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Christoph Hellwig <[EMAIL PROTECTED]> wrote: > >You get that multiple page call with kiobufs for free... No, you don't. kiobufs are crap. Face it. They do NOT allow proper multi-page scatter gather, regardless of what the kiobuf PR department has said. I've

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dan Hollis
On Wed, 10 Jan 2001, Andrew Morton wrote: > y'know our pals have patented it? > http://www.delphion.com/details?pn=US05845280__ Bad faith patent? Actionable, treble damages? -Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Dan Hollis
On Tue, 9 Jan 2001, Ingo Molnar wrote: > :-) I think sendfile() should also have its logical extensions: > receivefile(). I dont know how the HPUX implementation works, but in > Linux, right now it's only possible to sendfile() from a file to a socket. > The logical extension of this is to allow

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Chris Evans wrote: > > but in 2.4, with the right patch from Jens, it doesnt suck anymore. ) > > Is this "right patch from Jens" on the radar for 2.4 inclusion? i do hope so! Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Jens Axboe wrote: > > > ever seen, this is why i quoted it - the talk was about block-IO > > > performance, and Stephen said that our block IO sucks. It used to suck, > > > but in 2.4, with the right patch from Jens, it doesnt suck anymore. ) > > > > Thats fine. Get me

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Benjamin C.R. LaHaise wrote: > I've already got fully async read and write working via a helper thread ^^^ > for doing the bmaps when the page is not uptodate in the page cache. ^^^

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Christoph Hellwig
On Tue, Jan 09, 2001 at 12:05:59PM +0100, Ingo Molnar wrote: > > On Tue, 9 Jan 2001, Christoph Hellwig wrote: > > > > 2.4. In any case, the zerocopy code is 'kiovec in spirit' (uses > > > vectors of struct page *, offset, size entities), > > > Yep. That is why I was so worried aboit the

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 05:16:40PM +0100, Ingo Molnar wrote: > On Tue, 9 Jan 2001, Stephen C. Tweedie wrote: > > i'm talking about kiovecs not kiobufs (because those are equivalent to a > fragmented packet - every packet fragment can be anywhere). Initializing a > kiovec involves touching a

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Stephen C. Tweedie
Hi, On Tue, Jan 09, 2001 at 12:30:39PM -0500, Benjamin C.R. LaHaise wrote: > On Tue, 9 Jan 2001, Ingo Molnar wrote: > > > this is why i ment that *right now* kiobufs are not suited for networking, > > at least the way we do it. Maybe if kiobufs had the same kind of internal > > structure as

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Chris Evans
On Tue, 9 Jan 2001, Ingo Molnar wrote: > This is one of the busiest and most complex block-IO Linux systems i've > ever seen, this is why i quoted it - the talk was about block-IO > performance, and Stephen said that our block IO sucks. It used to suck, > but in 2.4, with the right patch from

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Christoph Hellwig
On Tue, Jan 09, 2001 at 10:38:30AM -0500, Benjamin C.R. LaHaise wrote: > What you're completely ignoring is that sendpages is lacking a huge amount > of functionality that is *needed*. I can't implement clean async io on > top of sendpages -- it'll require keeping 1 task around per outstanding >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Manfred Spraul
sct wrote: > We've already got measurements showing how insane this is. Raw IO > requests, plus internal pagebuf contiguous requests from XFS, have to > get broken down into page-sized chunks by the current ll_rw_block() > API, only to get reassembled by the make_request code. It's >

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Jens Axboe
On Tue, Jan 09 2001, Alan Cox wrote: > > ever seen, this is why i quoted it - the talk was about block-IO > > performance, and Stephen said that our block IO sucks. It used to suck, > > but in 2.4, with the right patch from Jens, it doesnt suck anymore. ) > > Thats fine. Get me 128K-512K chunks

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Benjamin C.R. LaHaise
On Tue, 9 Jan 2001, Ingo Molnar wrote: > this is why i ment that *right now* kiobufs are not suited for networking, > at least the way we do it. Maybe if kiobufs had the same kind of internal > structure as sk_frag (ie. array of (page,offset,size) triples, not array > of pages), that would work

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Alan Cox
> ever seen, this is why i quoted it - the talk was about block-IO > performance, and Stephen said that our block IO sucks. It used to suck, > but in 2.4, with the right patch from Jens, it doesnt suck anymore. ) Thats fine. Get me 128K-512K chunks nicely streaming into my raid controller and

Re: [PLEASE-TESTME] Zerocopy networking patch, 2.4.0-1

2001-01-09 Thread Ingo Molnar
On Tue, 9 Jan 2001, Alan Cox wrote: > > > We have already shown that the IO-plugging API sucks, I'm afraid. > > > > it might not be important to others, but we do hold one particular > > SPECweb99 world record: on 2-way, 2 GB RAM, testing a load with a full > > And its real world value is

  1   2   3   >