Re: Is sendfile all that sexy?

2001-02-21 Thread Pavel Machek
Hi! > > And no, I don't actually hink that sendfile() is all that hot. It was > > _very_ easy to implement, and can be considered a 5-minute hack to give > > a feature that fit very well in the MM architecture, and that the Apache > > folks had already been using on other architectures. > > The

Re: Is sendfile all that sexy?

2001-02-21 Thread Pavel Machek
Hi! And no, I don't actually hink that sendfile() is all that hot. It was _very_ easy to implement, and can be considered a 5-minute hack to give a feature that fit very well in the MM architecture, and that the Apache folks had already been using on other architectures. The current

Re: Is sendfile all that sexy?

2001-01-26 Thread Anton Blanchard
Hi Dave, How are the VB withdrawal symptoms going? :) > Anton, why are you always returning -1 (which means error for the > smb_message[] array functions) when using sendfile? Returning -1 tells the higher level code that we actually sent the bytes out ourselves and not to bother doing it.

Re: Is sendfile all that sexy?

2001-01-26 Thread David S. Miller
Anton Blanchard writes: > diff -u -u -r1.257 reply.c > --- source/smbd/reply.c 2001/01/24 19:34:53 1.257 > +++ source/smbd/reply.c 2001/01/26 05:38:53 > @@ -2383,6 +2391,51 @@ ... > +while(nread) { > +int nwritten; > +nwritten =

Re: Is sendfile all that sexy?

2001-01-26 Thread David S. Miller
Anton Blanchard writes: diff -u -u -r1.257 reply.c --- source/smbd/reply.c 2001/01/24 19:34:53 1.257 +++ source/smbd/reply.c 2001/01/26 05:38:53 @@ -2383,6 +2391,51 @@ ... +while(nread) { +int nwritten; +nwritten = sendfile(smbd_server_fd(),

Re: Is sendfile all that sexy?

2001-01-26 Thread Anton Blanchard
Hi Dave, How are the VB withdrawal symptoms going? :) Anton, why are you always returning -1 (which means error for the smb_message[] array functions) when using sendfile? Returning -1 tells the higher level code that we actually sent the bytes out ourselves and not to bother doing it.

Re: Is sendfile all that sexy?

2001-01-25 Thread Anton Blanchard
> Do you have it at a URL? The patch is small so I have attached it to this email. It should apply to the samba CVS tree. Remember this is still a hack and I need to add code to ensure the file is not truncated and we sendfile() less than we promised. (After talking to tridge and davem, this

Re: Is sendfile all that sexy?

2001-01-25 Thread James Sutherland
On Thu, 25 Jan 2001, bert hubert wrote: > On Thu, Jan 25, 2001 at 09:06:33AM +, James Sutherland wrote: > > > performance than it would for an httpd, because of the long-lived > > sessions, but rewriting it as a state machine (no forking, threads or > > other crap, just use non-blocking

Re: Is sendfile all that sexy?

2001-01-25 Thread Sasi Peter
On Thu, 25 Jan 2001, Anton Blanchard wrote: > I have patches for samba to do sendfile. Making a tux module does not make > sense to me, especially since we are nowhere near the limits of samba in > userspace. Once userspace samba can run no faster, then we should think > about other options. Do

Re: Is sendfile all that sexy?

2001-01-25 Thread Anton Blanchard
> No plans for samba to use sendfile? Even better make it a tux-like module? > (that would enable Netware-Linux like performance with the standard > kernel... would be cool afterall ;) I have patches for samba to do sendfile. Making a tux module does not make sense to me, especially since we

Re: Is sendfile all that sexy?

2001-01-25 Thread bert hubert
On Thu, Jan 25, 2001 at 09:06:33AM +, James Sutherland wrote: > performance than it would for an httpd, because of the long-lived > sessions, but rewriting it as a state machine (no forking, threads or > other crap, just use non-blocking I/O) would probably make much more > sense. >From a

Re: Is sendfile all that sexy?

2001-01-25 Thread James Sutherland
On Thu, 25 Jan 2001, Alan Cox wrote: > > I think, that is not what we need. Once Ingo wrote, that since HTTP > > serving can also be viewed as a kind of fileserving, it should be > > possible to create a TUX like module for the same framwork, that serves > > using the SMB protocol instead of

Re: Is sendfile all that sexy?

2001-01-25 Thread James Sutherland
On Thu, 25 Jan 2001, Alan Cox wrote: I think, that is not what we need. Once Ingo wrote, that since HTTP serving can also be viewed as a kind of fileserving, it should be possible to create a TUX like module for the same framwork, that serves using the SMB protocol instead of HTTP...

Re: Is sendfile all that sexy?

2001-01-25 Thread bert hubert
On Thu, Jan 25, 2001 at 09:06:33AM +, James Sutherland wrote: performance than it would for an httpd, because of the long-lived sessions, but rewriting it as a state machine (no forking, threads or other crap, just use non-blocking I/O) would probably make much more sense. From a kernel

Re: Is sendfile all that sexy?

2001-01-25 Thread Anton Blanchard
No plans for samba to use sendfile? Even better make it a tux-like module? (that would enable Netware-Linux like performance with the standard kernel... would be cool afterall ;) I have patches for samba to do sendfile. Making a tux module does not make sense to me, especially since we are

Re: Is sendfile all that sexy?

2001-01-25 Thread Sasi Peter
On Thu, 25 Jan 2001, Anton Blanchard wrote: I have patches for samba to do sendfile. Making a tux module does not make sense to me, especially since we are nowhere near the limits of samba in userspace. Once userspace samba can run no faster, then we should think about other options. Do you

Re: Is sendfile all that sexy?

2001-01-25 Thread James Sutherland
On Thu, 25 Jan 2001, bert hubert wrote: On Thu, Jan 25, 2001 at 09:06:33AM +, James Sutherland wrote: performance than it would for an httpd, because of the long-lived sessions, but rewriting it as a state machine (no forking, threads or other crap, just use non-blocking I/O) would

Re: Is sendfile all that sexy?

2001-01-25 Thread Anton Blanchard
Do you have it at a URL? The patch is small so I have attached it to this email. It should apply to the samba CVS tree. Remember this is still a hack and I need to add code to ensure the file is not truncated and we sendfile() less than we promised. (After talking to tridge and davem, this

Re: Is sendfile all that sexy?

2001-01-24 Thread Alan Cox
> I think, that is not what we need. Once Ingo wrote, that since HTTP > serving can also be viewed as a kind of fileserving, it should be > possible to create a TUX like module for the same framwork, that serves > using the SMB protocol instead of HTTP... Kernel SMB is basically not a sane

Re: Is sendfile all that sexy?

2001-01-24 Thread James Sutherland
On Wed, 24 Jan 2001, Sasi Peter wrote: > > AIUI, Jeff Merkey was working on loading "userspace" apps into the > kernel > > to tackle this sort of problem generically. I don't know if he's > tried it > > with Samba - the forking would probably be a problem... > > I think, that is not what we

Re: Is sendfile all that sexy?

2001-01-24 Thread Sasi Peter
> AIUI, Jeff Merkey was working on loading "userspace" apps into the kernel > to tackle this sort of problem generically. I don't know if he's tried it > with Samba - the forking would probably be a problem... I think, that is not what we need. Once Ingo wrote, that since HTTP serving can

Re: Is sendfile all that sexy?

2001-01-24 Thread James Sutherland
On Wed, 24 Jan 2001, Sasi Peter wrote: > On 14 Jan 2001, Linus Torvalds wrote: > > > The only obvious use for it is file serving, and as high-performance > > file serving tends to end up as a kernel module in the end anyway (the > > only hold-out is samba, and that's been discussed too),

Re: Is sendfile all that sexy?

2001-01-24 Thread James Sutherland
On Wed, 24 Jan 2001, Sasi Peter wrote: On 14 Jan 2001, Linus Torvalds wrote: The only obvious use for it is file serving, and as high-performance file serving tends to end up as a kernel module in the end anyway (the only hold-out is samba, and that's been discussed too), "sendfile()"

Re: Is sendfile all that sexy?

2001-01-24 Thread Sasi Peter
AIUI, Jeff Merkey was working on loading "userspace" apps into the kernel to tackle this sort of problem generically. I don't know if he's tried it with Samba - the forking would probably be a problem... I think, that is not what we need. Once Ingo wrote, that since HTTP serving can also

Re: Is sendfile all that sexy?

2001-01-24 Thread James Sutherland
On Wed, 24 Jan 2001, Sasi Peter wrote: AIUI, Jeff Merkey was working on loading "userspace" apps into the kernel to tackle this sort of problem generically. I don't know if he's tried it with Samba - the forking would probably be a problem... I think, that is not what we need. Once

Re: Is sendfile all that sexy?

2001-01-24 Thread Alan Cox
I think, that is not what we need. Once Ingo wrote, that since HTTP serving can also be viewed as a kind of fileserving, it should be possible to create a TUX like module for the same framwork, that serves using the SMB protocol instead of HTTP... Kernel SMB is basically not a sane idea.

Re: Is sendfile all that sexy?

2001-01-23 Thread Sasi Peter
On 14 Jan 2001, Linus Torvalds wrote: > The only obvious use for it is file serving, and as high-performance > file serving tends to end up as a kernel module in the end anyway (the > only hold-out is samba, and that's been discussed too), "sendfile()" > really is more a proof of concept than

Re: Is sendfile all that sexy?

2001-01-23 Thread James Sutherland
On Tue, 23 Jan 2001, Helge Hafting wrote: > James Sutherland wrote: > > > > On Mon, 22 Jan 2001, Helge Hafting wrote: > > > > > And when the next user wants the same webpage/file you read it from > > > the RAID again? Seems to me you loose the benefit of caching stuff in > > > memory with this

Re: Is sendfile all that sexy?

2001-01-23 Thread Helge Hafting
James Sutherland wrote: > > On Mon, 22 Jan 2001, Helge Hafting wrote: > > > And when the next user wants the same webpage/file you read it from > > the RAID again? Seems to me you loose the benefit of caching stuff in > > memory with this scheme. Sure - the RAID controller might have some > >

Re: Is sendfile all that sexy?

2001-01-23 Thread Helge Hafting
James Sutherland wrote: On Mon, 22 Jan 2001, Helge Hafting wrote: And when the next user wants the same webpage/file you read it from the RAID again? Seems to me you loose the benefit of caching stuff in memory with this scheme. Sure - the RAID controller might have some cache, but

Re: Is sendfile all that sexy?

2001-01-23 Thread James Sutherland
On Tue, 23 Jan 2001, Helge Hafting wrote: James Sutherland wrote: On Mon, 22 Jan 2001, Helge Hafting wrote: And when the next user wants the same webpage/file you read it from the RAID again? Seems to me you loose the benefit of caching stuff in memory with this scheme. Sure -

Re: Is sendfile all that sexy?

2001-01-23 Thread Sasi Peter
On 14 Jan 2001, Linus Torvalds wrote: The only obvious use for it is file serving, and as high-performance file serving tends to end up as a kernel module in the end anyway (the only hold-out is samba, and that's been discussed too), "sendfile()" really is more a proof of concept than

Re: Is sendfile all that sexy?

2001-01-22 Thread Ion Badulescu
On Mon, 22 Jan 2001 12:01:23 -0800 (PST), David Lang <[EMAIL PROTECTED]> wrote: > how about always_defragment (or whatever the option is now called) so that > your routing box always reassembles packets and then fragments them to the > correct size for the next segment? wouldn't this do the job?

Re: Is sendfile all that sexy?

2001-01-22 Thread David Lang
:37:07 -0700 > From: Val Henson <[EMAIL PROTECTED]> > To: David Lang <[EMAIL PROTECTED]> > Cc: [EMAIL PROTECTED], Linus Torvalds <[EMAIL PROTECTED]> > Subject: Re: Is sendfile all that sexy? > > On Mon, Jan 22, 2001 at 10:27:58AM -0800, David Lang wrote: &g

Re: Is sendfile all that sexy?

2001-01-22 Thread Val Henson
On Mon, Jan 22, 2001 at 10:27:58AM -0800, David Lang wrote: > On Mon, 22 Jan 2001, Val Henson wrote: > > > There is a use for an optimized socket->socket transfer - proxying > > high speed TCP connections. It would be exciting if the zerocopy > > networking framework led to a decent

Re: Is sendfile all that sexy?

2001-01-22 Thread Linus Torvalds
On Mon, 22 Jan 2001, Val Henson wrote: > On Wed, Jan 17, 2001 at 11:32:35AM -0800, Linus Torvalds wrote: > > > > However, for socket->socket, we would not have such an advantage. A > > socket->socket sendfile() would not avoid any copies the way the > > networking is done today. That _may_

Re: Is sendfile all that sexy?

2001-01-22 Thread David Lang
On Mon, 22 Jan 2001, Val Henson wrote: > There is a use for an optimized socket->socket transfer - proxying > high speed TCP connections. It would be exciting if the zerocopy > networking framework led to a decent socket->socket transfer. if you are proxying connextions you should really be

Re: Is sendfile all that sexy?

2001-01-22 Thread Val Henson
On Wed, Jan 17, 2001 at 11:32:35AM -0800, Linus Torvalds wrote: > In article <[EMAIL PROTECTED]>, > Ben Mansell <[EMAIL PROTECTED]> wrote: > > > >The current sendfile() has the limitation that it can't read data from > >a socket. Would it be another 5-minute hack to remove this limitation, so >

Re: Is sendfile all that sexy?

2001-01-22 Thread James Sutherland
On Mon, 22 Jan 2001, Helge Hafting wrote: > And when the next user wants the same webpage/file you read it from > the RAID again? Seems to me you loose the benefit of caching stuff in > memory with this scheme. Sure - the RAID controller might have some > cache, but it is usually smaller than

Re: Is sendfile all that sexy?

2001-01-22 Thread Helge Hafting
James Sutherland wrote: > > On Sat, 20 Jan 2001, Linus Torvalds wrote: > > > > > > > On Sat, 20 Jan 2001, Roman Zippel wrote: > > > > > > On Sat, 20 Jan 2001, Linus Torvalds wrote: > > > > > > > But point-to-point also means that you don't get any real advantage from > > > > doing things like

Re: Is sendfile all that sexy?

2001-01-22 Thread Helge Hafting
James Sutherland wrote: On Sat, 20 Jan 2001, Linus Torvalds wrote: On Sat, 20 Jan 2001, Roman Zippel wrote: On Sat, 20 Jan 2001, Linus Torvalds wrote: But point-to-point also means that you don't get any real advantage from doing things like device-to-device DMA.

Re: Is sendfile all that sexy?

2001-01-22 Thread James Sutherland
On Mon, 22 Jan 2001, Helge Hafting wrote: And when the next user wants the same webpage/file you read it from the RAID again? Seems to me you loose the benefit of caching stuff in memory with this scheme. Sure - the RAID controller might have some cache, but it is usually smaller than main

Re: Is sendfile all that sexy?

2001-01-22 Thread Val Henson
On Wed, Jan 17, 2001 at 11:32:35AM -0800, Linus Torvalds wrote: In article [EMAIL PROTECTED], Ben Mansell [EMAIL PROTECTED] wrote: The current sendfile() has the limitation that it can't read data from a socket. Would it be another 5-minute hack to remove this limitation, so you could

Re: Is sendfile all that sexy?

2001-01-22 Thread David Lang
On Mon, 22 Jan 2001, Val Henson wrote: There is a use for an optimized socket-socket transfer - proxying high speed TCP connections. It would be exciting if the zerocopy networking framework led to a decent socket-socket transfer. if you are proxying connextions you should really be looking

Re: Is sendfile all that sexy?

2001-01-22 Thread Linus Torvalds
On Mon, 22 Jan 2001, Val Henson wrote: On Wed, Jan 17, 2001 at 11:32:35AM -0800, Linus Torvalds wrote: However, for socket-socket, we would not have such an advantage. A socket-socket sendfile() would not avoid any copies the way the networking is done today. That _may_ change, of

Re: Is sendfile all that sexy?

2001-01-22 Thread Val Henson
On Mon, Jan 22, 2001 at 10:27:58AM -0800, David Lang wrote: On Mon, 22 Jan 2001, Val Henson wrote: There is a use for an optimized socket-socket transfer - proxying high speed TCP connections. It would be exciting if the zerocopy networking framework led to a decent socket-socket

Re: Is sendfile all that sexy?

2001-01-22 Thread David Lang
:07 -0700 From: Val Henson [EMAIL PROTECTED] To: David Lang [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], Linus Torvalds [EMAIL PROTECTED] Subject: Re: Is sendfile all that sexy? On Mon, Jan 22, 2001 at 10:27:58AM -0800, David Lang wrote: On Mon, 22 Jan 2001, Val Henson wrote: There is a use

Re: Is sendfile all that sexy?

2001-01-22 Thread Ion Badulescu
On Mon, 22 Jan 2001 12:01:23 -0800 (PST), David Lang [EMAIL PROTECTED] wrote: how about always_defragment (or whatever the option is now called) so that your routing box always reassembles packets and then fragments them to the correct size for the next segment? wouldn't this do the job? It

Re: Is sendfile all that sexy?

2001-01-21 Thread David Woodhouse
On Sat, 20 Jan 2001, Linus Torvalds wrote: > There's no no-no here: you can even create the "struct page"s on demand, > and create a dummy local zone that contains them that they all point back > to. It should be trivial - nobody else cares about those pages or that > zone anyway. > > This is

RE: Is sendfile all that sexy?

2001-01-21 Thread LA Walsh
FYI - Another use sendfile(2) might be used for. Suppose you were to generate large amounts of data -- maybe kernel profiling data, audit data, whatever, in the kernel. You want to pull that data out as fast as possible and write it to a disk or network socket. Normally, I

Re: Is sendfile all that sexy?

2001-01-21 Thread kuznet
Hello! > "struct page" tricks, some macros etc WILL NOT WORK. In particular, we do > not currently have a good "page_to_bus/phys()" function. That means that > anybody trying to do DMA to this page is currently screwed, simply because > he has no good way of getting the physical address. We

Re: Is sendfile all that sexy?

2001-01-21 Thread Ingo Molnar
On Sun, 21 Jan 2001, James Sutherland wrote: > For many applications, yes - but think about a file server for a > moment. 99% of the data read from the RAID (or whatever) is really > aimed at the appropriate NIC - going via main memory would just slow > things down. patently wrong. Compare the

Re: Is sendfile all that sexy?

2001-01-21 Thread James Sutherland
On Sat, 20 Jan 2001, Linus Torvalds wrote: > > > On Sat, 20 Jan 2001, Roman Zippel wrote: > > > > On Sat, 20 Jan 2001, Linus Torvalds wrote: > > > > > But point-to-point also means that you don't get any real advantage from > > > doing things like device-to-device DMA. Because the links are > >

Re: Is sendfile all that sexy?

2001-01-21 Thread James Sutherland
On Sat, 20 Jan 2001, Linus Torvalds wrote: On Sat, 20 Jan 2001, Roman Zippel wrote: On Sat, 20 Jan 2001, Linus Torvalds wrote: But point-to-point also means that you don't get any real advantage from doing things like device-to-device DMA. Because the links are asynchronous,

Re: Is sendfile all that sexy?

2001-01-21 Thread Ingo Molnar
On Sun, 21 Jan 2001, James Sutherland wrote: For many applications, yes - but think about a file server for a moment. 99% of the data read from the RAID (or whatever) is really aimed at the appropriate NIC - going via main memory would just slow things down. patently wrong. Compare the

Re: Is sendfile all that sexy?

2001-01-21 Thread kuznet
Hello! "struct page" tricks, some macros etc WILL NOT WORK. In particular, we do not currently have a good "page_to_bus/phys()" function. That means that anybody trying to do DMA to this page is currently screwed, simply because he has no good way of getting the physical address. We already

RE: Is sendfile all that sexy?

2001-01-21 Thread LA Walsh
FYI - Another use sendfile(2) might be used for. Suppose you were to generate large amounts of data -- maybe kernel profiling data, audit data, whatever, in the kernel. You want to pull that data out as fast as possible and write it to a disk or network socket. Normally, I

Re: Is sendfile all that sexy?

2001-01-21 Thread David Woodhouse
On Sat, 20 Jan 2001, Linus Torvalds wrote: There's no no-no here: you can even create the "struct page"s on demand, and create a dummy local zone that contains them that they all point back to. It should be trivial - nobody else cares about those pages or that zone anyway. This is very

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: > But think like a good hardware designer. > > In 99% of all cases, where do you want the results of a read to end up? > Where do you want the contents of a write to come from? > > Right. Memory. > > Now, optimize for the common case. Make the

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: > Now, there are things to look out for: when you do these kinds of dummy > "struct page" tricks, some macros etc WILL NOT WORK. In particular, we do > not currently have a good "page_to_bus/phys()" function. That means that > anybody trying to do

RE: Is sendfile all that sexy?

2001-01-20 Thread David Schwartz
> I'm _not_ seeing the point for a high-performance link to have a generic > packet buffer. > > Linus Well suppose your RAID controller can take over control of disks distributed throughout your I/O subsystem. If you assume the bandwidth of the I/O subsystem is not the

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001, Roman Zippel wrote: > > On Sat, 20 Jan 2001, Linus Torvalds wrote: > > > But point-to-point also means that you don't get any real advantage from > > doing things like device-to-device DMA. Because the links are > > asynchronous, you need buffers in between them anyway,

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001, Roman Zippel wrote: > > AFAIK as long as that dummy page struct is only used in the page cache, > that should work, but you get new problems as soon as you map the page > also into a user process (grep for CONFIG_DISCONTIGMEM under > include/asm-mips64 to see the needed

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: > But point-to-point also means that you don't get any real advantage from > doing things like device-to-device DMA. Because the links are > asynchronous, you need buffers in between them anyway, and there is no > bandwidth advantage of not going

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: > There's no no-no here: you can even create the "struct page"s on demand, > and create a dummy local zone that contains them that they all point back > to. It should be trivial - nobody else cares about those pages or that > zone anyway. AFAIK as

Re: Is sendfile all that sexy?

2001-01-20 Thread Mo McKinlay
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Today, Linus Torvalds ([EMAIL PROTECTED]) wrote: > Just wait. My crystal ball is infallible. One of these days, that line will be your downfall :-) *grins* Mo. - -- Mo McKinlay [EMAIL PROTECTED] -

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On 20 Jan 2001, Kai Henningsen wrote: > > Then again, I could easily see those I/O devices go the general embedded > route, which in a decade or two could well mean they run some sort of > embedded Linux on the controller. > > Which would make some features rather easy to implement. I'm

Re: Is sendfile all that sexy?

2001-01-20 Thread Kai Henningsen
[EMAIL PROTECTED] (Linus Torvalds) wrote on 18.01.01 in <[EMAIL PROTECTED]>: > (Short and sweet: most hogh-performance people want point-to-point serial > line IO with no hops, because it's a known art to make that go fast. No > general-case routing in hardware - if you want to go as fast as

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001 [EMAIL PROTECTED] wrote: > > Actually, as long as there is no "struct page" there _are_ problems. > > This is why the NUMA stuff was brought up - it would require that there > > be a mem_map for the PCI pages.. (to do ref-counting etc). > > I see. > > Is this strong

Re: Is sendfile all that sexy?

2001-01-20 Thread kuznet
Hello! > Actually, as long as there is no "struct page" there _are_ problems. > This is why the NUMA stuff was brought up - it would require that there > be a mem_map for the PCI pages.. (to do ref-counting etc). I see. Is this strong "no-no-no"? What is obstacle to allow "struct page" to sit

Re: Is sendfile all that sexy?

2001-01-20 Thread kuznet
Hello! Actually, as long as there is no "struct page" there _are_ problems. This is why the NUMA stuff was brought up - it would require that there be a mem_map for the PCI pages.. (to do ref-counting etc). I see. Is this strong "no-no-no"? What is obstacle to allow "struct page" to sit

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001 [EMAIL PROTECTED] wrote: Actually, as long as there is no "struct page" there _are_ problems. This is why the NUMA stuff was brought up - it would require that there be a mem_map for the PCI pages.. (to do ref-counting etc). I see. Is this strong "no-no-no"? What

Re: Is sendfile all that sexy?

2001-01-20 Thread Kai Henningsen
[EMAIL PROTECTED] (Linus Torvalds) wrote on 18.01.01 in [EMAIL PROTECTED]: (Short and sweet: most hogh-performance people want point-to-point serial line IO with no hops, because it's a known art to make that go fast. No general-case routing in hardware - if you want to go as fast as the

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On 20 Jan 2001, Kai Henningsen wrote: Then again, I could easily see those I/O devices go the general embedded route, which in a decade or two could well mean they run some sort of embedded Linux on the controller. Which would make some features rather easy to implement. I'm not

Re: Is sendfile all that sexy?

2001-01-20 Thread Mo McKinlay
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Today, Linus Torvalds ([EMAIL PROTECTED]) wrote: Just wait. My crystal ball is infallible. One of these days, that line will be your downfall :-) *grins* Mo. - -- Mo McKinlay [EMAIL PROTECTED] -

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: There's no no-no here: you can even create the "struct page"s on demand, and create a dummy local zone that contains them that they all point back to. It should be trivial - nobody else cares about those pages or that zone anyway. AFAIK as

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: But point-to-point also means that you don't get any real advantage from doing things like device-to-device DMA. Because the links are asynchronous, you need buffers in between them anyway, and there is no bandwidth advantage of not going

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001, Roman Zippel wrote: AFAIK as long as that dummy page struct is only used in the page cache, that should work, but you get new problems as soon as you map the page also into a user process (grep for CONFIG_DISCONTIGMEM under include/asm-mips64 to see the needed

Re: Is sendfile all that sexy?

2001-01-20 Thread Linus Torvalds
On Sat, 20 Jan 2001, Roman Zippel wrote: On Sat, 20 Jan 2001, Linus Torvalds wrote: But point-to-point also means that you don't get any real advantage from doing things like device-to-device DMA. Because the links are asynchronous, you need buffers in between them anyway, and there

RE: Is sendfile all that sexy?

2001-01-20 Thread David Schwartz
I'm _not_ seeing the point for a high-performance link to have a generic packet buffer. Linus Well suppose your RAID controller can take over control of disks distributed throughout your I/O subsystem. If you assume the bandwidth of the I/O subsystem is not the

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: Now, there are things to look out for: when you do these kinds of dummy "struct page" tricks, some macros etc WILL NOT WORK. In particular, we do not currently have a good "page_to_bus/phys()" function. That means that anybody trying to do DMA

Re: Is sendfile all that sexy?

2001-01-20 Thread Roman Zippel
Hi, On Sat, 20 Jan 2001, Linus Torvalds wrote: But think like a good hardware designer. In 99% of all cases, where do you want the results of a read to end up? Where do you want the contents of a write to come from? Right. Memory. Now, optimize for the common case. Make the common

Re: Is sendfile all that sexy?

2001-01-19 Thread Pavel Machek
Hi! > > And no, I don't actually hink that sendfile() is all that hot. It was > > _very_ easy to implement, and can be considered a 5-minute hack to give > > a feature that fit very well in the MM architecture, and that the Apache > > folks had already been using on other architectures. > > The

Re: Is sendfile all that sexy?

2001-01-19 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote: >Hello! > >> It's about direct i/o from/to pages, > >Yes. Formally, there are no problems to send to tcp directly from io space. Actually, as long as there is no "struct page" there _are_ problems. This is why the NUMA stuff was brought

Re: Is sendfile all that sexy?

2001-01-19 Thread kuznet
Hello! > It's about direct i/o from/to pages, Yes. Formally, there are no problems to send to tcp directly from io space. But could someone explain me one thing. Does bus-mastering from io really work? And if it does, is it enough fast? At least, looking at my book on pci, I do not understand

Re: Is sendfile all that sexy?

2001-01-19 Thread Rogier Wolff
Linus Torvalds wrote: > > I wrote a driver for a zoran-chipset frame-grabber card. The "natural" > > way to save a video stream was exactly the way it came out of the > > card. And the card was structured that you could put on an "mpeg > > decoder" (or encoder) chip, and you could DMA the stream

Re: Is sendfile all that sexy? (fwd)

2001-01-19 Thread Andrea Arcangeli
On Fri, Jan 19, 2001 at 11:58:03AM +0100, Rogier Wolff wrote: > Now if we design the NUMA support correctly, just filling in "disk has > a seek-time of 10ms, and 20Mb per second throughput when accessed > linearly" NUMA may on it's own "tune" the swapper to do the right > thing. And once

Re: Is sendfile all that sexy? (fwd)

2001-01-19 Thread Rogier Wolff
Linus Torvalds wrote: > I do not know of _any_ disk controllers that let you map the controller > buffers over PCI. Which means that with current hardware, you have to > assume that the disk is the initiator of the PCI-PCI DMA requests. Agreed? I personally don't have driver-writing experience

Re: Is sendfile all that sexy?

2001-01-19 Thread Andre Hedrick
On Fri, 19 Jan 2001, Roman Zippel wrote: > Hi, > > On Thu, 18 Jan 2001, Linus Torvalds wrote: > > > > I agree, it's device dependent, but such hardware exists. > > > > Show me any practical case where the hardware actually exists. > > http://www.augan.com > > > I do not know of _any_ disk

Re: Is sendfile all that sexy?

2001-01-19 Thread Roman Zippel
Hi, On Thu, 18 Jan 2001, Linus Torvalds wrote: > > I agree, it's device dependent, but such hardware exists. > > Show me any practical case where the hardware actually exists. http://www.augan.com > I do not know of _any_ disk controllers that let you map the controller > buffers over PCI.

Re: Is sendfile all that sexy?

2001-01-19 Thread Roman Zippel
Hi, On Thu, 18 Jan 2001, Linus Torvalds wrote: I agree, it's device dependent, but such hardware exists. Show me any practical case where the hardware actually exists. http://www.augan.com I do not know of _any_ disk controllers that let you map the controller buffers over PCI. Which

Re: Is sendfile all that sexy?

2001-01-19 Thread Andre Hedrick
On Fri, 19 Jan 2001, Roman Zippel wrote: Hi, On Thu, 18 Jan 2001, Linus Torvalds wrote: I agree, it's device dependent, but such hardware exists. Show me any practical case where the hardware actually exists. http://www.augan.com I do not know of _any_ disk controllers that

Re: Is sendfile all that sexy? (fwd)

2001-01-19 Thread Rogier Wolff
Linus Torvalds wrote: I do not know of _any_ disk controllers that let you map the controller buffers over PCI. Which means that with current hardware, you have to assume that the disk is the initiator of the PCI-PCI DMA requests. Agreed? I personally don't have driver-writing experience for

Re: Is sendfile all that sexy? (fwd)

2001-01-19 Thread Andrea Arcangeli
On Fri, Jan 19, 2001 at 11:58:03AM +0100, Rogier Wolff wrote: Now if we design the NUMA support correctly, just filling in "disk has a seek-time of 10ms, and 20Mb per second throughput when accessed linearly" NUMA may on it's own "tune" the swapper to do the right thing. And once parametrized

Re: Is sendfile all that sexy?

2001-01-19 Thread Rogier Wolff
Linus Torvalds wrote: I wrote a driver for a zoran-chipset frame-grabber card. The "natural" way to save a video stream was exactly the way it came out of the card. And the card was structured that you could put on an "mpeg decoder" (or encoder) chip, and you could DMA the stream directly

Re: Is sendfile all that sexy?

2001-01-19 Thread kuznet
Hello! It's about direct i/o from/to pages, Yes. Formally, there are no problems to send to tcp directly from io space. But could someone explain me one thing. Does bus-mastering from io really work? And if it does, is it enough fast? At least, looking at my book on pci, I do not understand

Re: Is sendfile all that sexy?

2001-01-19 Thread Linus Torvalds
In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: Hello! It's about direct i/o from/to pages, Yes. Formally, there are no problems to send to tcp directly from io space. Actually, as long as there is no "struct page" there _are_ problems. This is why the NUMA stuff was brought up - it

Re: Is sendfile all that sexy?

2001-01-19 Thread Pavel Machek
Hi! And no, I don't actually hink that sendfile() is all that hot. It was _very_ easy to implement, and can be considered a 5-minute hack to give a feature that fit very well in the MM architecture, and that the Apache folks had already been using on other architectures. The current

Re: Is sendfile all that sexy?

2001-01-18 Thread Alan Cox
> Which in turn implies that the non-disk target hardware has to be able to > have a PCI-mapped memory buffer for the source or the destination, AND > they have to be able to cope with the fact that the data you get off the > disk will have to be the raw data at 512-byte granularity. And that

Re: Is sendfile all that sexy?

2001-01-18 Thread Linus Torvalds
In article <[EMAIL PROTECTED]>, Russell Leighton <[EMAIL PROTECTED]> wrote: > >"copy this fd to that one, and optimize that if you can" > >... isn't this Larry M's "splice" (http://www.bitmover.com/lm/papers/splice.ps)? We talked extensively about "splice()" with Larry. It was one of the

  1   2   3   >