Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Wed, 31 Jan 2001, Malcolm Beattie wrote:
> Without the raised tcp_wmem setting I was getting 81 MByte/s. With
> tcp_wmem set as above I got 86 MByte/s. Nice increase. Any other
> setting I can tweak apart from {r,w}mem_max and tcp_{w,r}mem? The CPU
> on the client (350 MHz PII) is the bottleneck: gensink4 maxes out at
> 69 Mbyte/s pulling TCP from the server and 94 Mbyte/s pushing. (The
> other system, 733 MHz PIII pushes >100MByte/s UDP with ttcp but the
> client drops most of it).
you can speed up the client significantly by using the MSG_TRUNC option
('truncate message'). It will zap incoming data without copying it into
user-space. (you can use this for the 'bulk transfer' part - the initial
protocol handling code needs to see the actual data.) This way you should
be able to saturate the server even more.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Wed, 31 Jan 2001, Malcolm Beattie wrote:
Without the raised tcp_wmem setting I was getting 81 MByte/s. With
tcp_wmem set as above I got 86 MByte/s. Nice increase. Any other
setting I can tweak apart from {r,w}mem_max and tcp_{w,r}mem? The CPU
on the client (350 MHz PII) is the bottleneck: gensink4 maxes out at
69 Mbyte/s pulling TCP from the server and 94 Mbyte/s pushing. (The
other system, 733 MHz PIII pushes 100MByte/s UDP with ttcp but the
client drops most of it).
you can speed up the client significantly by using the MSG_TRUNC option
('truncate message'). It will zap incoming data without copying it into
user-space. (you can use this for the 'bulk transfer' part - the initial
protocol handling code needs to see the actual data.) This way you should
be able to saturate the server even more.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Wed, 31 Jan 2001, Ingo Molnar wrote: > > On Tue, 30 Jan 2001, jamal wrote: > > > > - is this UDP or TCP based? (UDP i guess) > > > > > TCP > > well then i'd suggest to do: > > echo 10 10 10 > /proc/sys/net/ipv4/tcp_wmem > > does this make any difference? According to my notes, i dont see this. however, 262144 into /proc/sys/net/core/*mem_max/default. I have access to my h/ware this weekend. Hopefully i should get something better than ttcp to use. cheers, jamal - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Tue, 30 Jan 2001, jamal wrote: > > - is this UDP or TCP based? (UDP i guess) > > > TCP well then i'd suggest to do: echo 10 10 10 > /proc/sys/net/ipv4/tcp_wmem does this make any difference? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Wed, 31 Jan 2001, Ingo Molnar wrote: > > On Tue, 30 Jan 2001, jamal wrote: > > > Kernel | tput | sender-CPU | receiver-CPU | > > - > > 2.4.0-pre3 | 99MB/s | 87% | 23% | > > NSF||| | > > - > > 2.4.0-pre3 | 68 | 8% | 8% | > > +ZC SF| MB/s || | > > - > > isnt the CPU utilization difference amazing? :-) > With a caveat, sadly ;-> ttcp uses times() system call (or a diff of times() one at the beggining and another at the end). So the cpu measurements are not reflective. > a couple of questions: > > - is this UDP or TCP based? (UDP i guess) > TCP > - what wsize/rsize are you using? How do these requests look like on the > network, ie. are they suffieciently MTU-sized? yes. writes vary from 8K->64K but not much difference over the long period of time. > > - what happens if you run multiple instances of the testcode, does it > saturate bandwidth (or CPU)? This is something of great interest. I havent tried it. I should. I suspect this would be where the value of the ZC changes will become evident. cheers, jamal - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Tue, 30 Jan 2001, jamal wrote: > Kernel | tput | sender-CPU | receiver-CPU | > - > 2.4.0-pre3 | 99MB/s | 87% | 23% | > NSF||| | > - > 2.4.0-pre3 | 68 | 8% | 8% | > +ZC SF| MB/s || | > - isnt the CPU utilization difference amazing? :-) a couple of questions: - is this UDP or TCP based? (UDP i guess) - what wsize/rsize are you using? How do these requests look like on the network, ie. are they suffieciently MTU-sized? - what happens if you run multiple instances of the testcode, does it saturate bandwidth (or CPU)? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Tue, 30 Jan 2001, jamal wrote: Kernel | tput | sender-CPU | receiver-CPU | - 2.4.0-pre3 | 99MB/s | 87% | 23% | NSF||| | - 2.4.0-pre3 | 68 | 8% | 8% | +ZC SF| MB/s || | - isnt the CPU utilization difference amazing? :-) a couple of questions: - is this UDP or TCP based? (UDP i guess) - what wsize/rsize are you using? How do these requests look like on the network, ie. are they suffieciently MTU-sized? - what happens if you run multiple instances of the testcode, does it saturate bandwidth (or CPU)? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Wed, 31 Jan 2001, Ingo Molnar wrote: On Tue, 30 Jan 2001, jamal wrote: Kernel | tput | sender-CPU | receiver-CPU | - 2.4.0-pre3 | 99MB/s | 87% | 23% | NSF||| | - 2.4.0-pre3 | 68 | 8% | 8% | +ZC SF| MB/s || | - isnt the CPU utilization difference amazing? :-) With a caveat, sadly ;- ttcp uses times() system call (or a diff of times() one at the beggining and another at the end). So the cpu measurements are not reflective. a couple of questions: - is this UDP or TCP based? (UDP i guess) TCP - what wsize/rsize are you using? How do these requests look like on the network, ie. are they suffieciently MTU-sized? yes. writes vary from 8K-64K but not much difference over the long period of time. - what happens if you run multiple instances of the testcode, does it saturate bandwidth (or CPU)? This is something of great interest. I havent tried it. I should. I suspect this would be where the value of the ZC changes will become evident. cheers, jamal - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Still not sexy! (Re: sendfile+zerocopy: fairly sexy (nothing todo with ECN)
On Tue, 30 Jan 2001, jamal wrote: - is this UDP or TCP based? (UDP i guess) TCP well then i'd suggest to do: echo 10 10 10 /proc/sys/net/ipv4/tcp_wmem does this make any difference? Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
