Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

> On 03 May 2001 09:13:00 +0200, 
> [EMAIL PROTECTED] (Kai Henningsen) wrote:
> >[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in 
><[EMAIL PROTECTED]>:
> >
> >> PS: Hmm, how do you do timewarp for just one userland appliation with
> >> this installed?
> >
> >1. What on earth for?
> 
> Y10K testing :)
> 
> >2. How do you do it today, and why wouldn't that work?
> 
> LD_PRELOAD on a library that overrides gettimeofday().  I can see no
> reason why that would not continue to work.  What would stop working
> are timewarp modules that intercepted the syscall at the kernel level
> instead of user space level.

...and would break ptrace too.

-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

vsyscalls [was Re: X15 alpha release: as fast as TUX but in user space (fwd)]

[Pavel Machek]

Hi!

> > > PS: Hmm, how do you do timewarp for just one userland appliation with
> > > this installed?
> > 
> > 1. What on earth for?
> 
> Y2K testing was one previous example.
> 
> > 2. How do you do it today, and why wouldn't that work?
> 
> LD_PRELOAD and providing its still using a lib call it would. I dont see the
> original posters problem

LD_PRELOAD is not reliable: application may do syscall itself and find
out true time. But ptrace works currently and *is* reliable.

Problem is that vsyscalls ay take ability to use ptrace to fool apps away.
Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

vsyscalls [was Re: X15 alpha release: as fast as TUX but in user space (fwd)]

[Pavel Machek]

Hi!

   PS: Hmm, how do you do timewarp for just one userland appliation with
   this installed?
  
  1. What on earth for?
 
 Y2K testing was one previous example.
 
  2. How do you do it today, and why wouldn't that work?
 
 LD_PRELOAD and providing its still using a lib call it would. I dont see the
 original posters problem

LD_PRELOAD is not reliable: application may do syscall itself and find
out true time. But ptrace works currently and *is* reliable.

Problem is that vsyscalls ay take ability to use ptrace to fool apps away.
Pavel
-- 
Philips Velo 1: 1x4x8, 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

 On 03 May 2001 09:13:00 +0200, 
 [EMAIL PROTECTED] (Kai Henningsen) wrote:
 [EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in 
[EMAIL PROTECTED]:
 
  PS: Hmm, how do you do timewarp for just one userland appliation with
  this installed?
 
 1. What on earth for?
 
 Y10K testing :)
 
 2. How do you do it today, and why wouldn't that work?
 
 LD_PRELOAD on a library that overrides gettimeofday().  I can see no
 reason why that would not continue to work.  What would stop working
 are timewarp modules that intercepted the syscall at the kernel level
 instead of user space level.

...and would break ptrace too.

-- 
Philips Velo 1: 1x4x8, 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Hello,

I have uploaded a new release of X15 that hopefully solves all the RFC bugs.
I say hopefully because I haven't had the opportunity to fully test the
request pipelining. Is there anything to automatize such tests?

>From what I could measure X15 is still a good 5% faster than TUX.

You can find the file at: http://www.chromium.com/X15-Alpha-4.tgz

BTW: Next release (in a week or so) will be a beta and it will include source
code!

BTW2: I'll be away from my email in the next few days, so don't be amazed if
I'll be kind of slow replying to messages...

 - Fabio

Ingo Molnar wrote:

> yet another anomaly i noticed. X15 does not appear to handle pipelined
> HTTP/1.1 requests properly, it ignores the second request if two requests
> arrive in the same packet.
>
> SPECweb99 does not send pipelined requests, but a number of RL web clients
> do. (Mozilla, apt-get, etc.)
>
> Ingo
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Hello,

I have uploaded a new release of X15 that hopefully solves all the RFC bugs.
I say hopefully because I haven't had the opportunity to fully test the
request pipelining. Is there anything to automatize such tests?

From what I could measure X15 is still a good 5% faster than TUX.

You can find the file at: http://www.chromium.com/X15-Alpha-4.tgz

BTW: Next release (in a week or so) will be a beta and it will include source
code!

BTW2: I'll be away from my email in the next few days, so don't be amazed if
I'll be kind of slow replying to messages...

 - Fabio

Ingo Molnar wrote:

 yet another anomaly i noticed. X15 does not appear to handle pipelined
 HTTP/1.1 requests properly, it ignores the second request if two requests
 arrive in the same packet.

 SPECweb99 does not send pipelined requests, but a number of RL web clients
 do. (Mozilla, apt-get, etc.)

 Ingo

 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Davide Libenzi]

On 04-May-2001 Fabio Riccardi wrote:
> ok, I'm totally ignorant here, what is a pipelined request?



http://www.w3.org/Protocols/HTTP/Performance/Pipeline.html


A pipelined application implementation buffers its output before writing it to
the underlying TCP stack, roughly equivalent to what the Nagle algorithm does
for telnet connections.
These two buffering algorithms tend to interfere, and using them together will
often cause very significant performance degradation. For each connection, the
server maintains a
response buffer that it flushes either when full, or when there is no more
requests coming in on that connection, or before it goes idle. This buffering
enables aggregating responses
(for example, cache validation responses) into fewer packets even on a
high-speed network, and saving CPU time for the server. 






- Davide

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

ok, I'm totally ignorant here, what is a pipelined request?

btw: please be kind with my mistakes, X15 _is_ alpha code anyway... :)

 - Fabio

Ingo Molnar wrote:

> yet another anomaly i noticed. X15 does not appear to handle pipelined
> HTTP/1.1 requests properly, it ignores the second request if two requests
> arrive in the same packet.
>
> SPECweb99 does not send pipelined requests, but a number of RL web clients
> do. (Mozilla, apt-get, etc.)
>
> Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[dean gaudet]

um, presumably this new magic page won't eliminate the old syscall entry
points.  so just use those for UML.

-dean

On Fri, 4 May 2001, Pavel Machek wrote:

> Hi!
>
> > > > That means that for fooling closed-source statically-linked binary,
> > >
> > > If they are using glibc then you have the right to the object to link
> > > with the library and the library source under the LGPL. I dont know of any
> > > app using its own C lib
> >
> > Some don't use any libc at all, some just don't use it for the time call
> > that were talking about substituting.
> >
> > Lying about the time is a hack, pure and simple. It will still be possible
> > with magic pages. The fact that it will require more kernel hacking to
> > accomplish it is irrelevant.
>
> No. You are breaking self-virtualization here. That is not irrelevant.
>
> It used to require no kernel support before. Now it will require
> kernel support. That's step back. (Think uml).
>
>   Pavel
> --
> I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care."
> Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[bert hubert]

On Thu, May 03, 2001 at 09:19:15PM +0100, Alan Cox wrote:

> If they are using glibc then you have the right to the object to link
> with the library and the library source under the LGPL. I dont know of any
> app using its own C lib

qmail is nearly there.

-- 
http://www.PowerDNS.com  Versatile DNS Services  
Trilab   The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

> > > That means that for fooling closed-source statically-linked binary,
> > 
> > If they are using glibc then you have the right to the object to link
> > with the library and the library source under the LGPL. I dont know of any
> > app using its own C lib
> 
> Some don't use any libc at all, some just don't use it for the time call
> that were talking about substituting.
> 
> Lying about the time is a hack, pure and simple. It will still be possible
> with magic pages. The fact that it will require more kernel hacking to
> accomplish it is irrelevant.

No. You are breaking self-virtualization here. That is not irrelevant.

It used to require no kernel support before. Now it will require
kernel support. That's step back. (Think uml).

Pavel
-- 
I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

   That means that for fooling closed-source statically-linked binary,
  
  If they are using glibc then you have the right to the object to link
  with the library and the library source under the LGPL. I dont know of any
  app using its own C lib
 
 Some don't use any libc at all, some just don't use it for the time call
 that were talking about substituting.
 
 Lying about the time is a hack, pure and simple. It will still be possible
 with magic pages. The fact that it will require more kernel hacking to
 accomplish it is irrelevant.

No. You are breaking self-virtualization here. That is not irrelevant.

It used to require no kernel support before. Now it will require
kernel support. That's step back. (Think uml).

Pavel
-- 
I'm [EMAIL PROTECTED] In my country we have almost anarchy and I don't care.
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[bert hubert]

On Thu, May 03, 2001 at 09:19:15PM +0100, Alan Cox wrote:

 If they are using glibc then you have the right to the object to link
 with the library and the library source under the LGPL. I dont know of any
 app using its own C lib

qmail is nearly there.

-- 
http://www.PowerDNS.com  Versatile DNS Services  
Trilab   The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[dean gaudet]

um, presumably this new magic page won't eliminate the old syscall entry
points.  so just use those for UML.

-dean

On Fri, 4 May 2001, Pavel Machek wrote:

 Hi!

That means that for fooling closed-source statically-linked binary,
  
   If they are using glibc then you have the right to the object to link
   with the library and the library source under the LGPL. I dont know of any
   app using its own C lib
 
  Some don't use any libc at all, some just don't use it for the time call
  that were talking about substituting.
 
  Lying about the time is a hack, pure and simple. It will still be possible
  with magic pages. The fact that it will require more kernel hacking to
  accomplish it is irrelevant.

 No. You are breaking self-virtualization here. That is not irrelevant.

 It used to require no kernel support before. Now it will require
 kernel support. That's step back. (Think uml).

   Pavel
 --
 I'm [EMAIL PROTECTED] In my country we have almost anarchy and I don't care.
 Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/


-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

ok, I'm totally ignorant here, what is a pipelined request?

btw: please be kind with my mistakes, X15 _is_ alpha code anyway... :)

 - Fabio

Ingo Molnar wrote:

 yet another anomaly i noticed. X15 does not appear to handle pipelined
 HTTP/1.1 requests properly, it ignores the second request if two requests
 arrive in the same packet.

 SPECweb99 does not send pipelined requests, but a number of RL web clients
 do. (Mozilla, apt-get, etc.)

 Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Davide Libenzi]

On 04-May-2001 Fabio Riccardi wrote:
 ok, I'm totally ignorant here, what is a pipelined request?



http://www.w3.org/Protocols/HTTP/Performance/Pipeline.html

QUOTE
A pipelined application implementation buffers its output before writing it to
the underlying TCP stack, roughly equivalent to what the Nagle algorithm does
for telnet connections.
These two buffering algorithms tend to interfere, and using them together will
often cause very significant performance degradation. For each connection, the
server maintains a
response buffer that it flushes either when full, or when there is no more
requests coming in on that connection, or before it goes idle. This buffering
enables aggregating responses
(for example, cache validation responses) into fewer packets even on a
high-speed network, and saving CPU time for the server. 
/QUOTE





- Davide

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

I have fixed the stale header cache problem. Files are statted on every
request, no "practical" tricks.

Performance doesn't seem to have suffered :)

I also have added a cache garbage collector to close "old" file descriptors
and remove even older header cache entries. This should make sure that you
don't exceed your system resources. The definition of old and the sweep
frequency are user configurable.

You can download the new version
from: http://www.chromium.com/X15-Alpha-3.tgz

 - Fabio

Ingo Molnar wrote:

> On Tue, 1 May 2001, Fabio Riccardi wrote:
>
> > This is actually a bug in the current X15, I know how to fix it (with
> > negligible overhead) but I've been lazy :)
>
> yep, i think it's pretty straightforward: you have a cache of open file
> descriptors (like Squid i think), and you can start a background 'cache
> synchronization thread' that does a stat() on every open file's real VFS
> path, every second or so. This should have small overhead (the number of
> file descriptors cached should be limited anyway via some sort of LRU),
> and guarantees 'practical coherency', without having the overhead of
> immediate coherency. [total coherency is pointless anyway, not even the
> kernel guarantees it to all parallel VFS users.]
>
> Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Gregory Maxwell]

On Thu, May 03, 2001 at 09:19:15PM +0100, Alan Cox wrote:
> > That means that for fooling closed-source statically-linked binary,
> 
> If they are using glibc then you have the right to the object to link
> with the library and the library source under the LGPL. I dont know of any
> app using its own C lib

Some don't use any libc at all, some just don't use it for the time call
that were talking about substituting.

Lying about the time is a hack, pure and simple. It will still be possible
with magic pages. The fact that it will require more kernel hacking to
accomplish it is irrelevant.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

> That means that for fooling closed-source statically-linked binary,

If they are using glibc then you have the right to the object to link
with the library and the library source under the LGPL. I dont know of any
app using its own C lib

> 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[agrawal]

On Thu, 3 May 2001, Pavel Machek wrote:

> That means that for fooling closed-source statically-linked binary,
> you now need to patch kernel. That's regression; subterfugue.org could
> do this with normal user rights in 2.4.0.

This is particularly pretty, but something that might work:

1. a "deceiver" process creates a shared memory page, populates shared
   page with appropriate magic (perhaps copying from its own magic page?)
2. have subterfuge unmap the magic page for the fooled process, and map in
   the shared page in its place (assumes subterfuge can insert system
   calls, instead of just modifying them)
3. deceiver periodically updates magic page



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

> > > > Whatever happened to that hack that was discussed a year or two ago?
> > > > The one where (also on IA32) a magic page was set up by the kernel
> > > > containing code for fast system calls, and the kernel would write
> > > > calibation information to that magic page. The code written there
> > > > would use the TSC in conjunction with that calibration data.
> 
> > Pavel
> > PS: Hmm, how do you do timewarp for just one userland appliation with
> > this installed?
> 
> 1. Kernel solution: give that particular process a different magic page
> 2. User solution:   Don't obtain time from the magic page.
> 2a  By changing program source, if available
> 2b  By switching the c library, assuming it is used

That means that for fooling closed-source statically-linked binary,
you now need to patch kernel. That's regression; subterfugue.org could
do this with normal user rights in 2.4.0.
-- 
I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Gregory Maxwell]

On Thu, May 03, 2001 at 05:44:36PM +1000, Keith Owens wrote:
> On 03 May 2001 09:13:00 +0200, 
> [EMAIL PROTECTED] (Kai Henningsen) wrote:
> >[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in 
><[EMAIL PROTECTED]>:
> >
> >> PS: Hmm, how do you do timewarp for just one userland appliation with
> >> this installed?
> >
> >1. What on earth for?
> 
> Y10K testing :)
> 
> >2. How do you do it today, and why wouldn't that work?
> 
> LD_PRELOAD on a library that overrides gettimeofday().  I can see no
> reason why that would not continue to work.  What would stop working
> are timewarp modules that intercepted the syscall at the kernel level
> instead of user space level.

It would just have to be done a differnt way.. For those applications you
make a differnt magic page and redirect them there.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Helge Hafting]

Pavel Machek wrote:
> > >
> > > Whatever happened to that hack that was discussed a year or two ago?
> > > The one where (also on IA32) a magic page was set up by the kernel
> > > containing code for fast system calls, and the kernel would write
> > > calibation information to that magic page. The code written there
> > > would use the TSC in conjunction with that calibration data.

> Pavel
> PS: Hmm, how do you do timewarp for just one userland appliation with
> this installed?

1. Kernel solution: give that particular process a different magic page
2. User solution:   Don't obtain time from the magic page.
2a  By changing program source, if available
2b  By switching the c library, assuming it is used

Helge Hafting
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Ingo Oeser]

On Thu, May 03, 2001 at 05:44:36PM +1000, Keith Owens wrote:
> >2. How do you do it today, and why wouldn't that work?
> 
> LD_PRELOAD on a library that overrides gettimeofday().  I can see no
> reason why that would not continue to work. 

Static linkage?

> What would stop working
> are timewarp modules that intercepted the syscall at the kernel level
> instead of user space level.

That's what the poster talked about ;-)

Think subterfuge (sp?) and friends.

Regards

Ingo Oeser
-- 
10.+11.03.2001 - 3. Chemnitzer LinuxTag 
  been there and had much fun   
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

> > PS: Hmm, how do you do timewarp for just one userland appliation with
> > this installed?
> 
> 1. What on earth for?

Y2K testing was one previous example.

> 2. How do you do it today, and why wouldn't that work?

LD_PRELOAD and providing its still using a lib call it would. I dont see the
original posters problem


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Keith Owens]

On 03 May 2001 09:13:00 +0200, 
[EMAIL PROTECTED] (Kai Henningsen) wrote:
>[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in <[EMAIL PROTECTED]>:
>
>> PS: Hmm, how do you do timewarp for just one userland appliation with
>> this installed?
>
>1. What on earth for?

Y10K testing :)

>2. How do you do it today, and why wouldn't that work?

LD_PRELOAD on a library that overrides gettimeofday().  I can see no
reason why that would not continue to work.  What would stop working
are timewarp modules that intercepted the syscall at the kernel level
instead of user space level.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Kai Henningsen]

[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in <[EMAIL PROTECTED]>:

> PS: Hmm, how do you do timewarp for just one userland appliation with
> this installed?

1. What on earth for?

2. How do you do it today, and why wouldn't that work?


MfG Kai
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Kai Henningsen]

[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in [EMAIL PROTECTED]:

 PS: Hmm, how do you do timewarp for just one userland appliation with
 this installed?

1. What on earth for?

2. How do you do it today, and why wouldn't that work?


MfG Kai
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Keith Owens]

On 03 May 2001 09:13:00 +0200, 
[EMAIL PROTECTED] (Kai Henningsen) wrote:
[EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in [EMAIL PROTECTED]:

 PS: Hmm, how do you do timewarp for just one userland appliation with
 this installed?

1. What on earth for?

Y10K testing :)

2. How do you do it today, and why wouldn't that work?

LD_PRELOAD on a library that overrides gettimeofday().  I can see no
reason why that would not continue to work.  What would stop working
are timewarp modules that intercepted the syscall at the kernel level
instead of user space level.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

  PS: Hmm, how do you do timewarp for just one userland appliation with
  this installed?
 
 1. What on earth for?

Y2K testing was one previous example.

 2. How do you do it today, and why wouldn't that work?

LD_PRELOAD and providing its still using a lib call it would. I dont see the
original posters problem


-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Ingo Oeser]

On Thu, May 03, 2001 at 05:44:36PM +1000, Keith Owens wrote:
 2. How do you do it today, and why wouldn't that work?
 
 LD_PRELOAD on a library that overrides gettimeofday().  I can see no
 reason why that would not continue to work. 

Static linkage?

 What would stop working
 are timewarp modules that intercepted the syscall at the kernel level
 instead of user space level.

That's what the poster talked about ;-)

Think subterfuge (sp?) and friends.

Regards

Ingo Oeser
-- 
10.+11.03.2001 - 3. Chemnitzer LinuxTag http://www.tu-chemnitz.de/linux/tag
  been there and had much fun   
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Helge Hafting]

Pavel Machek wrote:
  
   Whatever happened to that hack that was discussed a year or two ago?
   The one where (also on IA32) a magic page was set up by the kernel
   containing code for fast system calls, and the kernel would write
   calibation information to that magic page. The code written there
   would use the TSC in conjunction with that calibration data.

 Pavel
 PS: Hmm, how do you do timewarp for just one userland appliation with
 this installed?

1. Kernel solution: give that particular process a different magic page
2. User solution:   Don't obtain time from the magic page.
2a  By changing program source, if available
2b  By switching the c library, assuming it is used

Helge Hafting
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Gregory Maxwell]

On Thu, May 03, 2001 at 05:44:36PM +1000, Keith Owens wrote:
 On 03 May 2001 09:13:00 +0200, 
 [EMAIL PROTECTED] (Kai Henningsen) wrote:
 [EMAIL PROTECTED] (Pavel Machek)  wrote on 30.04.01 in 
[EMAIL PROTECTED]:
 
  PS: Hmm, how do you do timewarp for just one userland appliation with
  this installed?
 
 1. What on earth for?
 
 Y10K testing :)
 
 2. How do you do it today, and why wouldn't that work?
 
 LD_PRELOAD on a library that overrides gettimeofday().  I can see no
 reason why that would not continue to work.  What would stop working
 are timewarp modules that intercepted the syscall at the kernel level
 instead of user space level.

It would just have to be done a differnt way.. For those applications you
make a differnt magic page and redirect them there.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

Whatever happened to that hack that was discussed a year or two ago?
The one where (also on IA32) a magic page was set up by the kernel
containing code for fast system calls, and the kernel would write
calibation information to that magic page. The code written there
would use the TSC in conjunction with that calibration data.
 
  Pavel
  PS: Hmm, how do you do timewarp for just one userland appliation with
  this installed?
 
 1. Kernel solution: give that particular process a different magic page
 2. User solution:   Don't obtain time from the magic page.
 2a  By changing program source, if available
 2b  By switching the c library, assuming it is used

That means that for fooling closed-source statically-linked binary,
you now need to patch kernel. That's regression; subterfugue.org could
do this with normal user rights in 2.4.0.
-- 
I'm [EMAIL PROTECTED] In my country we have almost anarchy and I don't care.
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[agrawal]

On Thu, 3 May 2001, Pavel Machek wrote:

 That means that for fooling closed-source statically-linked binary,
 you now need to patch kernel. That's regression; subterfugue.org could
 do this with normal user rights in 2.4.0.

This is particularly pretty, but something that might work:

1. a deceiver process creates a shared memory page, populates shared
   page with appropriate magic (perhaps copying from its own magic page?)
2. have subterfuge unmap the magic page for the fooled process, and map in
   the shared page in its place (assumes subterfuge can insert system
   calls, instead of just modifying them)
3. deceiver periodically updates magic page



-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

 That means that for fooling closed-source statically-linked binary,

If they are using glibc then you have the right to the object to link
with the library and the library source under the LGPL. I dont know of any
app using its own C lib

 

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Gregory Maxwell]

On Thu, May 03, 2001 at 09:19:15PM +0100, Alan Cox wrote:
  That means that for fooling closed-source statically-linked binary,
 
 If they are using glibc then you have the right to the object to link
 with the library and the library source under the LGPL. I dont know of any
 app using its own C lib

Some don't use any libc at all, some just don't use it for the time call
that were talking about substituting.

Lying about the time is a hack, pure and simple. It will still be possible
with magic pages. The fact that it will require more kernel hacking to
accomplish it is irrelevant.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

I have fixed the stale header cache problem. Files are statted on every
request, no practical tricks.

Performance doesn't seem to have suffered :)

I also have added a cache garbage collector to close old file descriptors
and remove even older header cache entries. This should make sure that you
don't exceed your system resources. The definition of old and the sweep
frequency are user configurable.

You can download the new version
from: http://www.chromium.com/X15-Alpha-3.tgz

 - Fabio

Ingo Molnar wrote:

 On Tue, 1 May 2001, Fabio Riccardi wrote:

  This is actually a bug in the current X15, I know how to fix it (with
  negligible overhead) but I've been lazy :)

 yep, i think it's pretty straightforward: you have a cache of open file
 descriptors (like Squid i think), and you can start a background 'cache
 synchronization thread' that does a stat() on every open file's real VFS
 path, every second or so. This should have small overhead (the number of
 file descriptors cached should be limited anyway via some sort of LRU),
 and guarantees 'practical coherency', without having the overhead of
 immediate coherency. [total coherency is pointless anyway, not even the
 kernel guarantees it to all parallel VFS users.]

 Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Our intention is to release X15 with an open source license.

This will happen as soon as the codebase stabilizes a bit, that is when we go
beta (in two - three weeks).

At the moment we just don't have the time...

The reason why I released the alpha binary version is that several people
would not believe that a user-space server with this level of performance
would be possible at all and several statements that I made on this list were
challenged.

Besides I really appreciate the feedback that I received so far from Ingo and
others, and I'd be very curious to know if anybody did any performance
evaluation at all.

 - Fabio

Zach Brown wrote:

> I've always been tempted to go back and take a real swing at a
> nice content server, but there's only so many hours in the day, and
> apache+thttpd+tux complete the problem space.  If X15 isn't released
> with an open license, I may be tempted yet again :)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Lincoln Dale]

Hi,

At 10:50 AM 2/05/2001 +0200, Ingo Molnar wrote:
>i think Zach's phhttpd is an important milestone as well, it's the first
>userspace webserver that shows how to use event-based, sigio-based async
>networking IO and sendfile() under Linux. (I believe it had some
>performance problems related to sigio queue overflow, these issues might
>be solved in the latest kernels.) The zerocopy enhancements should help
>phhttpd as well.

my experience with sigio-based event-handlers is that the net-gain of 
event-driven i/o is mitigated by the fact that SIGIO is based on signals.

the problem with signals for this purpose are:
  (a) you go thru a syncronization point in the kernel.  signals are protected
  by a spinlock.
  it doesn't scale with SMP.
  (b) SI_PAD_SIZE

explicitly, (b) means that you have an awful lot of memory-accesses going 
on for every signal.
my experience with the overhead is that it mitigates the advantages when 
you become bottlenecked on memory-bus-accesses.


cheers,

lincoln.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Fabio Riccardi]

>From my experience system calls are not an issue.

What costs a lot is moving data around, since modern CPUs spend most of their
time in memory/bus wait cycles...

 - Fabio

Linus Torvalds wrote:

> >I think that applies to all really high-performance servers.
>
> Note that it is definitely not always true.
>
> Linux system calls are reasonably light-weight.  And sometimes trying to
> avoid them ends up beaing _more_ work - because you might have to worry
> about synchronization and cache coherency in user mode.
>
> So the rule should be "avoid _unnecessary_ system calls".
>
> Linus
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Linus Torvalds]

In article <[EMAIL PROTECTED]>,
Matti Aarnio  <[EMAIL PROTECTED]> wrote:
>On Sun, Apr 29, 2001 at 02:16:43PM -0700, Jim Gettys wrote:
>...
>> "X is an exercise in avoiding system calls".  I think I said this around
>> 1984-1985.  
>>  - Jim
>
>I think that applies to all really high-performance servers.

Note that it is definitely not always true.

Linux system calls are reasonably light-weight.  And sometimes trying to
avoid them ends up beaing _more_ work - because you might have to worry
about synchronization and cache coherency in user mode. 

So the rule should be "avoid _unnecessary_ system calls".

Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Matti Aarnio]

On Sun, Apr 29, 2001 at 02:16:43PM -0700, Jim Gettys wrote:
...
> "X is an exercise in avoiding system calls".  I think I said this around
> 1984-1985.  
>   - Jim

I think that applies to all really high-performance servers.

Definitely it applies to ZMailer, which before did do time(2) some
1000-2 times per second during some high activity bursts (limited
essentially by the syscall speed).  These days there is shared memory
segment into which a server process updates the time value some 2-5
times per sec, and the consumer reads that -- likely now the consumer
bursts peak beyond 100 000 reads.

I think I took the idea from Interactive IX/386, which had magic
global segment mapped to all userspaces for few fast common tasks,
including gettimeofday() data.  That was around 1990, or a bit before
(I switched to Linux soon after.)Where they got the idea from,
that I haven't checked.

The basic algorithms and ideas we employ to do these wonders are
also described by Knuth in his "The Art of Computer Programming"
series.  And usually he is referring to some earlier publications.

> --
> Jim Gettys
> Technology and Corporate Development
> Compaq Computer Corporation
> [EMAIL PROTECTED]

/Matti Aarnio  <[EMAIL PROTECTED]>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Zach Brown]

> i think Zach's phhttpd is an important milestone as well, it's the first
> userspace webserver that shows how to use event-based, sigio-based async
> networking IO and sendfile() under Linux. (I believe it had some

*blush*

> performance problems related to sigio queue overflow, these issues might
> be solved in the latest kernels.) The zerocopy enhancements should help
> phhttpd as well.

oh, it has a bunch of problems :)  over-threading created complexity in
the fast path.  It always spends memory on a contiguous header region for
each connection, which may not be valid in the days of zero copy sendmsg.
It does IO in the fast path.  And looking back at it, I'm struck by how
naive most of the code is :) :)

I've always been tempted to go back and take a real swing at a
nice content server, but there's only so many hours in the day, and
apache+thttpd+tux complete the problem space.  If X15 isn't released
with an open license, I may be tempted yet again :)

- z
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Ingo Molnar]

On Wed, 2 May 2001, Andi Kleen wrote:

> > Whatever happened to that hack that was discussed a year or two ago?
> > The one where (also on IA32) a magic page was set up by the kernel
> > containing code for fast system calls, and the kernel would write
> > calibation information to that magic page. The code written there
> > would use the TSC in conjunction with that calibration data.
> >
> > There was much discussion about this idea, even Linus was keen on
> > it. But IIRC, nothing ever happened.
>
> It's already implemented in the x86-64 port, thanks to Andrea
> Arcangelli.

well, it was first prototyped in the vsyscall patches :-)

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Andi Kleen]

[sorry for the late answer -- i was involuntarily offline for a few days]

On Sat, Apr 28, 2001 at 04:56:27PM -0600, Richard Gooch wrote:
> Whatever happened to that hack that was discussed a year or two ago?
> The one where (also on IA32) a magic page was set up by the kernel
> containing code for fast system calls, and the kernel would write
> calibation information to that magic page. The code written there
> would use the TSC in conjunction with that calibration data.
> 
> There was much discussion about this idea, even Linus was keen on
> it. But IIRC, nothing ever happened.

It's already implemented in the x86-64 port, thanks to Andrea
Arcangelli.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Sun, 29 Apr 2001, Fabio Riccardi wrote:

> TUX has definitively been my performance yardstick for the development
> of X15, but I had many sources of inspiration for the X15
> architecture. Maybe the most relevant are the Flash Web Server (Pai,
> Druschel, Zwaenepoel), several Linus observations on this list about
> (web) server architecture and kernnel services, and the reading of the
> Hennessy & Patterson architecture books. [...]

i think Zach's phhttpd is an important milestone as well, it's the first
userspace webserver that shows how to use event-based, sigio-based async
networking IO and sendfile() under Linux. (I believe it had some
performance problems related to sigio queue overflow, these issues might
be solved in the latest kernels.) The zerocopy enhancements should help
phhttpd as well.

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Sun, 29 Apr 2001, Fabio Riccardi wrote:

 TUX has definitively been my performance yardstick for the development
 of X15, but I had many sources of inspiration for the X15
 architecture. Maybe the most relevant are the Flash Web Server (Pai,
 Druschel, Zwaenepoel), several Linus observations on this list about
 (web) server architecture and kernnel services, and the reading of the
 Hennessy  Patterson architecture books. [...]

i think Zach's phhttpd is an important milestone as well, it's the first
userspace webserver that shows how to use event-based, sigio-based async
networking IO and sendfile() under Linux. (I believe it had some
performance problems related to sigio queue overflow, these issues might
be solved in the latest kernels.) The zerocopy enhancements should help
phhttpd as well.

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Andi Kleen]

[sorry for the late answer -- i was involuntarily offline for a few days]

On Sat, Apr 28, 2001 at 04:56:27PM -0600, Richard Gooch wrote:
 Whatever happened to that hack that was discussed a year or two ago?
 The one where (also on IA32) a magic page was set up by the kernel
 containing code for fast system calls, and the kernel would write
 calibation information to that magic page. The code written there
 would use the TSC in conjunction with that calibration data.
 
 There was much discussion about this idea, even Linus was keen on
 it. But IIRC, nothing ever happened.

It's already implemented in the x86-64 port, thanks to Andrea
Arcangelli.

-Andi
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Ingo Molnar]

On Wed, 2 May 2001, Andi Kleen wrote:

  Whatever happened to that hack that was discussed a year or two ago?
  The one where (also on IA32) a magic page was set up by the kernel
  containing code for fast system calls, and the kernel would write
  calibation information to that magic page. The code written there
  would use the TSC in conjunction with that calibration data.
 
  There was much discussion about this idea, even Linus was keen on
  it. But IIRC, nothing ever happened.

 It's already implemented in the x86-64 port, thanks to Andrea
 Arcangelli.

well, it was first prototyped in the vsyscall patches :-)

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Zach Brown]

 i think Zach's phhttpd is an important milestone as well, it's the first
 userspace webserver that shows how to use event-based, sigio-based async
 networking IO and sendfile() under Linux. (I believe it had some

*blush*

 performance problems related to sigio queue overflow, these issues might
 be solved in the latest kernels.) The zerocopy enhancements should help
 phhttpd as well.

oh, it has a bunch of problems :)  over-threading created complexity in
the fast path.  It always spends memory on a contiguous header region for
each connection, which may not be valid in the days of zero copy sendmsg.
It does IO in the fast path.  And looking back at it, I'm struck by how
naive most of the code is :) :)

I've always been tempted to go back and take a real swing at a
nice content server, but there's only so many hours in the day, and
apache+thttpd+tux complete the problem space.  If X15 isn't released
with an open license, I may be tempted yet again :)

- z
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Matti Aarnio]

On Sun, Apr 29, 2001 at 02:16:43PM -0700, Jim Gettys wrote:
...
 X is an exercise in avoiding system calls.  I think I said this around
 1984-1985.  
   - Jim

I think that applies to all really high-performance servers.

Definitely it applies to ZMailer, which before did do time(2) some
1000-2 times per second during some high activity bursts (limited
essentially by the syscall speed).  These days there is shared memory
segment into which a server process updates the time value some 2-5
times per sec, and the consumer reads that -- likely now the consumer
bursts peak beyond 100 000 reads.

I think I took the idea from Interactive IX/386, which had magic
global segment mapped to all userspaces for few fast common tasks,
including gettimeofday() data.  That was around 1990, or a bit before
(I switched to Linux soon after.)Where they got the idea from,
that I haven't checked.

The basic algorithms and ideas we employ to do these wonders are
also described by Knuth in his The Art of Computer Programming
series.  And usually he is referring to some earlier publications.

 --
 Jim Gettys
 Technology and Corporate Development
 Compaq Computer Corporation
 [EMAIL PROTECTED]

/Matti Aarnio  [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Linus Torvalds]

In article [EMAIL PROTECTED],
Matti Aarnio  [EMAIL PROTECTED] wrote:
On Sun, Apr 29, 2001 at 02:16:43PM -0700, Jim Gettys wrote:
...
 X is an exercise in avoiding system calls.  I think I said this around
 1984-1985.  
  - Jim

I think that applies to all really high-performance servers.

Note that it is definitely not always true.

Linux system calls are reasonably light-weight.  And sometimes trying to
avoid them ends up beaing _more_ work - because you might have to worry
about synchronization and cache coherency in user mode. 

So the rule should be avoid _unnecessary_ system calls.

Linus
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Fabio Riccardi]

From my experience system calls are not an issue.

What costs a lot is moving data around, since modern CPUs spend most of their
time in memory/bus wait cycles...

 - Fabio

Linus Torvalds wrote:

 I think that applies to all really high-performance servers.

 Note that it is definitely not always true.

 Linux system calls are reasonably light-weight.  And sometimes trying to
 avoid them ends up beaing _more_ work - because you might have to worry
 about synchronization and cache coherency in user mode.

 So the rule should be avoid _unnecessary_ system calls.

 Linus
 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Lincoln Dale]

Hi,

At 10:50 AM 2/05/2001 +0200, Ingo Molnar wrote:
i think Zach's phhttpd is an important milestone as well, it's the first
userspace webserver that shows how to use event-based, sigio-based async
networking IO and sendfile() under Linux. (I believe it had some
performance problems related to sigio queue overflow, these issues might
be solved in the latest kernels.) The zerocopy enhancements should help
phhttpd as well.

my experience with sigio-based event-handlers is that the net-gain of 
event-driven i/o is mitigated by the fact that SIGIO is based on signals.

the problem with signals for this purpose are:
  (a) you go thru a syncronization point in the kernel.  signals are protected
  by a spinlock.
  it doesn't scale with SMP.
  (b) SI_PAD_SIZE

explicitly, (b) means that you have an awful lot of memory-accesses going 
on for every signal.
my experience with the overhead is that it mitigates the advantages when 
you become bottlenecked on memory-bus-accesses.


cheers,

lincoln.

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Our intention is to release X15 with an open source license.

This will happen as soon as the codebase stabilizes a bit, that is when we go
beta (in two - three weeks).

At the moment we just don't have the time...

The reason why I released the alpha binary version is that several people
would not believe that a user-space server with this level of performance
would be possible at all and several statements that I made on this list were
challenged.

Besides I really appreciate the feedback that I received so far from Ingo and
others, and I'd be very curious to know if anybody did any performance
evaluation at all.

 - Fabio

Zach Brown wrote:

 I've always been tempted to go back and take a real swing at a
 nice content server, but there's only so many hours in the day, and
 apache+thttpd+tux complete the problem space.  If X15 isn't released
 with an open license, I may be tempted yet again :)

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Tue, 1 May 2001, Fabio Riccardi wrote:

> This is actually a bug in the current X15, I know how to fix it (with
> negligible overhead) but I've been lazy :)

yep, i think it's pretty straightforward: you have a cache of open file
descriptors (like Squid i think), and you can start a background 'cache
synchronization thread' that does a stat() on every open file's real VFS
path, every second or so. This should have small overhead (the number of
file descriptors cached should be limited anyway via some sort of LRU),
and guarantees 'practical coherency', without having the overhead of
immediate coherency. [total coherency is pointless anyway, not even the
kernel guarantees it to all parallel VFS users.]

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

This is actually a bug in the current X15, I know how to fix it (with
negligible overhead) but I've been lazy :)

give me a few days...

 - Fabio

Ingo Molnar wrote:

> hm, another X15 caching artifact i noticed: i've changed the index.html
> file, and while X15 was still serving the old copy, TUX served the new
> file immediately.
>
> its cache is apparently not coherent with actual VFS contents. (ie. it's a
> read-once cache apparently?) TUX has some (occasionally significant)
> overhead due to non-cached VFS-lookups.
>
> Ingo
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

> > > In x86-64 there are special vsyscalls btw to solve this problem that export
> > > a lockless kernel gettimeofday()
> > 
> > Whatever happened to that hack that was discussed a year or two ago?
> > The one where (also on IA32) a magic page was set up by the kernel
> > containing code for fast system calls, and the kernel would write
> > calibation information to that magic page. The code written there
> > would use the TSC in conjunction with that calibration data.
> > 
> > There was much discussion about this idea, even Linus was keen on
> > it. But IIRC, nothing ever happened.
> > 
> 
> We discussed this at the Summit, not a year or two ago.  x86-64 has
> it, and it wouldn't be too bad to do in i386... just noone did.

Just wait what kind of problems it is able to bring on i386.

Pavel
PS: Hmm, how do you do timewarp for just one userland appliation with
this installed?

-- 
I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

hm, another X15 caching artifact i noticed: i've changed the index.html
file, and while X15 was still serving the old copy, TUX served the new
file immediately.

its cache is apparently not coherent with actual VFS contents. (ie. it's a
read-once cache apparently?) TUX has some (occasionally significant)
overhead due to non-cached VFS-lookups.

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Mon, 30 Apr 2001, Fabio Riccardi wrote:

> Ok I fixed it, the header date timestamp is updated with every
> request.
>
> Performance doesn't seem to have suffered significantly (less than
> 1%).

yep, expected that - doing a sendmsg()+sendfile() generates the same
packet structure, the only difference being that ~100-200 bytes are copied
between kernel-space and user-space.

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Sun, 29 Apr 2001, Fabio Riccardi wrote:

> I can disable header caching and see what happens, I'll add an option
> for this in the next X15 release.

what SPECweb99 performance do you get if you disable header caching? It
might not make all that much of a difference. (but it will make the
results more comparable with TUX.)

Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Sun, 29 Apr 2001, Fabio Riccardi wrote:

 I can disable header caching and see what happens, I'll add an option
 for this in the next X15 release.

what SPECweb99 performance do you get if you disable header caching? It
might not make all that much of a difference. (but it will make the
results more comparable with TUX.)

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Mon, 30 Apr 2001, Fabio Riccardi wrote:

 Ok I fixed it, the header date timestamp is updated with every
 request.

 Performance doesn't seem to have suffered significantly (less than
 1%).

yep, expected that - doing a sendmsg()+sendfile() generates the same
packet structure, the only difference being that ~100-200 bytes are copied
between kernel-space and user-space.

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

hm, another X15 caching artifact i noticed: i've changed the index.html
file, and while X15 was still serving the old copy, TUX served the new
file immediately.

its cache is apparently not coherent with actual VFS contents. (ie. it's a
read-once cache apparently?) TUX has some (occasionally significant)
overhead due to non-cached VFS-lookups.

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Pavel Machek]

Hi!

   In x86-64 there are special vsyscalls btw to solve this problem that export
   a lockless kernel gettimeofday()
  
  Whatever happened to that hack that was discussed a year or two ago?
  The one where (also on IA32) a magic page was set up by the kernel
  containing code for fast system calls, and the kernel would write
  calibation information to that magic page. The code written there
  would use the TSC in conjunction with that calibration data.
  
  There was much discussion about this idea, even Linus was keen on
  it. But IIRC, nothing ever happened.
  
 
 We discussed this at the Summit, not a year or two ago.  x86-64 has
 it, and it wouldn't be too bad to do in i386... just noone did.

Just wait what kind of problems it is able to bring on i386.

Pavel
PS: Hmm, how do you do timewarp for just one userland appliation with
this installed?

-- 
I'm [EMAIL PROTECTED] In my country we have almost anarchy and I don't care.
Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Ingo Molnar]

On Tue, 1 May 2001, Fabio Riccardi wrote:

 This is actually a bug in the current X15, I know how to fix it (with
 negligible overhead) but I've been lazy :)

yep, i think it's pretty straightforward: you have a cache of open file
descriptors (like Squid i think), and you can start a background 'cache
synchronization thread' that does a stat() on every open file's real VFS
path, every second or so. This should have small overhead (the number of
file descriptors cached should be limited anyway via some sort of LRU),
and guarantees 'practical coherency', without having the overhead of
immediate coherency. [total coherency is pointless anyway, not even the
kernel guarantees it to all parallel VFS users.]

Ingo

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[David S. Miller]

dean gaudet writes:
 > On Sun, 29 Apr 2001, David S. Miller wrote:
 > 
 > > If you do the TCP_CORK thing, what you end up with is a scatter gather
 > > entry in the SKB for the header bits, then the page cache segments.
 > 
 > so then the NIC would be sent a 3 entry gather list -- 1 entry for TCP/IP
 > headers, 1 for HTTP headers, and 1 for the initial page cache segment?

Basically.  It's weird because we could change tcp_sendmsg() to grab a
"little bit" of space in skb->data after the TCP headers area, but
that would screw all the memory allocation advantages carving up pages
gives us.

TCP used to be really rough on the memory subsystem, and in particular
going to a page carving scheme helped a lot in this area.

 > are there any NICs which take only 2 entry lists?  (boo hiss and curses
 > on such things if they exist!)

Tulip I think falls into this category, I could be wrong.  It has two
buffer pointers in the RX descriptor, but one might be able to chain
them.

Alexey added SG support to Tulip at some point, and I can probably dig
up the patch.  It doesn't do hw csumming, though.

Later,
David S. Miller
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[dean gaudet]

On Mon, 30 Apr 2001, Fabio Riccardi wrote:

> Ok I fixed it, the header date timestamp is updated with every request.
>
> Performance doesn't seem to have suffered significantly (less than 1%).

rad!

> BTW: Don't call me slime, I wasn't trying to cheat, I just didn't know that
> the date stamp was required to be really up-to-date.

sorry, i meant to put a smily on there :)


On Sun, 29 Apr 2001, David S. Miller wrote:

> If you do the TCP_CORK thing, what you end up with is a scatter gather
> entry in the SKB for the header bits, then the page cache segments.

so then the NIC would be sent a 3 entry gather list -- 1 entry for TCP/IP
headers, 1 for HTTP headers, and 1 for the initial page cache segment?

are there any NICs which take only 2 entry lists?  (boo hiss and curses
on such things if they exist!)

-dean

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Ok I fixed it, the header date timestamp is updated with every request.

Performance doesn't seem to have suffered significantly (less than 1%).

You can find the new release at:  http://www.chromium.com/X15-Alpha-2.tgz

BTW: Don't call me slime, I wasn't trying to cheat, I just didn't know that
the date stamp was required to be really up-to-date.

 - Fabio

dean gaudet wrote:

> On Sun, 29 Apr 2001, Fabio Riccardi wrote:
>
> > I can disable header caching and see what happens, I'll add an option
> > for this in the next X15 release.
>
> heh, well to be honest, i'd put the (permanent) caching of the Date header
> into the very slimy, benchmark-only trick category.  (right up there
> alongside running the HTTP and TCP stacks inside the NIC interrupt
> handler, which folks have done to get even better scores.)
>
> > Nevertheless I don't know how much this is interesting in real life,
> > since on the internet most static pages are cached on proxies. I agree
> > that the RFC asks for a date for the original response, but once the
> > response is cached what does this date mean?
>
> the Date is always the time the response was generated on the origin
> server.  (there are exceptions for clockless servers, but such servers
> have other limitations you wouldn't want -- notably they MUST NOT generate
> Last-Modified.)
>
> one example oddity you might experience with a non-increasing Date
> surrounds Last-Modified and Date, see section 13.3.3.  note that the rfc
> indicates that if Last-Modified is less than 60 seconds earlier than Date
> then Last-Modified is only a weak validator rather than a strong
> validator.  this would complicate range requests -- because weak
> validators can't be used with range requests.  if your server never
> updates the Date after the first time it serves an object then you'd
> potentially never get out of this 60 second window.
>
> (strong validators are guaranteed to change whenever the object changes...
> and Last-Modified isn't strong until some time has passed -- consider NFS
> mounted docroots, clock skew in the origin network, multiple file updates
> within a second, etc.)
>
> there are a bunch of other things that Date is used for, all of them are
> related to caching heuristics and rules.
>
> in theory you could claim that you're implementing a cache server rather
> than an origin server... i dunno what the SPEC committee will think when
> you try to submit results though :)
>
> so way back when sendfile() was being created i brought up the Date issue
> and pointed out that we needed more than a single "void *, size_t" to take
> care of headers.  eventually this discussion lead creation of TCP_CORK --
> so that a http server could use writev() to send a two element iov for the
> headers -- one element with everything that doesn't need to change, the
> next element with the Date.
>
> i also kind of expected the high performance servers to rebuild a Date:
> header every second for all of its threads to use.  (of course it's not
> that simple, you really want to keep a circular list of N dates... and
> just assume that after N seconds no thread could still be accessing an old
> Date.)
>
> is this too slow for some reason?  (does it play well with zero-copy?)
>
> -dean
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [EMAIL PROTECTED]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

> The point is: The code in that "magic page" that considers the
> tradeoff is KERNEL code, which is designed to care about such
> trade-offs for that machine. Glibc never knows this stuff and
> shouldn't, because it is already bloated.

glibc is bloated because it cares about such stuff and complex standards.
There is no reason to make a mess of the kernel when you can handle more
stuff nicely with the libraries.

Since glibc inlines most memcpy calls you'd need to build an MXT glibc,
which is doable. Uninlining most memcpy calls is a loss on some processors
and often a loss anyway as the copies are so small


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Jonathan Lundell]

At 12:29 AM -0700 2001-04-30, H. Peter Anvin wrote:
>"David S. Miller" wrote:
>>
>>  dean gaudet writes:
>>   > i was kind of solving a different problem with the code page 
>>though -- the
>>   > ability to use rdtsc on SMP boxes with processors of varying speeds and
>>   > synchronizations.
>>
>>  A better way to solve that problem is the way UltraSPARC-III do and
>>  future ia64 systems will, by way of a "system tick" register which
>>  increments at a constant rate regardless of how the cpus are clocked.
>>
>>  The "system tick" pulse goes into the processor, so it's still a local
>>  cpu register being accessed.  Think of it as a system bus clock cycle
>>  counter.
>>
>>  Granted, you probably couldn't make changes to the hardware you were
>>  working on at the time :-)
>>
>
>RDTSC in Crusoe processors does basically this.
>
>   -hpa

The Pentium III TSC has the bizarre characteristic, per Intel docs 
anyway, that only the low half can be written (as I recall the high 
half gets set to zero), making restoration problematical in certain 
power-management regimes. Hopefully the Crusoe does better.
-- 
/Jonathan Lundell.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[David S. Miller]

H. Peter Anvin writes:
 > RDTSC in Crusoe processors does basically this.

Hmmm, one of the advantages of using a seperate tick register for this
constant clock is that you can still do cycle accurate asm code
analysis even when the cpu is down clocked.

The joys of compatability I suppose :-)

Later,
David S. Miller
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[H. Peter Anvin]

"David S. Miller" wrote:
> 
> dean gaudet writes:
>  > i was kind of solving a different problem with the code page though -- the
>  > ability to use rdtsc on SMP boxes with processors of varying speeds and
>  > synchronizations.
> 
> A better way to solve that problem is the way UltraSPARC-III do and
> future ia64 systems will, by way of a "system tick" register which
> increments at a constant rate regardless of how the cpus are clocked.
> 
> The "system tick" pulse goes into the processor, so it's still a local
> cpu register being accessed.  Think of it as a system bus clock cycle
> counter.
> 
> Granted, you probably couldn't make changes to the hardware you were
> working on at the time :-)
> 

RDTSC in Crusoe processors does basically this.

-hpa

-- 
<[EMAIL PROTECTED]> at work, <[EMAIL PROTECTED]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[David S. Miller]

dean gaudet writes:
 > i was kind of solving a different problem with the code page though -- the
 > ability to use rdtsc on SMP boxes with processors of varying speeds and
 > synchronizations.

A better way to solve that problem is the way UltraSPARC-III do and
future ia64 systems will, by way of a "system tick" register which
increments at a constant rate regardless of how the cpus are clocked.

The "system tick" pulse goes into the processor, so it's still a local
cpu register being accessed.  Think of it as a system bus clock cycle
counter.

Granted, you probably couldn't make changes to the hardware you were
working on at the time :-)

Later,
David S. Miller
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[David S. Miller]

dean gaudet writes:
 > is this too slow for some reason?  (does it play well with zero-copy?)

His trick ends up with a minimal set of scatter gather entries.
That's the whole gain behind the trick he's doing.

If you do the TCP_CORK thing, what you end up with is a scatter gather
entry in the SKB for the header bits, then the page cache segments.

Even if we had the HP sendfile() interface iovec garbage, we would end
up with the same number of SKB iovec entries as for the TCP_CORK case
today.

What TUX basically does is build up the header by hand in a scribble
page it uses for header builing, passes that to tcp_sendpage() with
MSG_MORE set, then it initiates the sendfile() part.  The final effect
inside the networking is basically equivalent to using
TCP_CORK+sendfile() in userspace, the only difference being that:

1) the scratch page for the headers is maintained per-socket by TCP
2) the header is copied once from user to kernel

I would find it amusing to see what adding the header+file caching
trick to TUX would do to it's results :-)

Later,
David S. Miller
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[David S. Miller]

dean gaudet writes:
  is this too slow for some reason?  (does it play well with zero-copy?)

His trick ends up with a minimal set of scatter gather entries.
That's the whole gain behind the trick he's doing.

If you do the TCP_CORK thing, what you end up with is a scatter gather
entry in the SKB for the header bits, then the page cache segments.

Even if we had the HP sendfile() interface iovec garbage, we would end
up with the same number of SKB iovec entries as for the TCP_CORK case
today.

What TUX basically does is build up the header by hand in a scribble
page it uses for header builing, passes that to tcp_sendpage() with
MSG_MORE set, then it initiates the sendfile() part.  The final effect
inside the networking is basically equivalent to using
TCP_CORK+sendfile() in userspace, the only difference being that:

1) the scratch page for the headers is maintained per-socket by TCP
2) the header is copied once from user to kernel

I would find it amusing to see what adding the header+file caching
trick to TUX would do to it's results :-)

Later,
David S. Miller
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[David S. Miller]

dean gaudet writes:
  i was kind of solving a different problem with the code page though -- the
  ability to use rdtsc on SMP boxes with processors of varying speeds and
  synchronizations.

A better way to solve that problem is the way UltraSPARC-III do and
future ia64 systems will, by way of a system tick register which
increments at a constant rate regardless of how the cpus are clocked.

The system tick pulse goes into the processor, so it's still a local
cpu register being accessed.  Think of it as a system bus clock cycle
counter.

Granted, you probably couldn't make changes to the hardware you were
working on at the time :-)

Later,
David S. Miller
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[H. Peter Anvin]

David S. Miller wrote:
 
 dean gaudet writes:
   i was kind of solving a different problem with the code page though -- the
   ability to use rdtsc on SMP boxes with processors of varying speeds and
   synchronizations.
 
 A better way to solve that problem is the way UltraSPARC-III do and
 future ia64 systems will, by way of a system tick register which
 increments at a constant rate regardless of how the cpus are clocked.
 
 The system tick pulse goes into the processor, so it's still a local
 cpu register being accessed.  Think of it as a system bus clock cycle
 counter.
 
 Granted, you probably couldn't make changes to the hardware you were
 working on at the time :-)
 

RDTSC in Crusoe processors does basically this.

-hpa

-- 
[EMAIL PROTECTED] at work, [EMAIL PROTECTED] in private!
Unix gives you enough rope to shoot yourself in the foot.
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[David S. Miller]

H. Peter Anvin writes:
  RDTSC in Crusoe processors does basically this.

Hmmm, one of the advantages of using a seperate tick register for this
constant clock is that you can still do cycle accurate asm code
analysis even when the cpu is down clocked.

The joys of compatability I suppose :-)

Later,
David S. Miller
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Jonathan Lundell]

At 12:29 AM -0700 2001-04-30, H. Peter Anvin wrote:
David S. Miller wrote:

  dean gaudet writes:
i was kind of solving a different problem with the code page 
though -- the
ability to use rdtsc on SMP boxes with processors of varying speeds and
synchronizations.

  A better way to solve that problem is the way UltraSPARC-III do and
  future ia64 systems will, by way of a system tick register which
  increments at a constant rate regardless of how the cpus are clocked.

  The system tick pulse goes into the processor, so it's still a local
  cpu register being accessed.  Think of it as a system bus clock cycle
  counter.

  Granted, you probably couldn't make changes to the hardware you were
  working on at the time :-)


RDTSC in Crusoe processors does basically this.

   -hpa

The Pentium III TSC has the bizarre characteristic, per Intel docs 
anyway, that only the low half can be written (as I recall the high 
half gets set to zero), making restoration problematical in certain 
power-management regimes. Hopefully the Crusoe does better.
-- 
/Jonathan Lundell.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Alan Cox]

 The point is: The code in that magic page that considers the
 tradeoff is KERNEL code, which is designed to care about such
 trade-offs for that machine. Glibc never knows this stuff and
 shouldn't, because it is already bloated.

glibc is bloated because it cares about such stuff and complex standards.
There is no reason to make a mess of the kernel when you can handle more
stuff nicely with the libraries.

Since glibc inlines most memcpy calls you'd need to build an MXT glibc,
which is doable. Uninlining most memcpy calls is a loss on some processors
and often a loss anyway as the copies are so small


-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Ok I fixed it, the header date timestamp is updated with every request.

Performance doesn't seem to have suffered significantly (less than 1%).

You can find the new release at:  http://www.chromium.com/X15-Alpha-2.tgz

BTW: Don't call me slime, I wasn't trying to cheat, I just didn't know that
the date stamp was required to be really up-to-date.

 - Fabio

dean gaudet wrote:

 On Sun, 29 Apr 2001, Fabio Riccardi wrote:

  I can disable header caching and see what happens, I'll add an option
  for this in the next X15 release.

 heh, well to be honest, i'd put the (permanent) caching of the Date header
 into the very slimy, benchmark-only trick category.  (right up there
 alongside running the HTTP and TCP stacks inside the NIC interrupt
 handler, which folks have done to get even better scores.)

  Nevertheless I don't know how much this is interesting in real life,
  since on the internet most static pages are cached on proxies. I agree
  that the RFC asks for a date for the original response, but once the
  response is cached what does this date mean?

 the Date is always the time the response was generated on the origin
 server.  (there are exceptions for clockless servers, but such servers
 have other limitations you wouldn't want -- notably they MUST NOT generate
 Last-Modified.)

 one example oddity you might experience with a non-increasing Date
 surrounds Last-Modified and Date, see section 13.3.3.  note that the rfc
 indicates that if Last-Modified is less than 60 seconds earlier than Date
 then Last-Modified is only a weak validator rather than a strong
 validator.  this would complicate range requests -- because weak
 validators can't be used with range requests.  if your server never
 updates the Date after the first time it serves an object then you'd
 potentially never get out of this 60 second window.

 (strong validators are guaranteed to change whenever the object changes...
 and Last-Modified isn't strong until some time has passed -- consider NFS
 mounted docroots, clock skew in the origin network, multiple file updates
 within a second, etc.)

 there are a bunch of other things that Date is used for, all of them are
 related to caching heuristics and rules.

 in theory you could claim that you're implementing a cache server rather
 than an origin server... i dunno what the SPEC committee will think when
 you try to submit results though :)

 so way back when sendfile() was being created i brought up the Date issue
 and pointed out that we needed more than a single void *, size_t to take
 care of headers.  eventually this discussion lead creation of TCP_CORK --
 so that a http server could use writev() to send a two element iov for the
 headers -- one element with everything that doesn't need to change, the
 next element with the Date.

 i also kind of expected the high performance servers to rebuild a Date:
 header every second for all of its threads to use.  (of course it's not
 that simple, you really want to keep a circular list of N dates... and
 just assume that after N seconds no thread could still be accessing an old
 Date.)

 is this too slow for some reason?  (does it play well with zero-copy?)

 -dean

 -
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to [EMAIL PROTECTED]
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[dean gaudet]

On Mon, 30 Apr 2001, Fabio Riccardi wrote:

 Ok I fixed it, the header date timestamp is updated with every request.

 Performance doesn't seem to have suffered significantly (less than 1%).

rad!

 BTW: Don't call me slime, I wasn't trying to cheat, I just didn't know that
 the date stamp was required to be really up-to-date.

sorry, i meant to put a smily on there :)


On Sun, 29 Apr 2001, David S. Miller wrote:

 If you do the TCP_CORK thing, what you end up with is a scatter gather
 entry in the SKB for the header bits, then the page cache segments.

so then the NIC would be sent a 3 entry gather list -- 1 entry for TCP/IP
headers, 1 for HTTP headers, and 1 for the initial page cache segment?

are there any NICs which take only 2 entry lists?  (boo hiss and curses
on such things if they exist!)

-dean

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[David S. Miller]

dean gaudet writes:
  On Sun, 29 Apr 2001, David S. Miller wrote:
  
   If you do the TCP_CORK thing, what you end up with is a scatter gather
   entry in the SKB for the header bits, then the page cache segments.
  
  so then the NIC would be sent a 3 entry gather list -- 1 entry for TCP/IP
  headers, 1 for HTTP headers, and 1 for the initial page cache segment?

Basically.  It's weird because we could change tcp_sendmsg() to grab a
little bit of space in skb-data after the TCP headers area, but
that would screw all the memory allocation advantages carving up pages
gives us.

TCP used to be really rough on the memory subsystem, and in particular
going to a page carving scheme helped a lot in this area.

  are there any NICs which take only 2 entry lists?  (boo hiss and curses
  on such things if they exist!)

Tulip I think falls into this category, I could be wrong.  It has two
buffer pointers in the RX descriptor, but one might be able to chain
them.

Alexey added SG support to Tulip at some point, and I can probably dig
up the patch.  It doesn't do hw csumming, though.

Later,
David S. Miller
[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[dean gaudet]

On Sun, 29 Apr 2001, Fabio Riccardi wrote:

> I can disable header caching and see what happens, I'll add an option
> for this in the next X15 release.

heh, well to be honest, i'd put the (permanent) caching of the Date header
into the very slimy, benchmark-only trick category.  (right up there
alongside running the HTTP and TCP stacks inside the NIC interrupt
handler, which folks have done to get even better scores.)

> Nevertheless I don't know how much this is interesting in real life,
> since on the internet most static pages are cached on proxies. I agree
> that the RFC asks for a date for the original response, but once the
> response is cached what does this date mean?

the Date is always the time the response was generated on the origin
server.  (there are exceptions for clockless servers, but such servers
have other limitations you wouldn't want -- notably they MUST NOT generate
Last-Modified.)

one example oddity you might experience with a non-increasing Date
surrounds Last-Modified and Date, see section 13.3.3.  note that the rfc
indicates that if Last-Modified is less than 60 seconds earlier than Date
then Last-Modified is only a weak validator rather than a strong
validator.  this would complicate range requests -- because weak
validators can't be used with range requests.  if your server never
updates the Date after the first time it serves an object then you'd
potentially never get out of this 60 second window.

(strong validators are guaranteed to change whenever the object changes...
and Last-Modified isn't strong until some time has passed -- consider NFS
mounted docroots, clock skew in the origin network, multiple file updates
within a second, etc.)

there are a bunch of other things that Date is used for, all of them are
related to caching heuristics and rules.

in theory you could claim that you're implementing a cache server rather
than an origin server... i dunno what the SPEC committee will think when
you try to submit results though :)

so way back when sendfile() was being created i brought up the Date issue
and pointed out that we needed more than a single "void *, size_t" to take
care of headers.  eventually this discussion lead creation of TCP_CORK --
so that a http server could use writev() to send a two element iov for the
headers -- one element with everything that doesn't need to change, the
next element with the Date.

i also kind of expected the high performance servers to rebuild a Date:
header every second for all of its threads to use.  (of course it's not
that simple, you really want to keep a circular list of N dates... and
just assume that after N seconds no thread could still be accessing an old
Date.)

is this too slow for some reason?  (does it play well with zero-copy?)

-dean








-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Andrea Arcangeli]

On Sun, Apr 29, 2001 at 04:18:27PM -0400, Gregory Maxwell wrote:
> having both the code and a comprehensive jump-table might become tough in a

In the x86-64 implementation there's no jump table. The original design
had a jump table but Peter raised the issue that indirect jumps are very
costly and he suggested to jump to a fixed virtual address instead, I
agreed with his suggestion. So this is what I implemented for x86-64
with regard to the userspace vsyscall API (which will be used by glibc):

enum vsyscall_num {
__NR_vgettimeofday,
__NR_vtime,
};

#define VSYSCALL_ADDR(vsyscall_nr) (VSYSCALL_START+VSYSCALL_SIZE*(vsyscall_nr))

the linker can prelink the vsyscall virtual address into the binary as a
weak symbol and the dynamic linker will need to patch it only if
somebody is overriding the weak symbol with a LD_PRELOAD.

Virtual address space is relatively cheap. Currently the 64bit
vgettimeofday bytecode + data is nearly 200 bytes, and the first two
slots are large 512bytes each. So with 1024 bytes we do the whole thing,
and we still have space for further 6 vsyscalls without paying any
additional tlb entry.

(the implementation of the above #define will change shortly but the
VSYSCALL_ADDR() API for glibc will remain the same)

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Andrea Arcangeli]

On Sun, Apr 29, 2001 at 09:38:04PM +0200, Jamie Lokier wrote:
> Fwiw, modern x86 has global TLB entries too.

my x86-64 implementation is marking the tlb entry global of course (so
it's not flushed during context switch):

#define __PAGE_KERNEL_VSYSCALL \
(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED)
#define MAKE_GLOBAL(x) __pgprot((x) | _PAGE_GLOBAL)
#define PAGE_KERNEL_VSYSCALL MAKE_GLOBAL(__PAGE_KERNEL_VSYSCALL)

static void __init map_vsyscall(void)
{
extern char __vsyscall_0;
unsigned long physaddr_page0 = (unsigned long) &__vsyscall_0 - 
__START_KERNEL_map;

__set_fixmap(VSYSCALL_FIRST_PAGE, physaddr_page0, PAGE_KERNEL_VSYSCALL);
}

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Richard Gooch]

H. Peter Anvin writes:
> The thing that made me say we discussed this last month was
> Richard's comment that it had already been implemented (which it
> has, by Andrea, for x86-64.)  The idea of doing it for i386 has been
> kicked around for

Correction: I didn't say it had been implemented. I just asked what
happened to the idea. I never saw it go into i386.

Regards,

Richard
Permanent: [EMAIL PROTECTED]
Current:   [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Richard Gooch]

Gregory Maxwell writes:
> On Sun, Apr 29, 2001 at 10:11:59PM +0200, Ingo Oeser wrote:
> [snip]
> > The point is: The code in that "magic page" that considers the
> > tradeoff is KERNEL code, which is designed to care about such
> > trade-offs for that machine. Glibc never knows this stuff and
> > shouldn't, because it is already bloated.
> > 
> > We get the full win here, for our "compile the kernel for THIS
> > machine to get maximum performance"-strategy.
> > 
> > People tend to compile the kernel, but not the glibc.
> > 
> > Just let the benchmarks, Linus and Ulrich decide ;-)
> 
> The kernel can even customize the page at runtime if it needs to, such as
> changing algorithims to deal with lock contention.
> 
> Of course, this page will need to present a stable interface to
> glibc, and having both the code and a comprehensive jump-table might
> become tough in a single page...

Sure. IIRC, Linus talked about "a few pages".

Regards,

Richard
Permanent: [EMAIL PROTECTED]
Current:   [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Richard Gooch]

Ingo Oeser writes:
> On Sun, Apr 29, 2001 at 12:48:06PM -0600, Richard Gooch wrote:
> > Ingo Oeser writes:
> > > There we have 10x faster memmove/memcpy/bzero for 1K blocks
> > > granularity (== alignment is 1K and size is multiple of 1K), that
> > > is done by the memory controller.
> > This sounds different to me. Using the memory controller is (should
> > be!) a privileged operation, thus it requires a system call. This is
> > quite different from code in a magic page, which is excuted entirely
> > in user-space. The point of the magic page is to avoid the syscall
> > overhead.
> 
> Yes, but we currently have more than 10K cycles for doing
> memset of a page. If we do an syscall, we have around 600-900
> (don't know exactly), which is still less.
> 
> The point is: The code in that "magic page" that considers the
> tradeoff is KERNEL code, which is designed to care about such
> trade-offs for that machine.

Um, yes. I don't disagree with that. I'm just saying the two issues
are conceptually separate, and should be considered independently.

> Glibc never knows this stuff and shouldn't, because it is already
> bloated.

True, true and true.

Regards,

Richard
Permanent: [EMAIL PROTECTED]
Current:   [EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Jim Gettys]

> 
> Short summary: depending on how much you were talking general idea versus
> specifics, you can go arbitrarily far back (I wouldn't be surprised if
> shared memory techniques were used regularly before memory protection.)
> 
> Fair?

Very fair.

> 
> Not to pick on you or anyone else, but it is well-known to everyone
> except the U.S. patent office that "there are no new ideas in computer
> science." :)
> 


Exactly why I noted in my mail that I didn't consider it novel even back then; just
a good engineering idea that we went ahead and used a long time ago...
- Jim
--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[H. Peter Anvin]

Jim Gettys wrote:
> 
> The "put the time into a magic location in shared memory" goes back...
>

Short summary: depending on how much you were talking general idea versus
specifics, you can go arbitrarily far back (I wouldn't be surprised if
shared memory techniques were used regularly before memory protection.)

Fair?

Not to pick on you or anyone else, but it is well-known to everyone
except the U.S. patent office that "there are no new ideas in computer
science." :)

-hpa

-- 
<[EMAIL PROTECTED]> at work, <[EMAIL PROTECTED]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

I can disable header caching and see what happens, I'll add an option for this
in the next X15 release.

Nevertheless I don't know how much this is interesting in real life, since on
the internet most static pages are cached on proxies. I agree that the
RFC asks for a date for the original response, but once the response is cached
what does this date mean?

 - Fabio

Ingo Molnar wrote:

> Fabio,
>
> i noticed one weirdness in the Date-field handling of X15. X15 appears to
> cache the Date field too, which is contrary to RFCs:
>
>  earth2:~> wget -s http://localhost/index.html -O - 2>/dev/null | grep Date
>  Date: Sat Apr 28 10:15:14 2001
>  earth2:~> date
>  Sat Apr 28 10:32:40 CEST 2001
>
> ie. there is already a 15 minutes difference between the 'origin date of
> the reply' and the actual date of the reply. (i started X15 up 15 minutes
> ago.)
>
> per RFC 2616:
> .
> The Date general-header field represents the date and time at which the
> message was originated, [...]
>
> Origin servers MUST include a Date header field in all responses, [...]
> .
>
> i considered the caching of the Date field for TUX too, and avoided it
> exactly due to this issue, to not violate this 'MUST' item in the RFC. It
> can be reasonably expected from a web server to have a 1-second accurate
> Date: field.
>
> the header-caching in X15 gives it an edge against TUX, obviously, but IMO
> it's a questionable practice.
>
> if caching of headers was be allowed then we could the obvious trick of
> sendfile()ing complete web replies (first header, then body).
>
> Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Jim Gettys]

The "put the time into a magic location in shared memory" goes back, as
far as I know, to Bob Scheifler or myself for the X Window System, sometime
around 1984 or 1985: we put it into a page of shared memory where we used
a circular buffer scheme to put input events (keyboard/mice), so that
we could avoid the read system call overhead to get these events (and
more importantly, check between each request if there was input to
process).  I don't think we ever claimed it was novel, just that we did
it that way (I'd have to ask Bob if he had heard of that before we did
it).  We put it into the same piece of memory we put the circular event
buffer, avoiding both the get-time-of day calls, but also the much more
expensive reads that would have been required (we put the events into a
circular buffer, with the kernel only updating one value, and user space
updating the other value defining the circular buffer).

In X, it is important for interactivity to get input events and send them
to clients ASAP: just note the effect of Keith Packard's recent implementation
of "silken mouse", where signals are used to deliver events to the X server.
This finally has made mouse tracking (done in user space on Linux; generally
done by kernel drivers on most UNIX boxes) what we were getting on 1 mip machines
under load (Keith has also done more than this with his new internal X
scheduler, which prevents clients from monopolizing the X server anywhere
like the old implementation).

This shared memory technique is very powerful to allow a client application to know if
it needs to do a system call, and is very useful for high performance servers
(like X), where a system call is way too expensive.

I've certainly mentioned this technique in the past in the Web community
(but HTTP servers are processing requests about 1/100-1/1000 the rate of
an X server, which gets into the millions of requests/second on current machines.

So if you want to get user space to really go fast, sometimes you resort
to such trickery  I think the technique has real value: the interesting
question is should there be general kernel facilities to make this easy
(we did it via ugly hacks on VAX and MIPS boxes) for kernel facilities
to provide.

"X is an exercise in avoiding system calls".  I think I said this around
1984-1985.  
- Jim

--
Jim Gettys
Technology and Corporate Development
Compaq Computer Corporation
[EMAIL PROTECTED]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space

[Fabio Riccardi]

Linux 2.4 is surely one of the most advanced OSs ever happened, especially
from the optimization point of view and for the admirable economy of concepts
on which it lies. I definitively hope that X15 helps reinforcing the success
to this amazing system.

TUX has definitively been my performance yardstick for the development of
X15, but I had many sources of inspiration for the X15 architecture. Maybe
the most relevant are the Flash Web Server (Pai, Druschel, Zwaenepoel),
several Linus observations on this list about (web) server architecture and
kernnel services, and the reading of the Hennessy & Patterson architecture
books. Last but not least, aside from some heated discussions, research in
microkernel architecture has taught us many lessons on how to achieve an
efficient model of interaction across separate addressing spaces.

If i have to make some sort of educated guess and point at where the current
bottleneck lies for web server performance, I would say that it is somewhere
between the memory subsystem and the PCI bus.

With zero-copy sendfile data movement is not an issue anymore, asynchronous
network IO allows for really inexpensive thread scheduling, and system call
invocation adds a very negligible overhead in Linux. What we are left with
now is purely wait cycles, the CPUs and the NICs are contending for memory
and bus bandwidth. It would be really interesting to see where the network
shifts now that faster machines are becoming available.

On my whish list for future kernel developments I would definitively put disk
asynchronous IO and a more decent file descriptor passing implementation.
I'll detail this in subsequent messages.

I'll surely check out the impact of Ingo's patches on TUX performance
sometime this week.

I'd also like to reiterate my request for help for testing X15 on higher end
server architectures.

X15 is still very young alpha code and I can surely improve its performance
in many ways.

 - Fabio

Ingo Molnar wrote:

> On Fri, 27 Apr 2001, Fabio Riccardi wrote:
>
> > I'd like to announce the first release of X15 Alpha 1, a _user space_
> > web server that is as fast as TUX.
>
> great, the first TUX clone! ;-)
>
> This should put the accusations to rest that Linux got the outstandingly
> high SPECweb99 scores only because the webserver was in kernel-space. It's
> the 2.4 kernel's high performance that enabled those results, having the
> web-server in kernel-space didnt have much effect. TUX was and remains a
> testbed to test high-performance webserving (and FTP serving), without the
> API-exporting overhead of userspace.
>
> [i suspect the small performance advantage of X15 is due to subtle
> differences in the SPECweb99 user-space module: eg. while the TUX code was
> written, tested and ready to use mmap()-enabled
> TUXAPI_alloc_read_objectbuf(), it wasnt enabled actually. I sent Fabio a
> mail how to enable it, perhaps he can do some tests to confirm this
> suspicion?]
>
> doing a TUX 2.0 SPECweb99 benchmark on the latest -ac kernels, 86% of time
> is spent in generic parts of the kernel, 12% of time is spent in the
> user-space SPECweb99 module, and only 2% of time is spent in TUX-specific
> kernel code.
>
> doing the same test with the original TUX 1.0 code shows that more than
> 50% of CPU time was spent in TUX-specific code.
>
> what does this mean? In the roughly 6 months since TUX 1.0 was released,
> we moved much of the TUX 1.0 -only improvements into the generic kernel
> (most of which was made available to user-space as well), and TUX itself
> became smaller and smaller (and used more and more generic parts of the
> kernel). So in effect X15 is executing 50% TUX code :-)
>
> (there are still a number of performance improvement patches pending that
> are not integrated yet: the pagecache extreme-scalability patch and the
> smptimers patch. These patches speed both X15 and TUX up.)
>
> (there is one thing though that can never be 'exported to user-space': to
> isolate possibly untrusted binary application code from the server itself,
> without performance degradation. So we always have to be mentally open to
> the validity of kernel-space services.)
>
> Ingo

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Arjan van de Ven]

In article <[EMAIL PROTECTED]> you wrote:
> Yes, but we currently have more than 10K cycles for doing
> memset of a page.

make that 3800 or so. (700 Mhz AMD Duron)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[H. Peter Anvin]

Followup to:  <[EMAIL PROTECTED]>
By author:dean gaudet <[EMAIL PROTECTED]>
In newsgroup: linux.dev.kernel
>
> On Sun, 29 Apr 2001, Jeff Garzik wrote:
> 
> > "H. Peter Anvin" wrote:
> > > We discussed this at the Summit, not a year or two ago.  x86-64 has
> > > it, and it wouldn't be too bad to do in i386... just noone did.
> >
> > It came up long before that.  I refer to the technique in a post dated
> > Nov 17, even though I can't find the original.
> > http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg13584.html
> >
> > Initiated by a post from (iirc) Dean Gaudet, we found out that
> > gettimeofday was one particular system call in the Apache fast path that
> > couldn't be optimized well, or moved out of the fast path.  After a
> > couple of suggestions for improving things, Linus chimed in with the
> > magic page suggestion.
> 
> heheh.  i can't claim that i was the first ever to think of this.  but
> here's the post i originally made on the topic.  iirc a few folks said
> "security horror!"... then last year ingo and linus (and probably others)
> came up with a scheme everyone was happy with.
> 
> i was kind of solving a different problem with the code page though -- the
> ability to use rdtsc on SMP boxes with processors of varying speeds and
> synchronizations.
> 

The thing that made me say we discussed this last month was Richard's
comment that it had already been implemented (which it has, by Andrea,
for x86-64.)  The idea of doing it for i386 has been kicked around for
years, originally as a way to handle INT 0x80 vs SYSENTER vs SYSCALL,
which I think is part of why it never got implemented, since handling
multiple flavours of system calls apparently causes some pain in the
system call entry/exit path.

The handling of a few things like gettimeofday etc. was something we
observed could be added on top at that time, but was largely
considered secondary.

-hpa

-- 
<[EMAIL PROTECTED]> at work, <[EMAIL PROTECTED]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Gregory Maxwell]

On Sun, Apr 29, 2001 at 10:11:59PM +0200, Ingo Oeser wrote:
[snip]
> The point is: The code in that "magic page" that considers the
> tradeoff is KERNEL code, which is designed to care about such
> trade-offs for that machine. Glibc never knows this stuff and
> shouldn't, because it is already bloated.
> 
> We get the full win here, for our "compile the kernel for THIS
> machine to get maximum performance"-strategy.
> 
> People tend to compile the kernel, but not the glibc.
> 
> Just let the benchmarks, Linus and Ulrich decide ;-)

The kernel can even customize the page at runtime if it needs to, such as
changing algorithims to deal with lock contention.

Of course, this page will need to present a stable interface to glibc, and
having both the code and a comprehensive jump-table might become tough in a
single page...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: X15 alpha release: as fast as TUX but in user space (fwd)

[Ingo Oeser]

On Sun, Apr 29, 2001 at 12:48:06PM -0600, Richard Gooch wrote:
> Ingo Oeser writes:
> > There we have 10x faster memmove/memcpy/bzero for 1K blocks
> > granularity (== alignment is 1K and size is multiple of 1K), that
> > is done by the memory controller.
> This sounds different to me. Using the memory controller is (should
> be!) a privileged operation, thus it requires a system call. This is
> quite different from code in a magic page, which is excuted entirely
> in user-space. The point of the magic page is to avoid the syscall
> overhead.

Yes, but we currently have more than 10K cycles for doing
memset of a page. If we do an syscall, we have around 600-900
(don't know exactly), which is still less.

The point is: The code in that "magic page" that considers the
tradeoff is KERNEL code, which is designed to care about such
trade-offs for that machine. Glibc never knows this stuff and
shouldn't, because it is already bloated.

We get the full win here, for our "compile the kernel for THIS
machine to get maximum performance"-strategy.

People tend to compile the kernel, but not the glibc.

Just let the benchmarks, Linus and Ulrich decide ;-)

Regards

Ingo Oeser
-- 
10.+11.03.2001 - 3. Chemnitzer LinuxTag 
  been there and had much fun   
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/