Re: in-kernel HTTP Server for FreeBSD?
Terry Lambert wrote: What about HyperTransport? (Not that I know anything about it, but those nice AMD guys keep mentioning it in sales garbage :) They keep mentioning SledgeHammer, too... Have you seen silicon for either one of them yet? I don't pretend to know much about any of this, but nVidia's new nForce chipset claims to support AMD's HyperTransport. It's mentioned on the bottom of http://www.nvidia.com/view.asp?IO=nppa and possibly in one of the tech brief PDF's in more detail. And nForce motherboards are certainly available to buy right now. Andrew. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Hey, On Sun, Feb 17, 2002 at 06:33:43AM -0800, Hiten Pandya wrote: hi all, [ I am not currently subscribed to -questions, please CC a copy to me, thanks. :) ] I beleive this question is not intended for the -hackers list, but might be interesting. I have also cc'ed this to the -questions list. Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? Ive never seen or heard of one, at least not one available to the general public. Services should be executed in the user space, thats what its for. Cross-polluting these user/kernel environments is to be discouraged. Thats my opinion anyway. Thanks, Regards, -- Hiten Pandya -- [EMAIL PROTECTED] __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- Dominic To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Hiten Pandya [EMAIL PROTECTED] writes: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
* Dag-Erling Smorgrav ([EMAIL PROTECTED]) wrote: Hiten Pandya [EMAIL PROTECTED] writes: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. Don't functions like FreeBSD's zero-copy sendfile() provide similar performance benefits without the massive security issues? If the remaining speed hog of switching to userspace to process the request itself is causing noticable bottlenecks, I think that's a sign you need more Pentium than the service needs moving to the kernel :) -- Thomas 'Freaky' Hurst - [EMAIL PROTECTED] - http://www.aagh.net/ - One of the disadvantages of having children is that they eventually get old enough to give you presents they make at school. -- Robert Byrne To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On Sun, Feb 17, 2002 at 06:33:43AM -0800, Hiten Pandya wrote: Hi, Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? As others have pointed out, it may have big hacker value, but no one would use it for anything serious. AFAIK there's no such thing for FreeBSD, but one thing I remember, is that once the Linux kernel incorporated the zero copy netowrking code, userland HTTP servicing like Tux saw it's performance increase on par with khttpd, so it seems that's not worth to run a http server in kernel space. Cheers, -- Miguel Mendez - [EMAIL PROTECTED] GPG Public Key :: http://energyhq.homeip.net/files/pubkey.txt EnergyHQ :: http://www.energyhq.tk FreeBSD - The power to serve! msg31886/pgp0.pgp Description: PGP signature
Re: in-kernel HTTP Server for FreeBSD?
Thomas Hurst [EMAIL PROTECTED] writes: Don't functions like FreeBSD's zero-copy sendfile() provide similar performance benefits without the massive security issues? sendfile() isn't zero-copy, it's just two-less-copies. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory non-zero-copy usually goes device - buffer - user space - buffer - device (2 copy operations) zero-copy device - buffer/userspace/shared - device -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. well .. So let's turn the question upside-down, and ask Is there a web server or -accelerator for FreeBSD with similar performance as with khttpd or Tux? -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: well .. So let's turn the question upside-down, and ask Is there a web server or -accelerator for FreeBSD with similar performance as with khttpd or Tux? Have you tried thttpd or boa? DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
there is one. Written by [EMAIL PROTECTED] I think it's proprietary but maybe not.. (CC'd) On Sun, 17 Feb 2002, Dominic Marks wrote: Hey, On Sun, Feb 17, 2002 at 06:33:43AM -0800, Hiten Pandya wrote: hi all, [ I am not currently subscribed to -questions, please CC a copy to me, thanks. :) ] I beleive this question is not intended for the -hackers list, but might be interesting. I have also cc'ed this to the -questions list. Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? Ive never seen or heard of one, at least not one available to the general public. Services should be executed in the user space, thats what its for. Cross-polluting these user/kernel environments is to be discouraged. Thats my opinion anyway. Thanks, Regards, -- Hiten Pandya -- [EMAIL PROTECTED] __ Do You Yahoo!? Yahoo! Sports - Coverage of the 2002 Olympic Games http://sports.yahoo.com To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- Dominic To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Hey, On Sun, Feb 17, 2002 at 07:15:10PM +0100, Roy Sigurd Karlsbakk wrote: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. well .. So let's turn the question upside-down, and ask Is there a web server or -accelerator for FreeBSD with similar performance as with khttpd or Tux? Zeus is the best overall. A lot of people also seem to have good performance from tHttpd and Boa. When you say 'similar performance' how many Mbit/s can Tux achieve (for static and dynamic content aggregated). -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- Dominic To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Have you tried thttpd or boa? thttpd chrashed when 2 simultanous downloads was started (large files) haven tried boa... DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory To an MCSE, maybe. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On 17 Feb 2002, Dag-Erling Smorgrav wrote: Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory To an MCSE, maybe. I think Roy is right. AFAIK the term zero copy was invented by Van Jacobsen ca. 1990 to describe an optimized TCP stack he had working with the Witless interface project he did with HP, while he was still at LBL. Witless was an FDDI interface with interesting properties -- still well worth studying today. And, there was one copy in the TCP for Witless. You had to read zero copy to mean Zero copies more than the absolute minimum. When we did the MINI interface at the SRC (ca. 1994), which had one less copy than Witless, we jokingly called it a -1 copy interface. ron To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
In article [EMAIL PROTECTED], Julian Elischer [EMAIL PROTECTED] wrote: there is one. Written by [EMAIL PROTECTED] I think it's proprietary but maybe not.. (CC'd) Yes, I wrote one. Yes, it's proprietary. To those who dismissed it as a dumb idea: broaden your minds. It's extremely useful for certain specialized applications. One obvious example is as part of a testbed for performance testing various kinds of network appliances. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
* John Polstra ([EMAIL PROTECTED]) wrote: To those who dismissed it as a dumb idea: broaden your minds. It wasn't dismissed as a dumb idea, more an idea nobody would use for a production webserver, which I doubt includes: a testbed for performance testing various kinds of network appliances. Despite how useful it might be for benchmarking :) -- Thomas 'Freaky' Hurst - [EMAIL PROTECTED] - http://www.aagh.net/ - Never eat more than you can lift. -- Miss Piggy To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory To an MCSE, maybe. strange ... It's interesting that Dr. Scient Paal Halvorsen that recently finished his thesis about the subject 'Memory buffering / caching in multimedia streaming systems'. This has the following description The INSTANCE operating system enhancements show great improvements for multimedia streaming. In this thesis, we want to look at buffering/caching/prefetching mechanisms suitable for multimedia streams combined with the INSTANCE zero-copy data path. See http://www.ifi.uio.no/~paalh/index2.html for more info hm Am I an idiot here or have someone else misunderstood? -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory To an MCSE, maybe. strange ... [...] So what would you call direct DMA from the disk controller to the network adapter? Minus-one-copy? And even in the sendfile(2) case, data sometimes *is* copied in-core to satisfy alignment requirements etc. Stop using buzzwords just because they give you a woody. (and yes, even a Dr. Scient can be mistaken. Papers don't make you smart, you know - though I wouldn't expect someone who brags about being an MCSE and MCNE to understand that) DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
So what would you call direct DMA from the disk controller to the network adapter? Minus-one-copy? And even in the sendfile(2) case, data sometimes *is* copied in-core to satisfy alignment requirements etc. Stop using buzzwords just because they give you a woody. buzzwords or whatever - I really dont care. But can you please point at *one* application/system/whatever that is using HDD-NIC DMA? (and yes, even a Dr. Scient can be mistaken. Papers don't make you smart, you know - though I wouldn't expect someone who brags about being an MCSE and MCNE to understand that) er.. So - if you certify within a product, you'll probably become dumber? Grow up -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
now all we need is to connnect it to sysctl remote configuration of your kernel via web interface (runs screaming from room) On Sun, 17 Feb 2002, John Polstra wrote: In article [EMAIL PROTECTED], Julian Elischer [EMAIL PROTECTED] wrote: there is one. Written by [EMAIL PROTECTED] I think it's proprietary but maybe not.. (CC'd) Yes, I wrote one. Yes, it's proprietary. To those who dismissed it as a dumb idea: broaden your minds. It's extremely useful for certain specialized applications. One obvious example is as part of a testbed for performance testing various kinds of network appliances. John -- John Polstra John D. Polstra Co., Inc.Seattle, Washington USA Disappointment is a good sign of basic intelligence. -- Chögyam Trungpa To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
As far as I can see about publishing your Product Status is really immature you know. To be posting this in a open source community is be asking for an confrontation. It is not about being smart of dump, to keep in track and learning the beauty of technology but even to make bigger sense in a way this community helps each other the best way is to cut the Status crap at the end of your signature. Enjoy your Certifcates on the wall in your living room in a private manner you know what I mean? . But bragging about that stuff in your mail is not a way to earn respect. I really dislike to see this beef, I respect you and I'm proud for your achievements but for the future keep it low profile, or else it would only be used against you. And can somebody please help me with the no buffer space problem I posted yesterday ;-) Should I say more? Marcel de Vries, Swim Diploma, Firt Aid, And I don't give a F##K At 20:15 17-02-2002 +0100, Roy Sigurd Karlsbakk wrote: So what would you call direct DMA from the disk controller to the network adapter? Minus-one-copy? And even in the sendfile(2) case, data sometimes *is* copied in-core to satisfy alignment requirements etc. Stop using buzzwords just because they give you a woody. buzzwords or whatever - I really dont care. But can you please point at *one* application/system/whatever that is using HDD-NIC DMA? (and yes, even a Dr. Scient can be mistaken. Papers don't make you smart, you know - though I wouldn't expect someone who brags about being an MCSE and MCNE to understand that) er.. So - if you certify within a product, you'll probably become dumber? Grow up -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Can we move the personal slights off of -hackers? This is extraordinarily unprofessional. If you need to do this in a public forum, do it on -chat. On Sun, 17 Feb 2002, Marcel de Vries wrote: As far as I can see about publishing your Product Status is really immature you know. To be posting this in a open source community is be asking for an confrontation. It is not about being smart of dump, to keep in track and learning the beauty of technology but even to make bigger sense in a way this community helps each other the best way is to cut the Status crap at the end of your signature. Enjoy your Certifcates on the wall in your living room in a private manner you know what I mean? . But bragging about that stuff in your mail is not a way to earn respect. I really dislike to see this beef, I respect you and I'm proud for your achievements but for the future keep it low profile, or else it would only be used against you. And can somebody please help me with the no buffer space problem I posted yesterday ;-) Should I say more? Marcel de Vries, Swim Diploma, Firt Aid, And I don't give a F##K At 20:15 17-02-2002 +0100, Roy Sigurd Karlsbakk wrote: So what would you call direct DMA from the disk controller to the network adapter? Minus-one-copy? And even in the sendfile(2) case, data sometimes *is* copied in-core to satisfy alignment requirements etc. Stop using buzzwords just because they give you a woody. buzzwords or whatever - I really dont care. But can you please point at *one* application/system/whatever that is using HDD-NIC DMA? (and yes, even a Dr. Scient can be mistaken. Papers don't make you smart, you know - though I wouldn't expect someone who brags about being an MCSE and MCNE to understand that) er.. So - if you certify within a product, you'll probably become dumber? Grow up -- Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA Computers are like air conditioners. They stop working when you open Windows. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On Sun, Feb 17, 2002 at 11:05:04AM -0800, John Polstra wrote: Yes, I wrote one. Yes, it's proprietary. One obvious example is as part of a testbed for performance testing various kinds of network appliances. hmm, Well, so, forgive my bluntness, in a nutshell, there is no in-kernel (un-proprietory) HTTP Server. Also, as John said, it is very useful in some situations. As you visit the kHTTPD website at: http://www.fenrus.demon.nl/ You will see how much performance it offers, compared to userland web servers. I don't mean polluting the kernel with this kind of stuff, but I wouldn't mind having an in-kernel HTTP Accelerator (for static pages). Nothing so fancy, but, just to do its job, which it is good at: Providing static pages. Anyway, I don't mean that people have to make it. I know I talk to too much, and do less, :), but I will start a project at SourceForge, unless somebody has better options. ;) In a nutshell, it is better to have an in-kernel web server, as some people might like using it, even though it doesn't matter what we say, hehe :) Thanks, Regards, -- Hiten Pandya -- The JFS4BSD Project -- [EMAIL PROTECTED] msg31906/pgp0.pgp Description: PGP signature
Re: in-kernel HTTP Server for FreeBSD?
Thomas Hurst wrote: * Dag-Erling Smorgrav ([EMAIL PROTECTED]) wrote: Hiten Pandya [EMAIL PROTECTED] writes: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. Don't functions like FreeBSD's zero-copy sendfile() provide similar performance benefits without the massive security issues? If the remaining speed hog of switching to userspace to process the request itself is causing noticable bottlenecks, I think that's a sign you need more Pentium than the service needs moving to the kernel :) Not actually. If you have followed the khttpd work at all, then you'll know that Ingo has actually done some admirable work in it. Basically, the entire fast path, including the TCP stack, is small enough that the whole thing fits into the CPU cache itself, without any cache misses. It's actually very fast, considering (there are ways to make it faster that are actually more applicable to the FreeBSD approach, and which would take Linux a lot of work, even if they started from the QLinux base). For static content, such as the images and other content minus the main (non-static) page itself, it's quite a significant win for first hits, which comprise the majority of HTTP traffic. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Miguel Mendez wrote: As others have pointed out, it may have big hacker value, but no one would use it for anything serious. AFAIK there's no such thing for FreeBSD, but one thing I remember, is that once the Linux kernel incorporated the zero copy netowrking code, userland HTTP servicing like Tux saw it's performance increase on par with khttpd, so it seems that's not worth to run a http server in kernel space. First, livelock means that top end load on a khttpd will be higher than top end load on a user http, where there is so much interrupt traffic that the user space never gets to run. Actually, Alfred claims to be the first person to pre-load porn into the FreeBSD kernel, and his approach seriously predates the khttpd work. 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
* Terry Lambert [EMAIL PROTECTED] [020217 14:41] wrote: Miguel Mendez wrote: As others have pointed out, it may have big hacker value, but no one would use it for anything serious. AFAIK there's no such thing for FreeBSD, but one thing I remember, is that once the Linux kernel incorporated the zero copy netowrking code, userland HTTP servicing like Tux saw it's performance increase on par with khttpd, so it seems that's not worth to run a http server in kernel space. First, livelock means that top end load on a khttpd will be higher than top end load on a user http, where there is so much interrupt traffic that the user space never gets to run. Actually, Alfred claims to be the first person to pre-load porn into the FreeBSD kernel, and his approach seriously predates the khttpd work. 8-). Yes, preloading pron into the kernel does make it serve quite a bit faster. :) I basically made an API that allowed one to load and unload objects from the kernel then use a sendfile(2) like syscall to send them. I was servicing 5000 connections a second for 1x1 gifs or several thousand per second (saturating 100mbit) for larger images. I also used kqueue. The real problem is that most of the generic web servers available (as well as most commercial ones) just suck for handling IO and events. A well thought out design can give you quite a perf boost without needing to stick the _entire_ thing into the kernel. -- -Alfred Perlstein [[EMAIL PROTECTED]] 'Instead of asking why a piece of software is using 1970s technology, start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Dominic Marks wrote: On Sun, Feb 17, 2002 at 07:15:10PM +0100, Roy Sigurd Karlsbakk wrote: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? God forbid! Lots of hack value, sure, but not something you'd seriously consider for production use. well .. So let's turn the question upside-down, and ask Is there a web server or -accelerator for FreeBSD with similar performance as with khttpd or Tux? Zeus is the best overall. A lot of people also seem to have good performance from tHttpd and Boa. When you say 'similar performance' how many Mbit/s can Tux achieve (for static and dynamic content aggregated). The real figure of merit you should be looking for here is requests per second -- HTTP 1.1; if you measure 1.0 requests, you actually end up measuring connection per second, instead, which is not a real figure of merit. BTW: There's actually a couple of ways to get the average connections per second incredibly high -- on the order of 300,000 per second -- but they involve using techniques that are not really useful in real life, since over a certain level, your stall point moves to the number of simultaneous connections you can support vs. the 2 MSL for the TIME_WAIT. Any number over around 30,000 CPS is therefore a ramp-up number, and is not sustainable over a long duration. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Roy Sigurd Karlsbakk wrote: sendfile() isn't zero-copy, it's just two-less-copies. zero-copy means zero copy-operations within memory To an MCSE, maybe. strange ... It's interesting that Dr. Scient Paal Halvorsen that recently finished his thesis about the subject 'Memory buffering / caching in multimedia streaming systems'. This has the following description The INSTANCE operating system enhancements show great improvements for multimedia streaming. In this thesis, we want to look at buffering/caching/prefetching mechanisms suitable for multimedia streams combined with the INSTANCE zero-copy data path. See http://www.ifi.uio.no/~paalh/index2.html for more info hm Am I an idiot here or have someone else misunderstood? First, the URL reference you give is bad; it's one of those terrible web pages that has a stupid frame that prevents you getting the real URL. The publications URL is: http://www.ifi.uio.no/~paalh/publications/ Second, there's no publication there with this title; the closest one is where he's the junior name on the paper: Evaluation of a Zero-Copy Protocol Implementation. Third, this paper is bases on the idea that data is being replicated in main memory in lots of places; that doesn't really happen on FreeBSD, which uses a unified VM and buffer cache. Fourth, I think the reason he made the joke about MSCE's was that there are ways of doing true zero copy, using DMA directly between devices. There are a couple of people that have done this by, for example, rewriting the Tigon II firmware to permit page aligned buffers, and DMA'ing directly between disk and ethernet controllers. Technically, that's not zero copy, either, since there's a copy on the disk, and one in some controller memory. Zero copy usually means zero unnecessary copies; but what someone thinks of as necessary is really based on their bias towards an existing implementation. 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Hey, On Sun, Feb 17, 2002 at 10:28:32PM +, Hiten Pandya wrote: On Sun, Feb 17, 2002 at 11:05:04AM -0800, John Polstra wrote: Yes, I wrote one. Yes, it's proprietary. One obvious example is as part of a testbed for performance testing various kinds of network appliances. hmm, Well, so, forgive my bluntness, in a nutshell, there is no in-kernel (un-proprietory) HTTP Server. Also, as John said, it is very useful in some situations. As you visit the kHTTPD website at: http://www.fenrus.demon.nl/ You will see how much performance it offers, compared to userland web servers. I don't mean polluting the kernel with this kind of stuff, but I wouldn't mind having an in-kernel HTTP Accelerator (for static pages). Im sure nobody would resent even having the option, and I agree that it can be a useful tool in some circumstances. However for the vast majority of cases it is not appropriate and the lack of having one is no great loss. You could equally make an argument that any service should be offered in the kernel, which would be bad. Nothing so fancy, but, just to do its job, which it is good at: Providing static pages. Anyway, I don't mean that people have to make it. I know I talk to too much, and do less, :), but I will start a project at SourceForge, unless somebody has better options. ;) Good Luck In a nutshell, it is better to have an in-kernel web server, as some people might like using it, even though it doesn't matter what we say, hehe :) Can you make me and in-kernel text editor while your there? :) Thanks, Regards, -- Hiten Pandya -- The JFS4BSD Project -- [EMAIL PROTECTED] -- Dominic To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
At 14:59 17-2-2002 +, Dominic Marks wrote: On Sun, Feb 17, 2002 at 06:33:43AM -0800, Hiten Pandya wrote: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? Ive never seen or heard of one, at least not one available to the general public. Services should be executed in the user space, thats what its for. Cross-polluting these user/kernel environments is to be discouraged. I've heard of a netgraph module that does HTTP, but I'm not sure where unfortunately. Also this sort of thing can be useful for embedded devices. My EUR0.02 DocWilco To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Dag-Erling Smorgrav wrote: (and yes, even a Dr. Scient can be mistaken. Papers don't make you smart, you know - though I wouldn't expect someone who brags about being an MCSE and MCNE to understand that) Person A: How can you tell when someone is lying about having a PhD? Person B: I don't know, how? Person A: Aha! *You* don't have a PhD! -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
the netgraph ng_ksocket is the basis of how it was done. On Sun, 17 Feb 2002, Hiten Pandya wrote: On Sun, Feb 17, 2002 at 11:05:04AM -0800, John Polstra wrote: Yes, I wrote one. Yes, it's proprietary. One obvious example is as part of a testbed for performance testing various kinds of network appliances. hmm, Well, so, forgive my bluntness, in a nutshell, there is no in-kernel (un-proprietory) HTTP Server. Also, as John said, it is very useful in some situations. As you visit the kHTTPD website at: http://www.fenrus.demon.nl/ You will see how much performance it offers, compared to userland web servers. I don't mean polluting the kernel with this kind of stuff, but I wouldn't mind having an in-kernel HTTP Accelerator (for static pages). Nothing so fancy, but, just to do its job, which it is good at: Providing static pages. Anyway, I don't mean that people have to make it. I know I talk to too much, and do less, :), but I will start a project at SourceForge, unless somebody has better options. ;) In a nutshell, it is better to have an in-kernel web server, as some people might like using it, even though it doesn't matter what we say, hehe :) Thanks, Regards, -- Hiten Pandya -- The JFS4BSD Project -- [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
there is a netgraph module in standard systems (ng_ksocket) that does the socket part but youstill need to write the http node.. John Polstra's one is proprietary. On Mon, 18 Feb 2002, Rogier R. Mulhuijzen wrote: At 14:59 17-2-2002 +, Dominic Marks wrote: On Sun, Feb 17, 2002 at 06:33:43AM -0800, Hiten Pandya wrote: Is there any In-Kernel HTTP Server for FreeBSD, like there is kHTTPD for Linux? Ive never seen or heard of one, at least not one available to the general public. Services should be executed in the user space, thats what its for. Cross-polluting these user/kernel environments is to be discouraged. I've heard of a netgraph module that does HTTP, but I'm not sure where unfortunately. Also this sort of thing can be useful for embedded devices. My EUR0.02 DocWilco To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On Sun, 17 Feb 2002, Terry Lambert wrote: Fourth, I think the reason he made the joke about MSCE's was that there are ways of doing true zero copy, using DMA directly between devices. There are a couple of people that have done this by, for example, rewriting the Tigon II firmware to permit page aligned buffers, and DMA'ing directly between disk and ethernet controllers. Technically, that's not zero copy, either, since there's a copy on the disk, and one in some controller memory. Zero copy usually means zero unnecessary copies; but what someone thinks of as necessary is really based on their bias towards an existing implementation. At TFS we did this on BSD4.3, Mach 2.5 and FreeBSD(2.x) with proprietary protocols and proprieary network cards with lots of onboard RAM. We divided the RAM up into buffers and when we needed to send data we programmed the disk controllers to move the data directly to the network cards by DMA. This was on old slow busses but it gave us an order of magnitude advantage in bang per buck over the oposition.. Allowed us to do as much as them but on much cheaper hardware :-) Julian To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: er.. So - if you certify within a product, you'll probably become dumber? Getting an MCSE or an MCNE doesn't necessarily make one dumb - though some might ask if one couldn't find anything better to do with one's (employer's) time and money. Believing that it's worth more than the paper it's printed on, however, and bragging about it in an open-source forum, raises serious questions about one's intellectual acumen. Now, a CCNE, on the other hand... DES (neither of the above) -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Zero copy usually means zero unnecessary copies; but what someone thinks of as necessary is really based on their bias towards an existing implementation. zero copy these days has come to mean no copies that involve the CPU. In my experiance, raw memory bandwidth to DMA packets to/from main memory is not the bottleneck on modern hardware. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
David Greenman wrote: Zero copy usually means zero unnecessary copies; but what someone thinks of as necessary is really based on their bias towards an existing implementation. zero copy these days has come to mean no copies that involve the CPU. In my experiance, raw memory bandwidth to DMA packets to/from main memory is not the bottleneck on modern hardware. I'll agree with your experience. At this point, the limiting factor is PCI bandwith, at least for general purpose hardware. The AGP approach, where the main memory *is* the NIC memory, gets around this limitation, but doesn't kick the CPU out of the picture, even then. In practice, I've found that it is sometimes better to let the CPU do the copying, rather than the DMA engine on the device, since it's often faster at it. This assumes that the CPU is not otherwise engaged (e.g. doing cryptography), and there's enough spare cycles around the CPU can do the job faster. Moore's law and Intel really screw with the business model required to build special purpose network processors, don't they? 8-) 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
I'll agree with your experience. At this point, the limiting factor is PCI bandwith, at least for general purpose hardware. I haven't found PCI bandwidth to be a problem, either, at least when using gigabit ethernet NICs on 64bit and/or 66MHz PCI. When one writes an efficient HTTP server that takes a tiny amount of memory per process and uses sendfile() to crank out the bits, the bottleneck becomes the CPU for doing context switches, packet header creation, and TCP protocol processing. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On Sun, Feb 17, 2002 at 05:30:08PM -0800, David Greenman wrote: I'll agree with your experience. At this point, the limiting factor is PCI bandwith, at least for general purpose hardware. I haven't found PCI bandwidth to be a problem, either, at least when using gigabit ethernet NICs on 64bit and/or 66MHz PCI. When one writes an Correct, though Terry probably meant that general purpose hw (read: cheap motherboards) usually have 32bit/33MHz PCI buses, so they can easily become a bottleneck especially if they are shared with other peripherals such as disk controllers, video acquisition boards, or multiple ethernet boards. cheers luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
In case anyone is interested, I've just submitted an update for the boa port: http://www.freebsd.org/cgi/query-pr.cgi?pr=35065 On Sunday 17 February 2002 01:26 pm, Dag-Erling Smorgrav wrote: Roy Sigurd Karlsbakk [EMAIL PROTECTED] writes: well .. So let's turn the question upside-down, and ask Is there a web server or -accelerator for FreeBSD with similar performance as with khttpd or Tux? Have you tried thttpd or boa? DES -- Paul Marquis [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Luigi Rizzo wrote: On Sun, Feb 17, 2002 at 05:30:08PM -0800, David Greenman wrote: I'll agree with your experience. At this point, the limiting factor is PCI bandwith, at least for general purpose hardware. I haven't found PCI bandwidth to be a problem, either, at least when using gigabit ethernet NICs on 64bit and/or 66MHz PCI. When one writes an Correct, though Terry probably meant that general purpose hw (read: cheap motherboards) usually have 32bit/33MHz PCI buses, so they can easily become a bottleneck especially if they are shared with other peripherals such as disk controllers, video acquisition boards, or multiple ethernet boards. 32x33 would *definitely bottleneck, as you say, with 32x33 = 1Gbit max @ burst rate, which is not sustainable. Actually, I was talking about the Super Micro 2x64 bit PCI with two Tigon III cards, with TCP processing to completion at interrupt, the problem in doing fast forwarding of flows becomes the PCI bus bandwith, whose top end is 64x66 = 4.4Mbit/S burst. Almost all of that is eaten, pushing data between the cards. Now add to that the overhead of doing crypto processing on a Broadcom part, also over the PCI bus, and there goes all your bandwidth. The same goes for the Tyan Tiger II, also a 2x64PCI board. For 4 cards (you have to go Serverworks for more than 2x64 bit PCI slots), even a stupid router drowns at 4 cards, and barely handles 3 cards. For that to work, your front side bus has to be fairly fast, and even then, you spend all your time copying data around for routing tables, etc.. HP has 10Gbit copper parts today, and PCI-X is looking more like vaporware, and will only double 64x66 PCI performance, putting the cap at 8Gbit. HP 10 Gigbit parts: http://www.hp.com/rnd/news/0500.htm Or just search for procurve. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
On Mon, 2002-02-18 at 18:12, Terry Lambert wrote: HP has 10Gbit copper parts today, and PCI-X is looking more like vaporware, and will only double 64x66 PCI performance, putting the cap at 8Gbit. What about HyperTransport? (Not that I know anything about it, but those nice AMD guys keep mentioning it in sales garbage :) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Actually, I was talking about the Super Micro 2x64 bit PCI with two Tigon III cards, with TCP processing to completion at interrupt, the problem in doing fast forwarding of flows becomes the PCI bus bandwith, whose top end is 64x66 = Um, I thought we were talking about HTTP servers, not IP routers. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
Daniel O'Connor wrote: On Mon, 2002-02-18 at 18:12, Terry Lambert wrote: HP has 10Gbit copper parts today, and PCI-X is looking more like vaporware, and will only double 64x66 PCI performance, putting the cap at 8Gbit. What about HyperTransport? (Not that I know anything about it, but those nice AMD guys keep mentioning it in sales garbage :) They keep mentioning SledgeHammer, too... Have you seen silicon for either one of them yet? -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: in-kernel HTTP Server for FreeBSD?
David Greenman wrote: Actually, I was talking about the Super Micro 2x64 bit PCI with two Tigon III cards, with TCP processing to completion at interrupt, the problem in doing fast forwarding of flows becomes the PCI bus bandwith, whose top end is 64x66 = Um, I thought we were talking about HTTP servers, not IP routers. How about IP routers managed by HTTP? 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
