Feature Request: dav_fs impersonation
I would like to set up a shared file server using apache and mod_dav_fs to share files with windows and linux clients using the dav protocol over an SSL connection that is authenticated using client certificates. The one thing it seems I can not do is get apache to impersonate the user when accessing the filesystem so that the user will be using their permissions to access the filesystem and not 'www-data'. Does anyone have any thoughts on this?
Re: Deleting the async-dev branch
Having a thread block to page in data that isn't currently in the cache is fine, as long as other threads are able to handle all the other requests. What I would really like to see is support for zero copy sends without using sendfile(). Right now I believe that apache is calling sendfile() on small chunks of data at a time, so that it can make sure the transfer is not stalled. This tends to let the pipeline stall for short periods between each sendfile() call. Also sendfile() can not be used at all to send data that is not in a file on disk. For instance, the output of a cgi script, or data that has been encrypted to send over SSL. On NT you can direct the kernel to set the send buffer size to zero ( setsockopt I think it was ) and that will prevent it from making the user-kernel copy when you send. To keep the pipeline full you just have to overlap multiple async send requests. This allows you to get sendfile() performance without its drawbacks. I hope to one day see linux and apache support this. I have wondered though, what it would be like to be able to asynchronously page fault in the cache pages to send, rather than relying on the send() call to synchronously fault in the pages if needed. That would save threads from blocking on disk IO, which might be nice. Brian Pane wrote: The async write completion stuff that's now in the trunk does nonblocking network writes, and it uses sendfile to do zero-copy transfer from the filesystem cache to the network on platforms that support it, but it doesn't try to do file I/O asynchronously. Thus it's possible for the current code to block on a disk read while trying to do a nonblocking socket write. (The Event MPM compensates for this by allowing multiple threads to do sendfile calls on different sockets concurrently; if one thread happens to block on a physical disk read, it doesn't delay other connections.) Having a means of doing nonblocking filesystem I/O, via either aio or something like mincore checks, might be a nice addition. Brian
Re: Deleting the async-dev branch
Actually you have to use a peg revision, not -r. If you try -r to look at the deleted branch, svn tries to find the named directory in the HEAD revision, then look up it's state as it existed in rev x. This will fail since it does not exist in the head revision, so instead you have to do something like: svn list http://repos/svn/[EMAIL PROTECTED] svn list -r x http://repos/svn/deleted fails because deleted does not exist anymore. By the way, what exactly is the async-dev branch and what are the future plans for it? I actually have spent this entire weekend doing research on the current state of support of async IO in linux and modified the dd program to use overlapped async IO when you pass it the iflag=direct option. It turned out that glibc's posix aio library simply performs sync IO in a background thread, so I ended up hacking together a basic aio implementation that directly uses the kernel's aio support. I'm still testing it but it looks like it yielded substantial cpu savings and throughput increases by keeping the drive directly reading into the userspace buffers. DMA transfers sure beat memcpy()s. I have been a big proponent of zero copy async IO for several years now so I am very interested in projects to further it's support and use. Max Bowsher wrote: Definitely not! You can always look at any deleted thing by looking back into the history with an appropriate svn --revision (-r) argument. The only significant effect of deleting a branch is to unequivocally indicate to everyone that a branch's sole reason for existence is now historical record, rather than active development. Max.
Re: pgp trust for https?
Exactly. It doesn't matter where you live and what government you live under, they issue you some form(s) of identification. A digital certificate is simply a digital form of identification. Why should the government keep using ink and paper documents for identification when digital ones are much better? I for one will be glad when I can just get a certificate in my email instead of waiting in line for 5 hours at the department of motor vehicles to renew my driver's license. Boyle Owen wrote: -Original Message- From: Nick Kew [mailto:[EMAIL PROTECTED] ... Personally, I feel this role belongs in the government. Any particular government? A few years ago I'd probably have agreed. With the most blatently corrupt government in living memory, that has less appeal. At first I thought this statment was inappropriate on an international list where we don't know where anybody lives. But then I realised it applies equally well to every government on the planet... Except, maybe, Iceland? Rgds, Owen Boyle Disclaimer: Any disclaimer attached to this message may be ignored.
Re: pgp trust for https?
Nick Kew wrote: Why would anyone have to do that? I'll trust a server as much as I trust the PGP key of the person who signed it. That's the same as trusting an httpd download because it's signed by someone whose key I trust. The question then is who is going to sign? You seem to be sugesting that a server accept signatures from anyone and everyone, and you would only trust the server if someone that you know and trust had already decided to trust the server and sign their certificate. That still leaves you in a position of either deciding to implicitly trust the site and sign it's certificate, which then causes all of your friends to trust it, or trusting the opinion of your friend who already decided ( based on what exactly? ) to trust the site and sign their cert. That doesn't provide much in the way of security, and is impossible to maintain -- a site can't accept a million signatures on their certificate from everyone who feels like signing it. It's usually signed by verispam, who make a habit of engaging in some very nasty business practices, from spamming to holding the 'net to ransom. They also bought the main competitor (thawte), leaving us short of competition amongst those widely recognised by browsers. With PGP it's my own trust, not theirs. You are quite free to set up your own root CA and encorage others to trust you. You are also free to decide to NOT trust certificates signed by verispam. Personally, I feel this role belongs in the government. That's where you get your birth certificate, driver's license, social security card, and other forms of 'official' ID. They may as well get rid of all the paper ID and just start issuing digital certificates. I seldom use pgp for email (and I hate it when people sign messages posted to a list like this). But I always use it to verify software I download from the 'net. And, unlike https, it tells me every time whether or not *I* trust the digital signature. How do you decide that such a signature is trustworthy and valid? You either have to know about their public key a priori, or know ( and trust ) another one that signed theirs, otherwise, you're just guessing that you can trust it.
Re: pgp trust for https?
Nick Kew wrote: Huh? The same person who installs the cert now. It's just a different signature. And for those who want a certificate authority, have such authorities (the more the better) sign *their* PGP keys. >From whomsoever is responsible for it. Maybe even more than one individual, in the case of an org with lots of techies. I'll sign my server. Same as I'll sign an httpd tarball if I roll one for public consumption. You sign your server. Where's the problem? I don't want to get involved in something where I have nothing substantial and new to contribute. "Encourage others to trust you" is a marketing job of which I would be totally incapable. Now you seem to be talking about self signed certificates. There are sites out there that use them, but the problem is, they provide no authentication. Sure, they allow encrypted communications, and if you save the certificate, then you can be sure that you are connecting to the same server the next time ( preventing man in the middle attacks ) but there's nothing to prevent a man in the middle the first time. Another example you mentioned is signing tarballs you make. Well if your certificate is not signed by a trusted third party, such as verisign or whomever, then how am I to know that it was YOU who signed it and not some random guy claiming to be you? Anyone can make a keypair, the signature of the CA is what authenticates it. Any particular government? A few years ago I'd probably have agreed. With the most blatently corrupt government in living memory, that has less appeal. True, but if things are that bad then you have bigger problems to worry about. As long as you still trust them to issue forms of paper identification, then digital ones should be no different. Why get a paper ID from the government to show to a CA to prove you are you, so they can sign your cert, rather than just have the gov't sign it directly? Sure. I do trust my own key, and those of quite a number of other people, including, for example, most of those with whom I would expect to share an SVN repository for development work. That's the kind of application where a PGP-signed server key is a clear winner. How so? Again, in a small group of people who know each other in real life and have a secure method of exchanting certificates, that works fine, but this breaks down when you have a larger group of people who do not know each other in real life and can swap their certs on disks when they see each other. And when my browser indicates that I don't trust a key, I can investigate in detail by fetching the public key and its signature(s), and make whatever other checks I see fit. Exactly the same as when I download a package from the 'net. Yes... and you can do that with the x.509 certificates that are used for https/SSL, so what's pgp needed for?
Re: pgp trust for https?
The big reason that comes to my mind is that users don't want to have to implicitly trust the server from the start, then register on the site by uploading their own key before secure communications can begin. The big advantage of a public certificate infrastructure is that the rest of us can trust someone we have never met before ( i.e. an https server you have never visited before ) because they present us with a certificate that is signed by a trusted third party. The server proves to the client that it is who it says it is, and the client can optionally prove its identity to the server with a client certificate. Either one allows all communications to be encrypted and you don't have to exchange private information first to do it. Personally, I don't understand the need for pgp. You can sign and encrypt email or IPsec just fine using x.509 certificates. Nick Kew wrote: We have grown accustomed to two separate trust mechanisms on the 'net; server certs signed by some authority, or the PGP web of trust. I would like to be able to use PGP trust over the web. That would mean (something like) installing a certificate on the server, and signing it with my PGP key. My browser, having access to and trust in, my public key, will then trust the server. No need for any dealings with verispam. Is there any technical reason this shouldn't happen?
Re: async write completion prototype
Nicely done. Have you done any benchmarking to see if this improved performance as one would expect? Would it be much more work to use true async IO instead of non blocking IO and polling? What about doing the same for reads, as well as writes? Brian Pane wrote: With the batch of commits I did this weekend, the Event MPM in the async-dev Subversion branch now does write completion in a nonblocking manner. Once an entire response has been generated and passed to the output filter chain, the MPM's poller/listener thread watches the connection for writability events. When the connection becomes writable, the poller thread sends it to one of the worker threads, which writes some more output. At this point, the event-handling code is ready for testing and review by other developers. The main changes on the async-dev branch (compared to the 2.3-dev trunk) are: 1. ap_core_output_filter: rewrite to do nonblocking writes whenever possible. 2. core, http module, and mod_logio: removed the generation of flush buckets where possible. 3. request cleanup and logging: the logger phase and subsequent destruction of the request's pool are now triggered by the destruction of an End-Of-Request bucket in the core output filter. 4. event MPM: asynchronous handling of CONN_STATE_WRITE_COMPLETION. There are several more things that need to be fixed in order to make the asynchronous write completion useful in a production release of httpd-2.x: - The main pollset in the Event MPM currently is sized to hold up to one socket descriptor per worker thread. With asynchronous keepalives and write completion, the pollset should accommodate many descriptors per thread. - The logic for starting more child processes, which Event inherited from Worker, is based on assumptions about the number of concurrent connections being equal to the number of threads. These assumptions aren't valid for a multiple-connections-per-thread MPM. - Similarly, there may be some changes needed in the flow control logic that the listener thread uses to decide whether it can do an accept. - The scoreboard probably needs a redesign. - It may be valuable to have a separate thread pool to run handlers that do arbitrarily lengthy processing, such as mod_perl and mod_php. Brian
Re: async write completion prototype
On what OS? Linux? NT supports async IO on sockets rather nicely, as does FreeBSD iirc. Paul Querna wrote: Phillip Susi wrote: Nicely done. Have you done any benchmarking to see if this improved performance as one would expect? Would it be much more work to use true async IO instead of non blocking IO and polling? What about doing the same for reads, as well as writes? All current async_io methods require you to read the data off disk, ie there is no async_sendfile() Reads are much harder in the current httpd. There are several core functions that would need to be rewritten first. -Paul
Re: async write completion prototype
On NT you can set the kernel buffer size on the socket to 0 ( with setockopt() or ioctlsocket()? ), and the NIC can DMA directly from the user buffers to send rather than copy to kernel space. This of course, requires that you keep more than one pending async operation so the nic allways has a buffer availible so it can keep the line saturated. If you memory map the source file from the disk, then zero copy IO can be done entirely from user space. Is this optimization not availible on linux or freebsd? As an alternative, you can bypass the cache and do direct async IO to the disk with zero copies. IIRC, this is supported on linux with the O_DIRECT flag. Doing this though, means that you will need to handle caching yourself, which might not be such a good idea. Does Linux not support O_DIRECT on sockets? By using this technique I have been able to achieve TransmitFile() performance levels entirely from user space, without any of the drawbacks of TransmitFile(). Specifically virtually zero cpu time is needed to saturate multiple 100 MBps links, pushing 11,820 KB/s, progress is known the entire time and can be canceled at any time, and a small handfull of threads can service thousonds of clients. Paul Querna wrote: Phillip Susi wrote: On what OS? Linux? NT supports async IO on sockets rather nicely, as does FreeBSD iirc. The event MPM doesn't run on NT at all, only Unixes. Yes, FreeBSD (and linux) support async_write().. But this requires that you read the file off of disk, and into a buffer, and then copy it back into the kernel. With a non-blocking sendfile, we can avoid all the data copying (aka 'zero copy'), and let the kernel do everything itself. There is currently no such thing as 'async sendfile', which would be the perfect solution for this use case. There have been various people mentioning it as an idea, but no one has gone out and done it. -Paul
Re: async write completion prototype
Non blocking is not async IO. It is not really possible to perform zero copy IO with non blocking IO semantics, you must have full async IO to issue multiple pending requests. Brian Akins wrote: Phillip Susi wrote: As an alternative, you can bypass the cache and do direct async IO to the disk with zero copies. IIRC, this is supported on linux with the O_DIRECT flag. Doing this though, means that you will need to handle caching yourself, which might not be such a good idea. Does Linux not support O_DIRECT on sockets? Can you not just set the socket to non-blocking using O_NONBLOCK?
Re: Bandwidth Limit on Windows
Probably has to do with sendfile, always turn it off on windows. I was seeing low throughput as well until I turned off sendfile, then over a 100 MBps network I got over 11,000 KB/s. Apparently apache has a rather horrible emulation of sendfile on windows for some reason or another. Ivan Barrera A. wrote: Hi All. I'm working on a bandwidth module for apache. My question is as follows : Recently, a user asked me why on windows, he can get higher bandwidth only with mod_bw. Without it, he can get about 1.6MB/s. Usind the mod, and setting unlimited , he gets ~9MB/s. I'm not on my develop system, so i haven't tested. The question is , why this happens ? does it have relation with the EnableMMAP and Senfile options ? Thanks.
Re: Bandwidth Limit on Windows
When I asked about this a month or two ago, someone explained that Apache uses TransmitFile() to implement sendfile but in a weird way that makes it really, really slow. Disabling sendfile in the apache config, and just using the mmap module gives the best throughput on windows. I don't know if apache mmaps the entire file if it is large, but if it is very large, I don't see how it can as you only have a limited amount of virtual memory to work with. Unless you run out of virtual memory ( i.e. lots of mapped files, or a single file that is several gigs ) then mapping the entire file isn't really a problem. William A. Rowe, Jr. wrote: Phillip Susi wrote: Probably has to do with sendfile, always turn it off on windows. I was seeing low throughput as well until I turned off sendfile, then over a 100 MBps network I got over 11,000 KB/s. Apparently apache has a rather horrible emulation of sendfile on windows for some reason or another. Sendfile != MMap. In the case of Sendfile, it's using the Win32 API TransmitFile(). In the case of MMap, the file is mapped into virtual memory, and loaded into physical memory. Apache prefers Sendfile, if both are available, IIRC. However, the way Apache/APR MMAPs the entire file is quite probably sub-optimal for some resources. Bill
Re: Bandwidth Limit on Windows
William A. Rowe, Jr. wrote: It's called swap-death. Forcing the system to begin paging both mapped files (which aren't swapped to the swap file since their contents are unchanged, e.g. your programs and dll files' code pages) and ram (which is thrown at the swap file) slows it down considerably. Bill This is not true. Mapping a full view of a large file does not cause any data to actually be read, and hence, any memory to be used. Data is only read from disk as the pages are faulted in on access. William A. Rowe, Jr. wrote: Yup. It's definately worth considering your alternatives. It's also worth looking at what socket-level timeouts exist, or if we can start plugging in some higher-level timeout within the MPM. When a process does nothing in certain phases of the request cycle after some timeout, simply close the socket from underneath APR. So how to figure out if we are making progress? I'm unsure and this all deserves another look at the current state of the Win32 sockets API, I haven't fallen down that well in a good long time :) Of course async would be wonderful, at some point, as apache 3.0 starts to draw interest. Bill I agree, trying to solve this kind of DoS attack in the httpd is kind of futile. Also async IO is the way to go imho, and I've been spending some spare time lately trying to figure out how hard it would be to modify apache to take advantage of async IO, which would allow for the kind of zero copy IO that sendfile and TransmitFile were created for, without the problems they cause. It also allows a small number of threads to scale to serving a very large number of clients.
Re: ap_process_connection
Except that it isn't REALLY defined there. The use of all these macros is giving me a headache. There are a number of papers out there on why this kind of macro (ab)use is bad, mainly because it makes it impossible to understand the code. server/connection.c has this: AP_IMPLEMENT_HOOK_RUN_FIRST(int,process_connection,(conn_rec *c),(c),DECLINED) Well what the heck does that do? There's no function body defined there. I'm guessing that macro does some tricks to achieve something like polymorphism in C, binding an object to a set of functions that should be called in a given event. I can't see what function is being bound to the process_connection event though. When I looked up the definition of this macro, it is itself defined using another macro. My eyes have now gone crossed. Maybe I'll try to understand these evil macros later. C.G. Lee wrote: Where is this function implemented? I can't seem to find it. When I was exploring the Apache2 source code, I couldn't understand the ap_process_connection(). Why is it there? What does it doing?? I don't know what you're doing, but the function ap_process_connection is in the server/connection.c.
ap_process_connection
Where is this function implemented? I can't seem to find it. I found this comment in a header: Note: In the future it will be possible for ap_process_connection to return to the MPM prior to finishing the entire connection; and the MPM will proceed with asynchronous handling for the connection; in the future the MPM may call ap_process_connection again -- but does not guarantee it will occur on the same thread as the first call. And I was wondering if this has been done yet, but I can't locate the implementation of ap_process_connection to find out. I'm trying to see if I can modify it to make asynchronous socket IO calls and return to the MPM before the entire conversation is complete. That way the winnt_mpm can use a small pool of worker threads to perform zero copy async IO using the IO completion port and become more fast, efficient, and scalable than it currently is.
SSL downloads faster than non SSL?
I decided to do some informal benchmark comparisons between using windows SMBFS and apache/webdav for file transfers. I ended up finding that apache is actually faster at sending files over an SSL connection than a plain connection. I downloaded a ~600 meg test file from the server using windows explorer webfolders, IE, and firefox. Firefox downloads the file in the 4000-5000 KB/s range when using an SSL connection. IE gets over 10,000 KB/s downloading over the secure connection. Both only are able to download at 300-600 KB/s using the non SSL connection though. This is, of course, all done over a 100 Mbps ethernet network that is minimally loaded, and I repeated the test a few times, clearing the browsers caches each time. Needless to say, this is very strange. My client machine is running windows 2000 pro, and the server is running windows 2003 server, with apache 2.0.52 installed. Does anyone have any ideas to possibly explain and correct these results?
Re: Single-threaded/non-blocking frontend and multi-threaded/processed backend
Why have a separate 'frontend' and 'backend'? One of these days when I get some spare time, I plan on rewriting mpm_winnt to use a small pool of threads which will use unbuffered zero copy overlapped IO and an IO completion port as the notification mechanism. The pool of worker threads will wait on the IOCP for incoming data and as data comes in on any of the connected sockets, it wakes one thread to process the data. The MPM would then pass the data on down the handler/filter chain, and it is currently my understanding that most of the time, the filters in the chain don't make other blocking system calls, but rather end up calling back into the apr/mpm to get more data, at which point the mpm would be free to handle IO on other sockets. This system would allow fast and efficient IO on the level of TransmitFile(), only not restricted to sending unmodified on disk files. I once wrote a simple FTP server using this mechanism and it was able to stream a large file over a fast ethernet network at 11,820 KB/s using virtually zero cpu time. I believe that the apache2 bucket brigade design is nearly ideal for this system as the various filters and handlers can produce and/or modify data as they see fit, and it will still be sent and received over the network without the need for the user/kernel buffer copies. It would also allow a small pool of worker threads to serve tens or even hundreds of thousands of clients. I believe that BSD has a similar system called kqueue and posix aio. To my knowledge, Linux has no equivalent. Olaf van der Spek wrote: On 4/13/05, Paul Querna [EMAIL PROTECTED] wrote: Olaf van der Spek wrote: Hi, I originally posted this feature request to the bug tracker, but Joe Orton suggested it'd post it here instead, so here it is. I'd like to see a new 'MPM' that basically works like this: A single or few-threaded, non-blocking frontend that accepts incoming connections, receives requests and sends responses. A multi-threaded or multi-processed backend that processes requests, generates responses and passes the responses to the frontend for sending to the client. The advantage of this scheme would be that you can handle a (much) bigger number of clients/persistent connections with less resources. The event MPM is a step in the right direction, but not 'good' enough. A scenario with multiple backends may be possible to, for example one running as user A, one as user B, etc. Or one multi-threaded backend for static content and one multi-processed backend for dynamic content (PHP). http://issues.apache.org/bugzilla/show_bug.cgi?id=32243 I agree with just about everything you said, at least in concept. I am willing to help out, but I don't have the time or resources to head up this type of project right now. If you do the initial work, I would love to help get such an MPM into the mainline tree. Wouldn't it be possible to start this 'transition' by writing a module that forwards the request to a separate backend process? That way, you could safely run the threaded MPM and still use PHP in a safe way.
Re: Post-2.2 Thoughts
Paul Querna wrote: So, here is my short-list-made-up-this-afternoon of things I would like to look at doing after 2.2 is branched/released as GA. I welcome additions too. 1) Eliminate the HTTP in HTTPD. I would like to be able to compile httpd with --disable-http. Currently the 'http core' is half mangled with the 'core core'. I like this idea. I think one of the best and most important things that apache is doing these days is trying to migrate away from being a web server and towards a generic server platform. Eventually I would like to be able to run apache to serve web content, email ( smtp as well as pop3 ), and subversion repositories and other file shares ( possible now ). 2) Add a major Protocol Module. I would like to add a major protocol module, besides http, into the mainline distribution. I believe adding a FTP Server Module would be the best one to start with. Many of the paradigms are the same, but it is different enough to help with Goal #1, eliminating the dependence on http. Sounds good. FTP would offer the path of least resistance and help to eliminate the dependence on http. I do not however, think that it has long term use in its own right, because ftp is being replaced with http/webdav. 3) Asynchronous Connection Level Filters. This one needs much more thought, but the basic goal is to allow the Event MPM to do all of the initial HTTP parsing without a worker. (eg, reading the headers). Once the headers are read, it would pass the request off to a worker. I don't think trying to make the different Hooks or Handlers Asynchronous will be that helpful, but Connection Level Filters would need a re-design to handle this idea. I don't personally have a clear picture how this would all work yet, and really want the input of everyone else. I'm not quite sure what you mean here, but that is probably because I'm new to hacking on apache, so forgive me if this is a stupid question. As I understand it now, the MPM receives data from the socket and places it into a bucket, attaches the bucket to the brigade, and calls the lowest level filter, passing it the bucket. Each filter in the chain then can modify the brigade, and can pass the bucket to the next filter, or possibly call back down the chain to the mpm to get more data. Eventually the brigade makes it to the handler which forms a reply and passes it back down the chain to the mpm to send out on the socket. Is this about correct? Where do workers fit in? 4) Configuration. I would like to look at enabling different configuration systems. A major change would be to allow more runtime configuration changes. Others have mentioned using XML -- Im not sold on it, and I don't think XML should be an exclusive method, but perhaps an alternative file format. I don't think I can do all of these myself. If we did all of them, it could be a '3.0'. I am not sure if the project is ready for a 3.0 yet, and in some ways, I think after 2.2 we should pick a few key things and target them for '2.4'. But, it doesn't matter, this is open source, not some commercial product :) These are the types of things I would love to talk about at the Hackathon before ApacheCon this summer. I am seeding them now, in the hope that other minds will start hacking on new ideas for httpd. -Paul
