Modules Communicating
Hi! I am writing my own module that handles all incoming requests. In some cases, I want this request to be handled by mod_python which I installed. In these cases, my module should change the requested file extension to .py, which is configured to be handled by mod_python. AddHandler mod_python .py I have tow questions: 1. How can I set the modules order, so my module will handle the request first, change its url, and then mod_python will handle it? 2. Which field in the request_rec I should change so it will take effect? I tried to change the URL key in subprocess_env table, but the request was not handled by mod_python. apr_table_set(r-subprocess_env, URL, strUriWithPyAtTheEnd); Thanks
Re: Modules Communicating
On 2012-08-22 09:31, Adi Selitzky wrote:
Hi!
I am writing my own module that handles all incoming requests. In some cases, I
want this request to be handled by mod_python which I installed. In these
cases, my module should change the requested file extension to .py, which is
configured to be handled by mod_python.
AddHandler mod_python .py
I have tow questions:
1. How can I set the modules order, so my module will handle the request first,
change its url, and then mod_python will handle it?
const char *succ[] = {mod_python.c, NULL};
ap_hook_handler(your_handler, NULL, succ, APR_HOOK_FIRST);
2. Which field in the request_rec I should change so it will take effect? I
tried to change the URL key in subprocess_env table, but the request was not
handled by mod_python.
AddHandler mod_python .py simply sets r-handler to mod_python if
r-filename contains .py. So I guess that the python processing is
triggered by r-handler being mod_python.
So you can try setting r-handler = mod_python and then return
DECLINED from your handler and forget about appending .py.
Sorin
Re: Limitations of mod_dbd - single server per vhost
On 21 Aug 2012, at 20:27, Mikhail T. wrote: From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example. That is by design: it is expected to meet most needs. If you wants more generality, go ahead and hack it. Also, the syntax of mod_rewrite's dbd/fastdbd maps doesn't even allow to specify the DB-server, Indeed, dbd support in rewritemap was only ever a quick hack. Some of us think mod_rewrite is long overdue retirement. implying, once again, there can only be one in a given context (entire server or virtual host). The missing capability could be useful to someone, who would want, for example, to check a local SQLite-database for data and, failing to find anything, attempt a query against a remote MySQL or Sybase server. Other enterprises may want to talk to multiple remote servers (potentially using different drivers). Well, you could hack it into mod_rewrite, but isn't that traveling further down the road of using mod_rewrite as a pseudo- programming language? The provision of If expr syntax and of mod_lua is supposed to give you better tools than single-IF (RewriteCond) and GOTO (RewriteRule). Also bear in mind that without a query cacheing regime comparable to what mod_authn_socache provides for authentication, that kind of complexity would most likely mean rather a steep overhead. I would've filed an enhancement-request, Feel free to file an enhancement request. Feel free to submit a patch. -- Nick Kew
Re: Limitations of mod_dbd - single server per vhost
On 22 Aug 2012, at 10:25 AM, Nick Kew wrote: From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example. That is by design: it is expected to meet most needs. If you wants more generality, go ahead and hack it. This is one of the things I'd like see minimised in the server implementation, the inclusion of arbitrary restrictions that later have to be lifted. I went through the proxy and cache implementations for httpd v2.4 and removed all the per-virtual-host restrictions from all directives that didn't require it, as too often I was approached saying we can't use the cache because feature X can only be set server wide and that breaks site Y. Ideally directives should be per-directory or per-location, unless there is a concrete reason otherwise. My rusty memory tells me that someone submitted a patch to fix this a while ago? Regards, Graham -- smime.p7s Description: S/MIME cryptographic signature
Ideas for an output filter for mod_lua
Hi dev@, I've been wondering (and tinkering with) the idea of creating output filters through mod_lua. If this has already been discussed, it was before my time here, so please forgive any redundant ideas. Essentially, what I'd like to do is be able to do the following: LuaOutputFilter myTestFilter /www/filter.lua filter_handle FilesMatch \.txt$ SetOutputFilter myTestFilter /FilesMatch and then, through coroutines or some such, make the following filter in Lua: -- [[ A simple filter mechanism ]] -- function filter_handle(r) coroutine.yield() -- Yield and wait for buckets while (buffer) do -- buckets are sent through the global var 'buffer' -- and set to 'nil' when there are no more buckets local escaped = r:escape_html(buffer) coroutine.yield(escaped) -- pass on the bucket end return end I've already made and tested a prototype of this, and it seems like something that could be of use to the public. So, any feedback, comments, thoughts on this? With regards, Daniel.
Re: Limitations of mod_dbd - single server per vhost
On 22 Aug 2012, at 11:43, Graham Leggett wrote: On 22 Aug 2012, at 10:25 AM, Nick Kew wrote: From reading the module's documentation, it seems, a single vhost can only use one DB-server (and only with one set of credentials): only a single DBDriver and DBDParams is possible, for example. That is by design: it is expected to meet most needs. If you wants more generality, go ahead and hack it. This is one of the things I'd like see minimised in the server implementation, the inclusion of arbitrary restrictions that later have to be lifted. My words you quoted are somewhat oversimplified. The design didn't say we'll only allow one backend ..., it was just a case of not introducing extra complexity. The implementation needs object persistence, of the kind we have in the per-server configuration but not per-directory. That was the original design constraint. I went through the proxy and cache implementations for httpd v2.4 and removed all the per-virtual-host restrictions from all directives that didn't require it, as too often I was approached saying we can't use the cache because feature X can only be set server wide and that breaks site Y. Ideally directives should be per-directory or per-location, unless there is a concrete reason otherwise. Indeed, I seem to recollect doing some of that for ProxyPass family right back in the days of 2.0, when it was indeed a bug[1]! But that's a different kettle of fish. My rusty memory tells me that someone submitted a patch to fix this a while ago? Now that you mention it, wasn't something committed? I think that whatever we have is either there in the code or excluded because there were more problems with it than anyone was prepared to fix. But ICBW. [1] starting with r231355 and http://marc.info/?l=apache-httpd-devm=110726027118798w=2 -- Nick Kew
Re: Ideas for an output filter for mod_lua
On 22 Aug 2012, at 12:00, Daniel Gruno wrote: So, any feedback, comments, thoughts on this? Basic concept looks fine. I guess we'd need more detail to say any more about it. Is the implementation 'clean' or does it involve hacks to core? If what you have is pure module then I'd see no reason not to drop mod_lua_filter (or is it mod_filter_lua?) into trunk on CTR. Meanwhile, some random thoughts: Is there a deep reason to limit it to output filters, or is that just what you happen to have implemented? Would your concept meaningfully generalise beyond application-level filters? -- Nick Kew
Re: [Vote] httpd 2.2.23 release
All looks fine on Windows. Steffen -Original Message- From: William A. Rowe Jr. Sent: Tuesday, August 21, 2012 9:25 PM Newsgroups: gmane.comp.apache.devel To: dev@httpd.apache.org Subject: [Vote] httpd 2.2.23 release Candidate binaries are available from http://httpd.apache.org/dev/dist/ - these do not yet constitute ASF releases. Win32 specific artifacts (x86 binary distribution and -win32-src.zip) will follow sometime later this evening. Feedback and edits to the draft announcement are greatly appreciated, since we need to better position 2.2 as 'just maintenance' and we can drop some of the 'shiny new' bits of that Announcement text, given that it is old hat now, and they should be reading the 2.4 Announce (which this document now links to). So, for your consideration, a vote... +/-1 [ ] Release httpd 2.2.23 as GA
Re: Limitations of mod_dbd - single server per vhost
On 22.08.2012 07:24, Nick Kew wrote: The implementation needs object persistence, of the kind we have in the per-server configuration but not per-directory. That was the original design constraint. I'd say, my complaint was a little misunderstood. It is not, that I wish to see separate DBD-configurations per location/directory, though it may be a good thing too. It is that I want to be able to use more than one server per vhost. This collection of servers can be maintained in the persistent object you had in mind, where the single server is maintained now. Perhaps, the DBD-statements should begin accepting one more (optional) argument -- the server's label: DBDriver first freetds DBDriver second mysql DBDParamsfirst DBDParamssecond ... Then RewriteMaps can then changed to operate thus: RewriteMap first dbd:first:SELECT RewriteMap second dbd:second:SELECT I don't think, I'm the best person to implement this -- whoever quick-hacked the dbd-maps in recently is, probably, better positioned to do this ;-) My employer's needs here are satisfied for now -- we decided to use text-maps for local cache and thus need only a single dbd-map for talking to database (if the local cache fails). But I can give it a try, if no one else does. Yours, -mi P.S. BTW, I fixed the FreeTDS/Sybase driver https://issues.apache.org/bugzilla/show_bug.cgi?id=53666 for APR/APU somewhat -- it has been absolutely broken and unusable for years, no kidding -- but don't see any movement on the ticket... Does anyone care?
Re: [ANNOUNCEMENT] Apache HTTP Server 2.4.3 Released
On 08/21/2012 06:55 AM, Jim Jagielski wrote: NOTE to Windows users: The issues with AcceptFilter None replacing Win32DisableAcceptEx appears to have resolved starting with version 2.4.3 make Apache httpd 2.4.x suitable for Windows servers. I noticed the following blurb from previous announcements went away: There is not yet a Windows binary distribution of httpd 2.4, but this is expected to be remedied soon as various dependencies graduate from beta to GA. Does this mean that binary distribution will be available soon? Will there also be a separate source distribution with the proper crlf for those tools that are stupid and don't work right? Also, I notice that the Unix build instructions were updated to deal with no longer bundling various dependencies (apr, apr-util, pcre) but the Windows build instructions: http://httpd.apache.org/docs/2.4/platform/win_compiling.html don't really discuss this. Is this something that will be updated, or will a separate windows source distribution still contain the dependencies under srclib? Thanks, Andy
Re: Limitations of mod_dbd - single server per vhost
On Wed, 22 Aug 2012 09:21:16 -0400 Mikhail T. mi+t...@aldan.algebra.com wrote: On 22.08.2012 07:24, Nick Kew wrote: The implementation needs object persistence, of the kind we have in the per-server configuration but not per-directory. That was the original design constraint. I'd say, my complaint was a little misunderstood. It is not, that I wish to see separate DBD-configurations per location/directory, though it may be a good thing too. Yes, the discussion got sidetracked. Probably my fault! Then RewriteMaps can then changed to operate thus: RewriteMapfirst dbd:first:SELECT RewriteMapsecond dbd:second:SELECT Indeed that would follow if we had multiple backends. Indeed, it could go further and capture multiple return fields, either by name or in the manner of regexp backreferences. This kind of incremental advance happens when someone feels motivated to hack it. I don't think, I'm the best person to implement this -- whoever quick-hacked the dbd-maps in recently is, probably, better positioned to do this ;-) It wasn't actually very recent: https://svn.eu.apache.org/viewvc?view=revisionrevision=454683 P.S. BTW, I fixed the FreeTDS/Sybase driver https://issues.apache.org/bugzilla/show_bug.cgi?id=53666 for APR/APU somewhat -- it has been absolutely broken and unusable for years, no kidding -- but don't see any movement on the ticket... Does anyone care? To be fair, I think that driver carried some health warnings! I think I've pointed a few people at the ODBC driver as an alternative. Do you have a strong reason to use FreeTDS rather than ODBC? One question here may be, does anyone have the platforms to test-drive your patch? It seems to do rather more than just fix a simple bug: you've removed the untainting code which was part of protecting against injection attacks. Have you implemented prepared statements properly for all backends? It worked for its original users at the time it was written, but I guess it's withered on the vine since then. Maybe it's some stumbling on some difference between the different FreeTDS backends: you might even be the first person to report trying it with Sybase! -- Nick Kew
Fixing apr_dbd_freetds (Re: Limitations of mod_dbd - single server per vhost)
Perhaps, this discussion should be happening on theticket https://issues.apache.org/bugzilla/show_bug.cgi?id=53666 itself? Oh, well :) Comments inline: On 22.08.2012 10:47, Nick Kew wrote: I think I've pointed a few people at the ODBC driver as an alternative. Do you have a strong reason to use FreeTDS rather than ODBC? Yes, ODBC is yet another layer of abstraction. I'd understand, if apr_dbd.c used purely ODBC, leaving the ODBC implementation to deal with the backends' native drivers. But if apr_dbd offers native drivers of its own, better to use those, if possible. That's my strong reason :-) One question here may be, does anyone have the platforms to test-drive your patch? I would not know... But, in my opinion, even without getting tested by anyone else, my patch is still a vast improvement over the current situation... It seems to do rather more than just fix a simple bug: That's because the bug was not simple -- the driver remained broken ever since the apr_dbd.c was changed to do the parsing of query-templates centrally instead of delegating the job to each driver. Other drivers got updated back then, but not the FreeTDS one :-( It still tried to do its own parsing, and was failing... you've removed the untainting code which was part of protecting against injection attacks. I don't think, the untainting was ever properly implemented (you can't do it right without prepared statements) and I don't believe, it is the driver's job to do it anyway. None of the other drivers do it either -- though they rely on the respective client-libraries. Sybase/FreeTDS client does not offer such checks -- so be it... The untainting was costly (done for each call) and not guaranteed -- better to leave it to the caller, IMHO. Have you implemented prepared statements properly for all backends? To the best of my knowledge, neither the Sybase's db-client library nor the FreeTDS reimplementation of it offer prepared statements http://lists.ibiblio.org/pipermail/freetds/2010q1/025493.html, unfortunately. The current implementation certainly does not use them. Sybase offers a newer client interface (ct-lib), that does have prepared statements (see ct_dynamic() http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.sdk_12.5.1.ctref/html/ctref/CHDIGFHG.htm), but FreeTDS does not provide it, so, for the driver to be compatible with both, it has to use the old db-interface... To me this seems like an acceptable caveat, as long as it is known and documented. My patch does not touch any of the other backends. Perhaps, some day I (or someone else) will implement apr_dbd_sybase.c -- using the ct-interface, which would give prepared statements and other improvements. But that would have to be maintained outside of Apache, because, foolishly, Sybase would not open-source the client... It would also be Sybase-only (no MS SQL Server). For the time being, my patch offers a major improvement over the status quo -- the driver becomes usable. I am, in fact, preparing to use it with RewriteMaps to do SEO for a giant site, that still uses an old Sybase-backed CMS. The RewriteRules will ensure, no tainted keys are passed to the queries and the DB-user's credentials will be limited to only SELECTs and only for a particular table to mitigate the risk of any SQL-injection attack. Yours, -mi
Re: Fixing apr_dbd_freetds (Re: Limitations of mod_dbd - single server per vhost)
On Wed, Aug 22, 2012 at 12:26 PM, Mikhail T. mi+t...@aldan.algebra.com wrote: Perhaps, this discussion should be happening on the ticket itself? Oh, well :) Comments inline: and/or d...@apr.apache.org On 22.08.2012 10:47, Nick Kew wrote: I think I've pointed a few people at the ODBC driver as an alternative. Do you have a strong reason to use FreeTDS rather than ODBC? Yes, ODBC is yet another layer of abstraction. I'd understand, if apr_dbd.c used purely ODBC, leaving the ODBC implementation to deal with the backends' native drivers. But if apr_dbd offers native drivers of its own, better to use those, if possible. That's my strong reason :-) One question here may be, does anyone have the platforms to test-drive your patch? I would not know... But, in my opinion, even without getting tested by anyone else, my patch is still a vast improvement over the current situation... It seems to do rather more than just fix a simple bug: That's because the bug was not simple -- the driver remained broken ever since the apr_dbd.c was changed to do the parsing of query-templates centrally instead of delegating the job to each driver. Other drivers got updated back then, but not the FreeTDS one :-( It still tried to do its own parsing, and was failing... you've removed the untainting code which was part of protecting against injection attacks. I don't think, the untainting was ever properly implemented (you can't do it right without prepared statements) and I don't believe, it is the driver's job to do it anyway. None of the other drivers do it either -- though they rely on the respective client-libraries. Sybase/FreeTDS client does not offer such checks -- so be it... The untainting was costly (done for each call) and not guaranteed -- better to leave it to the caller, IMHO. Have you implemented prepared statements properly for all backends? To the best of my knowledge, neither the Sybase's db-client library nor the FreeTDS reimplementation of it offer prepared statements, unfortunately. The current implementation certainly does not use them. Sybase offers a newer client interface (ct-lib), that does have prepared statements (see ct_dynamic()), but FreeTDS does not provide it, so, for the driver to be compatible with both, it has to use the old db-interface... To me this seems like an acceptable caveat, as long as it is known and documented. My patch does not touch any of the other backends. Perhaps, some day I (or someone else) will implement apr_dbd_sybase.c -- using the ct-interface, which would give prepared statements and other improvements. But that would have to be maintained outside of Apache, because, foolishly, Sybase would not open-source the client... It would also be Sybase-only (no MS SQL Server). For the time being, my patch offers a major improvement over the status quo -- the driver becomes usable. I am, in fact, preparing to use it with RewriteMaps to do SEO for a giant site, that still uses an old Sybase-backed CMS. The RewriteRules will ensure, no tainted keys are passed to the queries and the DB-user's credentials will be limited to only SELECTs and only for a particular table to mitigate the risk of any SQL-injection attack. Yours, -mi -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: Fixing apr_dbd_freetds
On 22.08.2012 12:55, Jeff Trawick wrote: and/or...@apr.apache.org You are right. Unfortunately, that list (like this one), requires one to subscribe before posting. I deeply resent such requirements and try to avoid such lists, when at all possible :-( Yours, -mi
Re: Ideas for an output filter for mod_lua
On 08/22/2012 01:36 PM, Nick Kew wrote: Basic concept looks fine. I guess we'd need more detail to say any more about it. Is the implementation 'clean' or does it involve hacks to core? If what you have is pure module then I'd see no reason not to drop mod_lua_filter (or is it mod_filter_lua?) into trunk on CTR. I don't see why it should be made a separate module - it relies and operates on the stuff already inside mod_lua, and making it a part of it only requires adding some 50-60 lines of code (ymmv). But ideas/opinions are of course welcome. As for the 'cleanliness' of the code, there are no hacks, it works similar to how the other hook functions in mod_lua work. Each call to a Lua(In|Out)putFilter would register a new filter with a callback to fx ap_lua_output_filter, which will then traverse a list of known scripts/handles, find the one associated with the filter name in question and run it, making the Lua script yield whenever it's parsed a bucket and sending it further down the chain. Apart from the actual bucket handling, the function and the directives behave just as the other hooks and the LuaMapHandler does. When called, it would register a file/function in an array, and when a call is made to that handler, it checks to see what is being called, and finds the file/function associated with it. i.e. if you had two directives: LuaOutputFilter filterA /www/filters.lua filter_a_func LuaOutputFilter filterB /www/filters.lua filter_b_func then an output, with any of those two filters attached, would call ap_lua_output_filter, which would look up the filter currently being run and then run the file/function mapped to that filter. I guess what separates this from the other hooks the most is that it relies on co-routines (aka non-blocking/event-based or whatever the newest buzz word for it is) to get the job done. Meanwhile, some random thoughts: Is there a deep reason to limit it to output filters, or is that just what you happen to have implemented? The same method could be applied to input filters as well, I just had an interest in trying it out for output filters, so that's what I hacked together before making this proposal. I'll try implementing an input filter tomorrow. Would your concept meaningfully generalise beyond application-level filters? I'm not entirely sure what you mean by this, could you elaborate? If you want some more sophisticated examples of what could be achieved with Lua filtering, I'd be happy to provide some more details on how this concept could be utilised. With regards, Daniel.
Re: Ideas for an output filter for mod_lua
On 22 Aug 2012, at 22:25, Daniel Gruno rum...@cord.dk wrote: Would your concept meaningfully generalise beyond application-level filters? I'm not entirely sure what you mean by this, could you elaborate? If you want some more sophisticated examples of what could be achieved with Lua filtering, I'd be happy to provide some more details on how this concept could be utilised. I don't know if this is another way of phrasing Nick's question or not, but would I be able to implement gzip Transfer-Encoding: just using Lua and this new directive? I found (bug 52860) it a bit tricky to achieve in C, so I think it could be harder still with the extra limitations of the Lua environment. My C code uses AP_FTYPE_TRANSCODE which I think is the right choice but few modules get involved at this filtering stage. -- Tim Bannister – is...@jellybaby.net smime.p7s Description: S/MIME cryptographic signature
Re: [Vote] httpd 2.2.23 release
On Tue, Aug 21, 2012 at 3:25 PM, William A. Rowe Jr. wr...@rowe-clan.net wrote: Candidate binaries are available from http://httpd.apache.org/dev/dist/ - these do not yet constitute ASF releases. Win32 specific artifacts (x86 binary distribution and -win32-src.zip) will follow sometime later this evening. Feedback and edits to the draft announcement are greatly appreciated, since we need to better position 2.2 as 'just maintenance' and we can drop some of the 'shiny new' bits of that Announcement text, given that it is old hat now, and they should be reading the 2.4 Announce (which this document now links to). So, for your consideration, a vote... [+1] Release httpd 2.2.23 as GA I get this testing both 2.2.22 and 2.2.23 on Linux 64-bit: # Failed test 2 in t/security/CVE-2008-2364.t at line 19 # Failed test 3 in t/security/CVE-2008-2364.t at line 22
