Re: HTTP/2 in massive file transfers

2016-03-08 Thread Joshua Marantz
That's a great benchmark, José. A couple of questions: 1. When using http 1.1, is your connection over SSL? I assume that your h2 connection is over ssl, and that might account for a speed penalty. 2. Have you tried this on other h2 implementations (h2o, nginx)? 3. Can you take some other data

Re: Sharing information across Apache child processes

2014-09-30 Thread Joshua Marantz
There's a variation on this theme you might consider. You can use a few counters in shared memory, and save config information in a file. If one of the child processes learns of a config update (say by handling an HTTP request for one from somewhere) it can update the file (atomically, say, via

Re: sequence of request

2014-07-24 Thread Joshua Marantz
Yes, in particular, the gdb feature that can help determine who is changing r-filename is hardware watchpoints: http://sourceware.org/gdb/onlinedocs/gdb/Set-Watchpoints.html 1. Set a breakpoint at a line of code where you believe r-filename is correct 2. Start up Apache with run -X 3. Issue a

associating an object with a request

2014-04-23 Thread Joshua Marantz
Hi, I want to create a data structure and associate it with a request_rec*, which will be accessed by multiple filters and handlers in our module. I know I can use the filter-ctx for general data structure storage, but I actually want something that will be accessed by multiple filters and

Re: associating an object with a request

2014-04-23 Thread Joshua Marantz
Thanks Eric. That looks perfect! I'll give it a try. On Wed, Apr 23, 2014 at 4:34 PM, Eric Covener cove...@gmail.com wrote: On Wed, Apr 23, 2014 at 4:31 PM, Joshua Marantz jmara...@google.com wrote: I want to create a data structure and associate it with a request_rec*, which

Re: problem with different version of openssl in Apache and apache module

2013-12-20 Thread Joshua Marantz
We faced this exact issue (openssl clashes with other linked-in versions) in mod_pagespeed and ngx_pagespeed, its nginx equivalent. We solved this problem in our Apache module because we linked mod_pagespeed.so hiding all the symbols other than the module entry-point into Apache. Here's the link

Re: Apache Buckets and Brigade

2013-05-01 Thread Joshua Marantz
I have a crazy idea for you. Maybe this is overkill but this sounds like it'd be natural to add to mod_pagespeed http://modpagespeed.com as a new filter. Here's some code you might use as a template

Re: Apache Buckets and Brigade

2013-05-01 Thread Joshua Marantz
On Wed, May 1, 2013 at 12:14 PM, Sindhi Sindhi sindhi@gmail.com wrote: Thanks to all for the reply. Josh, the concern I mentioned was, we may not want mod_pagespeed to modify the in-memory HTML content. The only change we may want to see in our HTML will be that the old strings are

Re: apr_memcache operation timeouts

2013-03-22 Thread Joshua Marantz
, Jeff Trawick traw...@gmail.com wrote: On Tue, Mar 12, 2013 at 1:56 PM, Jeff Trawick traw...@gmail.com wrote: On Mon, Mar 11, 2013 at 3:50 PM, Joshua Marantz jmara...@google.com wrote: ping! Please don't hesitate to push back and tell me if I can supply the patch or update in some

Re: apr_memcache operation timeouts

2013-03-11 Thread Joshua Marantz
the testing flow is for APR. I'd be happy to add unit-tests for that if someone points me to a change-list or patch-file that does it properly. -Josh On Thu, Nov 1, 2012 at 8:04 AM, Joshua Marantz jmara...@google.com wrote: I have completed a solution to this problem, which can be a drop-in update

Re: apr_memcache operation timeouts

2012-11-01 Thread Joshua Marantz
used to solve this problem, then please let me know if I can help with any changes required to get this into the main distribution, On Mon, Oct 22, 2012 at 5:21 PM, Joshua Marantz jmara...@google.com wrote: I've had some preliminary success with my own variant of apr_memcache.c (creatively called

Forcing Apache to exit on startup

2012-10-22 Thread Joshua Marantz
Hi, Our module has multiple confirmation parameters. There is a case where if you have option A and option B, then you must also specify option C, otherwise Bad things can happen that are a lot easier to debug on startup than they are after the server is running. I know how to force 'apachectl'

Re: apr_memcache operation timeouts

2012-10-22 Thread Joshua Marantz
. The socket timeout didn't work as well as this though. Does anyone have any theories as to why, or what could be done to the patch in https://issues.apache.org/bugzilla/show_bug.cgi?id=51065 to work? -Josh On Fri, Oct 19, 2012 at 9:25 AM, Joshua Marantz jmara...@google.com wrote: Following up: I tried

Re: apr_memcache operation timeouts

2012-10-19 Thread Joshua Marantz
of timing out the calls from my side. -Josh On Thu, Oct 18, 2012 at 10:46 AM, Joshua Marantz jmara...@google.comwrote: Thanks Jeff, that is very helpful. We are considering a course of action and before doing any work toward this, I'd like to understand the pitfalls from people that understand

apr_memcache operation timeouts

2012-10-17 Thread Joshua Marantz
Is there a mechanism to time out individual operations? If memcached freezes, then it appears my calls to 'get' will block until memcached wakes up. Is there any way to set a timeout for that call? I can repro this in my unit tests by sending a SIGSTOP to memcached before doing a 'get'? Here

Re: aprmemcache question

2012-10-13 Thread Joshua Marantz
. Is there any way to set a timeout for that call? I can repro this in my unit tests by sending a SIGSTOP to memcached before doing a 'get'. -Josh On Thu, Sep 27, 2012 at 4:37 PM, Joshua Marantz jmara...@google.com wrote: This helps a lot. I think 600 seconds seems like a fine idle-reap timeout

Re: aprmemcache question

2012-09-27 Thread Joshua Marantz
was not the case. Do you have a feel for the exact meaning of that TTL parameter to apr_memcache_server_create? -Josh On Thu, Sep 27, 2012 at 8:53 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: On Thu, Sep 27, 2012 at 4:05 AM, Joshua Marantz jmara...@google.com wrote: RE failing the build of my module

Re: aprmemcache question

2012-09-27 Thread Joshua Marantz
On Thu, Sep 27, 2012 at 10:58 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: If dlsym() is called with the special handle NULL, it is interpreted as a reference to the executable or shared object from which the call is being made. Thus a shared object can reference its own symbols. And

Re: aprmemcache question

2012-09-27 Thread Joshua Marantz
27, 2012 at 1:55 PM, Joshua Marantz jmara...@google.com wrote: That one call-site is HTTP_24/src/modules/cache/mod_socache_memcache.c, right? That was where I stole my args from. no, subversion As the TCP/IP layer is a lower level abstraction than bathe apr_memcache interface, I'm still

aprmemcache question

2012-09-26 Thread Joshua Marantz
Hi, I've been having some success with the apr_memcache_* functions. In load-tests, however, I'm finding a lot of timeouts with apr_memcache_multgetp. Specifically, the status returned with the individual elements is APR_TIMEUP. This leads me to wonder what the significance of the second to

Re: aprmemcache question

2012-09-26 Thread Joshua Marantz
developers? -Josh On Wed, Sep 26, 2012 at 6:20 PM, Jeff Trawick traw...@gmail.com wrote: On Wed, Sep 26, 2012 at 5:38 PM, Joshua Marantz jmara...@google.com wrote: Hi, I've been having some success with the apr_memcache_* functions. In load-tests, however, I'm finding a lot of timeouts

Re: aprmemcache question

2012-09-26 Thread Joshua Marantz
at 7:17 PM, Joshua Marantz jmara...@google.com wrote: +dev (sorry for the duplicate; my first attempt failed due to not being a subscriber). Keeping modules-dev on CC if that's appropriate. Thanks, Jeff, I was wondering if there was a units issue there. I'm still wondering if anyone can

Re: aprmemcache question

2012-09-26 Thread Joshua Marantz
at 7:31 PM, Joshua Marantz jmara...@google.com wrote: Looking at source, I see that Jeff's patch, and the 'ttl' parameter in general, is only referenced under '#if APR_HAS_THREADS'. When I load-tested and found the timeouts, I was testing under Apache 2.2 Prefork, and thus that patched

apr_memcache vs libmemcached

2012-07-19 Thread Joshua Marantz
Hi, Is anyone maintaining apr_memcache? Or should I stick with libmemcached? This page: http://code.google.com/p/memcached/wiki/Clients suggests apr_memcached is not maintained. Thanks! -Josh

Re: apr_memcache vs libmemcached

2012-07-19 Thread Joshua Marantz
. Thanks, -Josh On Thu, Jul 19, 2012 at 5:39 PM, Nick Kew n...@apache.org wrote: On Thu, 19 Jul 2012 16:41:51 -0400 Joshua Marantz jmara...@google.com wrote: Hi, Is anyone maintaining apr_memcache? Or should I stick with libmemcached? This page: http://code.google.com/p/memcached/wiki

Re: Get the directory of the module

2012-06-11 Thread Joshua Marantz
I think he wants to write bytecodes during module installation when he will have write access. During installation can you also write a .conf template showing your module where the bytecodes are? Josh On Jun 11, 2012 7:29 PM, Jerry Stuckle je...@smartechhomes.com wrote: On 6/11/2012 5:49

Re: Get the directory of the module

2012-06-11 Thread Joshua Marantz
be able to tell his module where the files are. On 6/11/2012 7:40 PM, Joshua Marantz wrote: I think he wants to write bytecodes during module installation when he will have write access. During installation can you also write a .conf template showing your module where the bytecodes are? Josh

Re: Determining mimetype of document

2012-06-08 Thread Joshua Marantz
On Fri, Jun 8, 2012 at 5:16 PM, Joe Lewis j...@joe-lewis.com wrote: On 06/08/2012 03:07 PM, Joshua Marantz wrote: Hi, I'm trying to figure out whether my filter, which currently runs upstream of mod_headers, is likely to see the correct mimetype in request-content_type. In particular, we need

Re: output filter ordering question: mod_deflate vs mod_headers vs mod_pagespeed

2012-05-17 Thread Joshua Marantz
of mod_filter? Thanks! -Josh On Thu, May 17, 2012 at 2:30 AM, Nick Kew n...@apache.org wrote: On 17 May 2012, at 04:24, Joshua Marantz wrote: Or is that insane /or dangerous? AP_FTYPE_ values are something of a blunt instrument, not ideal for cases where you care about ordering. Running

Re: output filter ordering question: mod_deflate vs mod_headers vs mod_pagespeed

2012-05-17 Thread Joshua Marantz
achieved the ordering I want yet. Is there something else I need to do? On Thu, May 17, 2012 at 9:27 AM, Joshua Marantz jmara...@google.com wrote: Thanks for the quick reply, Nick. I played around with this idea but couldn't get it to work. My filter now runs at AP_FTYPE_CONTENT_SET + 1, to make

output filter ordering question: mod_deflate vs mod_headers vs mod_pagespeed

2012-05-16 Thread Joshua Marantz
Hi, Currently, mod_pagespeed http://code.google.com/p/modpagespeed/ runs before mod_headers by registering as: ap_register_output_filter( kModPagespeedFilterName, instaweb_out_filter, NULL, static_castap_filter_type(AP_FTYPE_RESOURCE + 1)); mod_headers registers with:

Re: Inject html into response body

2011-12-04 Thread Joshua Marantz
What's your thinking on how you are going to parse the HTML to inject the code? Do you need to parse arbitrary HTML from any website or is this filter targeted at a particular site whose HTML is tailored for this purpose? -Josh On Sun, Dec 4, 2011 at 5:15 PM, r...@tuxteam.de wrote: On Sun,

Re: Question about malloc / realloc in module

2011-09-14 Thread Joshua Marantz
If you malloc/calloc/realloc without a free you will leak memory. Do you have some reason to believe that another module might prevent your module's calls to free() from being run? What do you have in mind specifically? You can also mimic realloc by just allocating more memory from the pool and

po

2011-09-01 Thread Joshua Marantz
Hi, I've been load-testing our module (mod_pagespeedhttp://code.google.com/speed/page-speed/docs/module.html) with httpd 2.2.16 built with these options: --enable-pool-debug --with-mpm=worker I've been getting periodic aborts from apr_table_addn that don't look like they are from my module.

Detecting which MPM a module is running in

2011-09-01 Thread Joshua Marantz
Hello from mod_pagespeed again. We are adding support for running in the Worker MPM, having spent most of our time since we launched the product sheltered in the prefork MPM where our multi-threading challenges are all of our own making. Having tuned our threading model for prefork, where all

Re: Vary:User-Agent, best practices, and making the web faster.

2011-06-05 Thread Joshua Marantz
compliance without falling off a performance cliff. -Josh On Sun, Jun 5, 2011 at 2:54 PM, Ben Noordhuis i...@bnoordhuis.nl wrote: On Sun, Jun 5, 2011 at 13:42, Joshua Marantz jmara...@google.com wrote: This is a case where the content varies based on user-agent. The recommendation

Vary:User-Agent, best practices, and making the web faster.

2011-06-04 Thread Joshua Marantz
Hi, We've been working on a lingering HTTP-compliance issue in mod_pagespeed: respecting Vary:User-Agent. mod_pagespeed needs to cache resources in order to optimize them. The economics of this make sense when the server optimizes a resource, and saves the optimization for serving to multiple

Re: Vary:User-Agent, best practices, and making the web faster.

2011-06-04 Thread Joshua Marantz
at 21:26, Joshua Marantz jmara...@google.com wrote: I think what we'd do is basically let mod_pagespeed ignore Vary:User-Agent if we saw that it was inserted per this exact pattern. This would, to be This seems like a stupendously bad idea. Warn about it in your docs, complain about

KeepAlive -- why is it off by default?

2011-04-11 Thread Joshua Marantz
webpagetest.org gives a big red *F *to web sites that do* *not have KeepAlive enabled. Yet, at least in the CentOS installation of Apache, it is off by default. Why is that? What are the reasons that KeepAlive should be off? Although it's trivial to turn it on in httpd.conf, many web sites do

Re: how to parse html content in handler

2011-03-25 Thread Joshua Marantz
mod_pagespeed's event-driven HTML parser is open source, and is written in C++: http://code.google.com/p/modpagespeed/source/browse/trunk/src/net/instaweb/htmlparse/public/html_parse.h http://code.google.com/p/modpagespeed/source/browse/trunk/src/net/instaweb/htmlparse/public/html_parse.hThis

Re: ordering output filters

2011-03-14 Thread Joshua Marantz
Thanks, Nick Ben! On Mon, Mar 14, 2011 at 1:31 PM, Nick Kew n...@apache.org wrote: AP_FTYPE_RESOURCE+1. That also leaves an admin the possibility of overriding it. I didn't realize these +1/-1 hacks were available for this API. This looks really simple is the direction I'm leaning.

Saving the original request URI ahead of a mod_rewrite

2011-03-12 Thread Joshua Marantz
Hi, A new bug has surfaced in mod_pagespeed that we understand, but would welcome advice on the best way to fix. The problem is tracked in http://code.google.com/p/modpagespeed/issues/detail?id=234. Briefly, mod_pagespeedhttp://code.google.com/speed/page-speed/docs/using_mod.htmlseeks to

Help trying to figure out why an output_filter is not called.

2011-01-05 Thread Joshua Marantz
One of the improvements mod_pagespeed is supposed to do to sites is extend the cache lifetime of their resources indefinitely by including a content hash in the URL. This is working well for a large number of sites, but I encountered one today where it does not work. To accomplish the cache

Re: Help trying to figure out why an output_filter is not called.

2011-01-05 Thread Joshua Marantz
Thanks again for the fast response, Ben! On Wed, Jan 5, 2011 at 8:57 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: On Wed, Jan 5, 2011 at 14:45, Joshua Marantz jmara...@google.com wrote: other filter be somehow finding our filter and killing it? Or sending the bytes directly to the network

Re: Help trying to figure out why an output_filter is not called.

2011-01-05 Thread Joshua Marantz
On Wed, Jan 5, 2011 at 10:43 AM, Ben Noordhuis i...@bnoordhuis.nl wrote: I guess we should eliminate FIXUP_HEADERS_OUT, FIXUP_HEADERS_ERR, and MOD_EXPIRES. Are there any other similar header-mucking-filters I need to kill? Moreover, expires_insert_filter runs as APR_HOOK_MIDDLE which

Re: Help trying to figure out why an output_filter is not called.

2011-01-05 Thread Joshua Marantz
/ Throttlebox affiliate program: http://www.bettercgi.com/affiliates/user/register.php On 01/05/2011 03:16:21 PM, Ben Noordhuis wrote: On Wed, Jan 5, 2011 at 22:03, Joshua Marantz jmara...@google.com wrote: Right you are. That's much simpler then. Thanks! My pleasure, Joshua. Two quick questions

Re: Overriding mod_rewrite from another module

2011-01-03 Thread Joshua Marantz
at 00:16, Joshua Marantz jmara...@google.com wrote: Thanks for the quick response and the promising idea for a hack. Looking at mod_rewrite.c this does indeed look a lot more surgical, if, perhaps, fragile, as mod_rewrite.c doesn't expose that string-constant in any formal interface (even

Re: Overriding mod_rewrite from another module

2011-01-03 Thread Joshua Marantz
with full authentication. I'd appreciate any comments on this approach. -Josh On Mon, Jan 3, 2011 at 11:40 AM, Joshua Marantz jmara...@google.com wrote: OK I tried to find a more robust alternative but could not. I was thinking I could duplicate whatever mod_rewrite was doing to set the request

Re: Overriding mod_rewrite from another module

2011-01-03 Thread Joshua Marantz
On Mon, Jan 3, 2011 at 4:50 PM, Ben Noordhuis i...@bnoordhuis.nl wrote: This means that returning OK from my handler does not prevent mod_authz_host's handler from being called. You're mistaken, Joshua. The access_checker hook by default is empty. mod_authz_host is a module and it can be

Re: Overriding mod_rewrite from another module

2011-01-03 Thread Joshua Marantz
On Mon, Jan 3, 2011 at 6:15 PM, Eric Covener cove...@gmail.com wrote: The access checking on mod_pagespeed resources is redundant, because the resource will either be served from cache (in which case it had to be authenticated to get into the cache in the first place) or will be decoded

Overriding mod_rewrite from another module

2010-12-31 Thread Joshua Marantz
I need to find the best way to prevent mod_rewrite from renaming resources that are generated by a different module, specifically mod_pagespeed. This needs to be done from within mod_pagespeed, rather than asking the site admin to tweak his rule set. By reading mod_rewrite.c, I found a mechanism

Re: Overriding mod_rewrite from another module

2010-12-31 Thread Joshua Marantz
, Joshua Marantz jmara...@google.com wrote: Is there a better way to solve the original problem: preventing mod_rewrite from corrupting mod_pagespeed's resources? From memory and from a quick peek at mod_rewrite.c: in your translate_name hook, set a mod_rewrite_rewritten note in r-notes

Re: Howto unittest httpd modules?

2010-11-22 Thread Joshua Marantz
For mod_pagespeed http://code.google.com/speed/page-speed/docs/module.htmlwe do 2 of the 3 testing modes discussed. We have the majority of our code running independent of Apache so it can be aggressively unit-tested with googletesthttp://code.google.com/p/googletest/ . We also have automated