Re: My first Apache module!

2011-08-25 Thread Sorin Manolache
On Wed, Aug 24, 2011 at 23:59, Chris London ch...@kwista.com wrote:
 Hey everyone,

 I don't like to bother mailing lists with beginner questions so I spent
 a few hours on Google looking for info and tutorials. I'm sure at least one
 of you has had that experience :) Anyway, I have finished my first module
 and it works exactly how I want it to.  I still have more features I want to
 add but I'm a little rusty with my C and even worse at Apache Modules. Would
 you mind taking a glance at my code to see if I'm doing anything wrong or
 have any memory leaks or anything?

 Here's the source code:
 https://github.com/chrislondon/Dynamic-Stylesheets/blob/master/mod_dss.c

 Feel free to fork my project:
 https://github.com/chrislondon/Dynamic-Stylesheets

 The README file (
 https://github.com/chrislondon/Dynamic-Stylesheets/blob/master/README) 
 contains
 usage info and the project contains a simple test case.

 Thanks everyone!

You open the file but then you exit the handler if r-header_only
without closing the file. Check for r-header_only before opening the
file.

In T_read you allocate cKey, cVal, and temp_node once for every
iteration of the outermost while loop in which *cCurrent == '@' , but
you free this memory only once, outside the while loop.

Also, you do not free correctly your linked list.

S


RewriteRule question

2011-08-25 Thread Denys

Hello dear community -

I need to create some kind of proxy using Apache Server.

I tried to use something like
ProxyPass /test1/test2/ http://another_server/test/test.dll but had no
success because ProxyPass require / suffix after the second URL (something
like http://another_server/test/test.dll). In my case adding / is not
working because now test.dll part is meant as a directory instead of a file
test.dll inside test directory (my IIS server thinks so).

After that I tried to use RewriteRule. For example:
RewriteRule /test1/test2/ http://another_server/test/test.dll. This time
redirection is working perfectly - I see the desired page, but the URL is
completely changed to the new one and I want end users to see the first part
of the URL unchanged (the part with the server name and port). E.g. if user
enters like http://server1/test1/test2/ I would like him to see something
like http://server1/test1/test2/... after redirection.

Are there any ability to achieve this?

Any kind of help is greatly appreciated.

Thanks in advance.
-- 
View this message in context: 
http://old.nabble.com/RewriteRule-question-tp32332668p32332668.html
Sent from the Apache HTTP Server - Module Writers mailing list archive at 
Nabble.com.



Re: RewriteRule question

2011-08-25 Thread Denys

Do you mean, Nick, that I have submitted the question to the wrong branch?



Nick Kew-3 wrote:
 
 On 25 Aug 2011, at 10:15, Denys wrote:
 
 Any kind of help is greatly appreciated.
 
 Start by reading TFM.
 
 If you still can't understand, try a user support forum.
 
 View this message in context:
 http://old.nabble.com/RewriteRule-question-tp32332668p32332668.html
 
 Oh, right, a website that appropriates our list and misleads (some) users.
 
 Sent from the Apache HTTP Server - Module Writers mailing list archive at
 Nabble.com.
 
 But that looks clear enough to me.  Is the website so two-faced as to be
 unclear about that,
 or do you struggle with English?
 
 -- 
 Nick Kew
 

-- 
View this message in context: 
http://old.nabble.com/RewriteRule-question-tp32332668p32332926.html
Sent from the Apache HTTP Server - Module Writers mailing list archive at 
Nabble.com.



Re: RewriteRule question

2011-08-25 Thread Nick Kew
On 25 Aug 2011, at 11:04, Denys wrote:

 
 Do you mean, Nick, that I have submitted the question to the wrong branch?

branch?  What's a branch?  This is a mailing list for module developers!

Sorry, my reply was OTT.  I just get a bit cross at some of the stuff that gets
misdirected from nabble.com.

-- 
Nick Kew

Re: RewriteRule question

2011-08-25 Thread Nick Kew

On 25 Aug 2011, at 11:04, Denys wrote:

 
 Do you mean, Nick, that I have submitted the question to the wrong branch?

branch?  What's a branch?  This is a mailing list for module developers!

Sorry, my reply was OTT.  I just get a bit cross at some of the stuff that gets
misdirected from nabble.com.

-- 
Nick Kew

Re: DoS with mod_deflate range requests

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Jim Jagielski wrote:
 OK then… we seem to be coalescing into some consensus here…
 basically, if the client sends stuff which is brain-dead stupid,
 we simply 2000 and send the whole kit-and-kaboodle.
 
 I'd like to propose that we update the byterange filter to perform
 the following:
 
   o coalesce all adjacent ranges, whether overlapping or not.
 (eg: 200-250,251-300  200-250,220-300 both merge to 200-300)

This may still confuse a broken client. Maybe we could omit that from 
the 2.2 patch for now and only commit to 2.3.

   o We count:
   the number of times a gap between ranges is 80bytes
   the number of times we hit a descendent range
(eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
 1)  the number of ranges total (post ascending merge)
 If any = some config-time limit, we send a 200
 
 This is a start and was chosen simply for ease of implementation…
 We can then expand it to be more functional…
 
 Comments?

Please also look at the patch at

http://mail-archives.apache.org/mod_mbox/httpd-
dev/201108.mbox/%3c201108250138.49474...@sfritsch.de%3E

which greatly reduces the memory needed for the range requests.
BTW, I won't have time to beat that into shape today. If anyone else 
has, please go ahead.


Re: Final draft / CVE-2011-3192

2011-08-25 Thread Dirk-Willem van Gulik
Thanks. Added to the interim draft update.

Dw.

On 25 Aug 2011, at 06:36, Steffen wrote:

 For Mitigation of Apache Range Header DoS Attack with mod_security, see
 also:
 
 http://blog.spiderlabs.com/2011/08/mitigation-of-apache-range-header-dos-attack.html
 
 
 - Original Message -
 From: Dirk-Willem van Gulik di...@webweaving.org
 Newsgroups: gmane.comp.apache.devel
 To: secur...@httpd.apache.org; dev@httpd.apache.org
 Sent: Wednesday, August 24, 2011 5:34 PM
 Subject: Final draft / CVE-2011-3192
 
 
 Thanks for all the help. All fixes included. Below will go out to announce
 at the top of the hour - unless I see a veto.
 
 Dw.
 
 
 
 
 Title:CVE-2011-3192: Range header DoS vulnerability Apache HTTPD 1.3/2.x
   Apache HTTPD Security ADVISORY
 
 Date: 20110824 1600Z
 Product:  Apache HTTPD Web Server
 Versions: Apache 1.3 all versions, Apache 2 all versions
 
 Description:
 
 
 A denial of service vulnerability has been found in the way the multiple
 overlapping ranges are handled by the Apache HTTPD server:
 
  http://seclists.org/fulldisclosure/2011/Aug/175
 
 An attack tool is circulating in the wild. Active use of this tools has
 been observed.
 
 The attack can be done remotely and with a modest number of requests can
 cause very significant memory and CPU usage on the server.
 
 The default Apache HTTPD installation is vulnerable.
 
 There is currently no patch/new version of Apache HTTPD which fixes this
 vulnerability. This advisory will be updated when a long term fix
 is available.
 
 A full fix is expected in the next 48 hours.
 
 Mitigation:
 
 
 However there are several immediate options to mitigate this issue until
 that time.
 
 1) Use SetEnvIf or mod_rewrite to detect a large number of ranges and then
either ignore the Range: header or reject the request.
 
Option 1: (Apache 2.0 and 2.2)
 
   # drop Range header when more than 5 ranges.
   # CVE-2011-3192
   SetEnvIf Range (,.*?){5,} bad-range=1
   RequestHeader unset Range env=bad-range
 
   # optional logging.
   CustomLog logs/range-CVE-2011-3192.log common env=bad-range
 
Option 2: (Also for Apache 1.3)
 
   # Reject request when more than 5 ranges in the Range: header.
   # CVE-2011-3192
   #
   RewriteEngine on
   RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$)
   RewriteRule .* - [F]
 
The number 5 is arbitrary. Several 10's should not be an issue and may be
required for sites which for example serve PDFs to very high end eReaders
or use things such complex http based video streaming.
 
 2) Limit the size of the request field to a few hundred bytes. Note that
 while
this keeps the offending Range header short - it may break other headers;
such as sizeable cookies or security fields.
 
   LimitRequestFieldSize 200
 
Note that as the attack evolves in the field you are likely to have
to further limit this and/or impose other LimitRequestFields limits.
 
See: http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestfieldsize
 
 3) Use mod_headers to completely dis-allow the use of Range headers:
 
   RequestHeader unset Range
 
Note that this may break certain clients - such as those used for
e-Readers and progressive/http-streaming video.
 
 4) Deploy a Range header count module as a temporary stopgap measure:
 
  http://people.apache.org/~dirkx/mod_rangecnt.c
 
Precompiled binaries for some platforms are available at:
 
 http://people.apache.org/~dirkx/BINARIES.txt
 
 5) Apply any of the current patches under discussion - such as:
 

 http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox/%3ccaapsnn2po-d-c4nqt_tes2rrwizr7urefhtkpwbc1b+k1dq...@mail.gmail.com%3e
 
 Actions:
 
 
 However there are several immediate options to mitigate this issue until
 that time.
 
 1) Use SetEnvIf or mod_rewrite to detect a large number of ranges and then
either ignore the Range: header or reject the request.
 
Option 1: (Apache 2.0 and 2.2)
 
   # drop Range header when more than 5 ranges.
   # CVE-2011-3192
   SetEnvIf Range (,.*?){5,} bad-range=1
   RequestHeader unset Range env=bad-range
 
   # optional logging.
   CustomLog logs/range-CVE-2011-3192.log common env=bad-range
 
Option 2: (Also for Apache 1.3)
 
   # Reject request when more than 5 ranges in the Range: header.
   # CVE-2011-3192
   #
   RewriteEngine on
   RewriteCond %{HTTP:range} !(^bytes=[^,]+(,[^,]+){0,4}$|^$)
   RewriteRule .* - [F]
 
The number 5 is arbitrary. Several 10's should not be an issue and may be
required for sites which for example serve PDFs to very high end eReaders
or use things such complex http based video streaming.
 
 2) Limit the size of the request field to a few hundred bytes. Note 

Re: Fixing Ranges

2011-08-25 Thread Dirk-Willem van Gulik

On 24 Aug 2011, at 22:16, Stefan Fritsch wrote:

 I have another idea: Instead of using apr_brigade_partition write a
 new function ap_brigade_copy_part that leaves the original brigade
 untouched. It would copy the necessary buckets to a new brigade and
 then split the first and last of those copied buckets as necessary and
 destroy the excess buckets. AFAICS, this would reduce the quadratic
 growth into linear. Do you think that would solve our problems?

This looks really rather need - it seems to tackle the core issue - that of 
memory; and it also makes it better for legitimate request with many ranges in 
it (while I never noticed before - going back and looking at them - them PDF 
requests do take a lot of memory).

Dw.



RE: DoS with mod_deflate range requests

2011-08-25 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Stefan Fritsch 
 Sent: Donnerstag, 25. August 2011 08:21
 To: dev@httpd.apache.org
 Subject: Re: DoS with mod_deflate  range requests
 
 On Thursday 25 August 2011, Jim Jagielski wrote:
  OK then... we seem to be coalescing into some consensus here...
  basically, if the client sends stuff which is brain-dead stupid,
  we simply 2000 and send the whole kit-and-kaboodle.
  
  I'd like to propose that we update the byterange filter to perform
  the following:
  
o coalesce all adjacent ranges, whether overlapping or not.
  (eg: 200-250,251-300  200-250,220-300 both merge to 200-300)
 
 This may still confuse a broken client. Maybe we could omit that from 
 the 2.2 patch for now and only commit to 2.3.

Sounds like a plan. Or make it configurable with a default of off in 2.2.x
and on in 2.3.x

 
o We count:
the number of times a gap between ranges is 80bytes
the number of times we hit a descendent range
 (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
  1)  the number of ranges total (post ascending merge)
  If any = some config-time limit, we send a 200
  
  This is a start and was chosen simply for ease of implementation...
  We can then expand it to be more functional...
  
  Comments?


Looks good. Plus we should implement the patch from Stefan below and then we
should be good.

 
 Please also look at the patch at
 
 http://mail-archives.apache.org/mod_mbox/httpd-
 dev/201108.mbox/%3c201108250138.49474...@sfritsch.de%3E
 
 which greatly reduces the memory needed for the range requests.
 BTW, I won't have time to beat that into shape today. If anyone else 
 has, please go ahead.
 

Regards

Rüdiger


Next update on CVE-2011-3192

2011-08-25 Thread Dirk-Willem van Gulik
I am keeping a draft at

http://people.apache.org/~dirkx/CVE-2011-3192.txt

Changes since last are:

-   version ranges more specific
-   vendor information added
-   backgrounder on relation to 2007 issues (see below to ensure I got this 
right).

I suggest we sent this out late Z time today (i.e. end of working day US) _if_ 
1) it is likely that we do not have a firm timeline for the full fix and 2) we 
have a bit more to add. Otherwise we skip to a final update with the fixing 
instructions for 2.0 and 2.2

Feedback welcome,

Thanks,

Dw.

Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
Tested and this does appear to both address the DoS as well as
reduce memory usage for excessive range requests…

+1 for adding this no matter what.

On Aug 24, 2011, at 7:38 PM, Stefan Fritsch wrote:

 On Thursday 25 August 2011, Greg Ames wrote:
 On Wed, Aug 24, 2011 at 5:16 PM, Stefan Fritsch s...@sfritsch.de 
 wrote:
 I have another idea: Instead of using apr_brigade_partition write
 a new function ap_brigade_copy_part that leaves the original
 brigade untouched. It would copy the necessary buckets to a new
 brigade and then split the first and last of those copied
 buckets as necessary and destroy the excess buckets. AFAICS,
 this would reduce the quadratic growth into linear. Do you think
 that would solve our problems?
 
 How does apr_brigade_partition contribute to quadratic growth? 
 Does the original brigade end up with a lot of one byte buckets?
 
 Yes, it splits the buckets in the original brigade, creating up to two 
 new buckets for every range. These split one-byte buckets are then 
 copied again for each of the subsequent ranges.
 
 The attached PoC patch does not change the original brigade and seems 
 to fix the DoS for me. It needs some more work and some review for 
 integer overflows, though. (apr_brigade_partition does some 
 interesting things there).
 range-linear.diff



Re: DoS with mod_deflate range requests

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 2:56 AM, Plüm, Rüdiger, VF-Group wrote:

 
 
 -Original Message-
 From: Stefan Fritsch 
 Sent: Donnerstag, 25. August 2011 08:21
 To: dev@httpd.apache.org
 Subject: Re: DoS with mod_deflate  range requests
 
 On Thursday 25 August 2011, Jim Jagielski wrote:
 OK then... we seem to be coalescing into some consensus here...
 basically, if the client sends stuff which is brain-dead stupid,
 we simply 2000 and send the whole kit-and-kaboodle.
 
 I'd like to propose that we update the byterange filter to perform
 the following:
 
  o coalesce all adjacent ranges, whether overlapping or not.
(eg: 200-250,251-300  200-250,220-300 both merge to 200-300)
 
 This may still confuse a broken client. Maybe we could omit that from 
 the 2.2 patch for now and only commit to 2.3.
 
 Sounds like a plan. Or make it configurable with a default of off in 2.2.x
 and on in 2.3.x
 
 
  o We count:
 the number of times a gap between ranges is 80bytes
 the number of times we hit a descendent range
   (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
 1)  the number of ranges total (post ascending merge)
If any = some config-time limit, we send a 200
 
 This is a start and was chosen simply for ease of implementation...
 We can then expand it to be more functional...
 
 Comments?
 
 
 Looks good. Plus we should implement the patch from Stefan below and then we
 should be good.
 

++1 (see other thread: Fixing Ranges)



Re: Fixing Ranges

2011-08-25 Thread Dirk-Willem van Gulik

On 25 Aug 2011, at 12:40, Jim Jagielski wrote:

 Tested and this does appear to both address the DoS as well as
 reduce memory usage for excessive range requests…
 
 +1 for adding this no matter what.

Yup - same here. Makes PDF serving a heck of a lot better too.

Dw.

 
 On Aug 24, 2011, at 7:38 PM, Stefan Fritsch wrote:
 
  On Thursday 25 August 2011, Greg Ames wrote:
  On Wed, Aug 24, 2011 at 5:16 PM, Stefan Fritsch s...@sfritsch.de
  wrote:
  I have another idea: Instead of using apr_brigade_partition write
  a new function ap_brigade_copy_part that leaves the original
  brigade untouched. It would copy the necessary buckets to a new
  brigade and then split the first and last of those copied
  buckets as necessary and destroy the excess buckets. AFAICS,
  this would reduce the quadratic growth into linear. Do you think
  that would solve our problems?
 
  How does apr_brigade_partition contribute to quadratic growth?
  Does the original brigade end up with a lot of one byte buckets?
 
  Yes, it splits the buckets in the original brigade, creating up to two
  new buckets for every range. These split one-byte buckets are then
  copied again for each of the subsequent ranges.
 
  The attached PoC patch does not change the original brigade and seems
  to fix the DoS for me. It needs some more work and some review for
  integer overflows, though. (apr_brigade_partition does some
  interesting things there).
  range-linear.diff
 
 



Re: Next update on CVE-2011-3192

2011-08-25 Thread Jim Jagielski
I have a feeling that we could push this out today…

I'm going to fold Stefan's path into trunk, and we should use
trunk (CTR) to polish up the patch as well as add whatever
other features we need. From there, backporting to 2.2/2.0
will be trivial.

On Aug 25, 2011, at 4:18 AM, Dirk-Willem van Gulik wrote:

 I am keeping a draft at
 
   http://people.apache.org/~dirkx/CVE-2011-3192.txt
 
 Changes since last are:
 
 - version ranges more specific
 - vendor information added
 - backgrounder on relation to 2007 issues (see below to ensure I got this 
 right).
 
 I suggest we sent this out late Z time today (i.e. end of working day US) 
 _if_ 1) it is likely that we do not have a firm timeline for the full fix and 
 2) we have a bit more to add. Otherwise we skip to a final update with the 
 fixing instructions for 2.0 and 2.2
 
 Feedback welcome,
 
 Thanks,
 
 Dw.



RE: Fixing Ranges

2011-08-25 Thread Plüm, Rüdiger, VF-Group
 

 -Original Message-
 From: Stefan Fritsch  
 Sent: Donnerstag, 25. August 2011 01:39
 To: dev@httpd.apache.org
 Subject: Re: Fixing Ranges
 
 On Thursday 25 August 2011, Greg Ames wrote:
  On Wed, Aug 24, 2011 at 5:16 PM, Stefan Fritsch s...@sfritsch.de 
 wrote:
   I have another idea: Instead of using apr_brigade_partition write
   a new function ap_brigade_copy_part that leaves the original
   brigade untouched. It would copy the necessary buckets to a new
   brigade and then split the first and last of those copied
   buckets as necessary and destroy the excess buckets. AFAICS,
   this would reduce the quadratic growth into linear. Do you think
   that would solve our problems?
  
  How does apr_brigade_partition contribute to quadratic growth? 
  Does the original brigade end up with a lot of one byte buckets?
 
 Yes, it splits the buckets in the original brigade, creating 
 up to two 
 new buckets for every range. These split one-byte buckets are then 
 copied again for each of the subsequent ranges.
 
 The attached PoC patch does not change the original brigade and seems 
 to fix the DoS for me. It needs some more work and some review for 
 integer overflows, though. (apr_brigade_partition does some 
 interesting things there).
 

I guess the following should be adjusted with this patch:

1. Do the same integer casting thing apr_brigade_partition does. So
   convert everything to apr_uint64_t and work with this.

2. Doing a read on a bucket can change its length. So we need to check
   afterwards if we still can split the bucket and if not might do a read
   on the next bucket until we are fine.

3. We should do the read bucket thing for the last bucket as well.

I think it would be best if you could commit this PoC patch as is (or with 
adjustments
according to the comments above) to trunk and we polish it up in trunk
until it is fine.

Regards

Rüdiger


RE: Next update on CVE-2011-3192

2011-08-25 Thread Plüm, Rüdiger, VF-Group
+1

Regards

Rüdiger 

 -Original Message-
 From: Jim Jagielski [mailto:j...@jagunet.com] 
 Sent: Donnerstag, 25. August 2011 14:13
 To: dev@httpd.apache.org
 Subject: Re: Next update on CVE-2011-3192
 
 I have a feeling that we could push this out today...
 
 I'm going to fold Stefan's path into trunk, and we should use
 trunk (CTR) to polish up the patch as well as add whatever
 other features we need. From there, backporting to 2.2/2.0
 will be trivial.
 
 On Aug 25, 2011, at 4:18 AM, Dirk-Willem van Gulik wrote:
 
  I am keeping a draft at
  
  http://people.apache.org/~dirkx/CVE-2011-3192.txt
  
  Changes since last are:
  
  -   version ranges more specific
  -   vendor information added
  -   backgrounder on relation to 2007 issues (see below to 
 ensure I got this right).
  
  I suggest we sent this out late Z time today (i.e. end of 
 working day US) _if_ 1) it is likely that we do not have a 
 firm timeline for the full fix and 2) we have a bit more to 
 add. Otherwise we skip to a final update with the fixing 
 instructions for 2.0 and 2.2
  
  Feedback welcome,
  
  Thanks,
  
  Dw.
 
 


RE: Fixing Ranges

2011-08-25 Thread Stefan Fritsch

On Thu, 25 Aug 2011, Plüm, Rüdiger, VF-Group wrote:
I think it would be best if you could commit this PoC patch as is (or 
with adjustments according to the comments above) to trunk and we polish 
it up in trunk until it is fine.


Somebody else go ahead and commit it, please. I won't have time until this 
evening, at least.


Cheers,
Stefan


Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
PoC folded into trunk… Let's go ;)

On Aug 25, 2011, at 8:16 AM, Plüm, Rüdiger, VF-Group wrote:

 
 
 -Original Message-
 From: Stefan Fritsch  
 Sent: Donnerstag, 25. August 2011 01:39
 To: dev@httpd.apache.org
 Subject: Re: Fixing Ranges
 
 On Thursday 25 August 2011, Greg Ames wrote:
 On Wed, Aug 24, 2011 at 5:16 PM, Stefan Fritsch s...@sfritsch.de 
 wrote:
 I have another idea: Instead of using apr_brigade_partition write
 a new function ap_brigade_copy_part that leaves the original
 brigade untouched. It would copy the necessary buckets to a new
 brigade and then split the first and last of those copied
 buckets as necessary and destroy the excess buckets. AFAICS,
 this would reduce the quadratic growth into linear. Do you think
 that would solve our problems?
 
 How does apr_brigade_partition contribute to quadratic growth? 
 Does the original brigade end up with a lot of one byte buckets?
 
 Yes, it splits the buckets in the original brigade, creating 
 up to two 
 new buckets for every range. These split one-byte buckets are then 
 copied again for each of the subsequent ranges.
 
 The attached PoC patch does not change the original brigade and seems 
 to fix the DoS for me. It needs some more work and some review for 
 integer overflows, though. (apr_brigade_partition does some 
 interesting things there).
 
 
 I guess the following should be adjusted with this patch:
 
 1. Do the same integer casting thing apr_brigade_partition does. So
   convert everything to apr_uint64_t and work with this.
 
 2. Doing a read on a bucket can change its length. So we need to check
   afterwards if we still can split the bucket and if not might do a read
   on the next bucket until we are fine.
 
 3. We should do the read bucket thing for the last bucket as well.
 
 I think it would be best if you could commit this PoC patch as is (or with 
 adjustments
 according to the comments above) to trunk and we polish it up in trunk
 until it is fine.
 
 Regards
 
 Rüdiger
 



Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
Now that the memory utilz is being fixed, we need to determine
what sort of usage we want to allow… I'm guessing that people
are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
as the guiding spec?

RE: Fixing Ranges

2011-08-25 Thread Plüm, Rüdiger, VF-Group
+1 for 2.3, -0 for 2.2. I guess for 2.2 we should only detect misuse (the 
definition of misuse
needs to configurable) and reply with a 200 if misuse is detected.

misuse would be

- too much ranges [Let the number be configurable with a sane default e.g. 100 
and a possible setting of 0 for unlimited]
- overlapping ranges (after sorting) [Let this detector be configurable with a 
default to on]

For 2.3 the last one could be 3 state:

off - Don't do anything about that
on - reply with 200 if misuse is detected.
optimize - Do sorts and merges and fill too small chunks between the ranges.

Default for 2.3 would be optimize.


Regards

Rüdiger 

 -Original Message-
 From: Jim Jagielski [mailto:j...@jagunet.com] 
 Sent: Donnerstag, 25. August 2011 16:25
 To: dev@httpd.apache.org
 Subject: Re: Fixing Ranges
 
 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow... I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?
 


Truly minor inconsistency in mod_rangecnt.c

2011-08-25 Thread Tom Evans
Hi Dirk-Willem, list.

I wasn't sure whether to mail this in, it is inconsequential; the
module is supposed to count the number of ranges, but it actually
counts the number of commas between ranges, leading to an off-by-one.
IE, a request with 6 ranges would not be rejected, where as the code
has #define MAXRANGEHEADERS (5).

Its truly minor, but made my test tool to determine whether a server
is vulnerable to give some false positives, as it was sending 5 ranges
and expecting a 417.

Cheers

Tom


Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
I'm playing around w/ ap_set_byterange() for the merging and
detection part, but that should not hold up release with the
optimized code…

I can do a 2.2.10 release with the byte range stuff once we
agree on the back port and confirm it fixes the problem...

Re: Fixing Ranges

2011-08-25 Thread Greg Ames
On Thu, Aug 25, 2011 at 10:25 AM, Jim Jagielski j...@jagunet.com wrote:

 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow… I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?


I'm no longer convinced that we *need* to merge/optimize the Range header.
Stefan's fix to the brigade plumbing has converted O(N^2) memory + CPU use
into something scalable.

Think about the effect of this attack on the CPU cache.  My laptop has a
Core i7 with a 64kB L1 data cache.  The cache lines are 64 bytes, so I have
1024 of them per core.  The Range header I'm using has ~1300 byterange
specifiers.  apr_brigade_partition in this case will fragment the brigade
into ~1300 buckets with length 1 plus one or two for the remainder of the
file.  Each of those has to be read into the L1 cache somewhere as
apr_bucket_split walks thru the bucket structures.  That fills the cache
with junk and throws out the useful information, like the request_req,
conn_req, the filter structures, and most of the stack.  So not only will
the apr_brigade_partition calls perform like a dog but the mainline code
will suffer too.

Greg


Re: Fixing Ranges

2011-08-25 Thread William A. Rowe Jr.
On 8/25/2011 11:45 AM, Greg Ames wrote:
 On Thu, Aug 25, 2011 at 10:25 AM, Jim Jagielski j...@jagunet.com 
 mailto:j...@jagunet.com
 wrote:
 
 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow… I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?
 
 
 I'm no longer convinced that we *need* to merge/optimize the Range header.  
 Stefan's fix
 to the brigade plumbing has converted O(N^2) memory + CPU use into something 
 scalable. 
 
 Think about the effect of this attack on the CPU cache.  My laptop has a Core 
 i7 with a
 64kB L1 data cache.  The cache lines are 64 bytes, so I have 1024 of them per 
 core.  The
 Range header I'm using has ~1300 byterange specifiers.  apr_brigade_partition 
 in this case
 will fragment the brigade into ~1300 buckets with length 1 plus one or two 
 for the
 remainder of the file.  Each of those has to be read into the L1 cache 
 somewhere as
 apr_bucket_split walks thru the bucket structures.  That fills the cache with 
 junk and
 throws out the useful information, like the request_req, conn_req, the filter 
 structures,
 and most of the stack.  So not only will the apr_brigade_partition calls 
 perform like a
 dog but the mainline code will suffer too.

For the raw data you are right... but this is insignificant compared to the
CPU and network cost of assembling and transmitting all of the individual
range delimiters/headers.


Re: Fixing Ranges

2011-08-25 Thread William A. Rowe Jr.
On 8/25/2011 11:24 AM, Jim Jagielski wrote:
 I'm playing around w/ ap_set_byterange() for the merging and
 detection part, but that should not hold up release with the
 optimized code…
 
 I can do a 2.2.10 release with the byte range stuff once we
 agree on the back port and confirm it fixes the problem...

I guess I'm a bit confused... so the net brigade/range patch should
be something reasonable anybody can apply to 2.0 / 2.2.  Let's get
that net patch published as a fresh advisory by the end of the day?

There doesn't seem to be a really good reason to release half of
the solution, if many of us agree that 'something more' should be
done, but it will take not only our consensus, but the http-wg group
server authors to find consensus on how servers will react to extra
quirky range requests starting at least in August '11.

I'd rather see 2.2.10 implement that entire solution, even if this
takes us a week.  Allowing 1-100,900-999,1-100,900-999 remains a
DoS, even if this is a trivial one, because the resources returned
exceed the reasonable resources required in bandwidth, cpu, even if
we never wasted memory.


Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
I agree we don't *need* to, but we should… 

And we do (now)

On Aug 25, 2011, at 12:45 PM, Greg Ames wrote:

 On Thu, Aug 25, 2011 at 10:25 AM, Jim Jagielski j...@jagunet.com wrote:
 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow… I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?
 
 I'm no longer convinced that we *need* to merge/optimize the Range header.  
 Stefan's fix to the brigade plumbing has converted O(N^2) memory + CPU use 
 into something scalable.  
 
 Think about the effect of this attack on the CPU cache.  My laptop has a Core 
 i7 with a 64kB L1 data cache.  The cache lines are 64 bytes, so I have 1024 
 of them per core.  The Range header I'm using has ~1300 byterange specifiers. 
  apr_brigade_partition in this case will fragment the brigade into ~1300 
 buckets with length 1 plus one or two for the remainder of the file.  Each of 
 those has to be read into the L1 cache somewhere as apr_bucket_split walks 
 thru the bucket structures.  That fills the cache with junk and throws out 
 the useful information, like the request_req, conn_req, the filter 
 structures, and most of the stack.  So not only will the 
 apr_brigade_partition calls perform like a dog but the mainline code will 
 suffer too.
 
 Greg



Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 1:45 PM, William A. Rowe Jr. wrote:

 On 8/25/2011 11:24 AM, Jim Jagielski wrote:
 I'm playing around w/ ap_set_byterange() for the merging and
 detection part, but that should not hold up release with the
 optimized code…
 
 I can do a 2.2.10 release with the byte range stuff once we
 agree on the back port and confirm it fixes the problem...
 
 I guess I'm a bit confused... so the net brigade/range patch should
 be something reasonable anybody can apply to 2.0 / 2.2.  Let's get
 that net patch published as a fresh advisory by the end of the day?
 
 There doesn't seem to be a really good reason to release half of
 the solution, if many of us agree that 'something more' should be
 done, but it will take not only our consensus, but the http-wg group
 server authors to find consensus on how servers will react to extra
 quirky range requests starting at least in August '11.
 
 I'd rather see 2.2.10 implement that entire solution, even if this
 takes us a week.  Allowing 1-100,900-999,1-100,900-999 remains a
 DoS, even if this is a trivial one, because the resources returned
 exceed the reasonable resources required in bandwidth, cpu, even if
 we never wasted memory.
 

Right now, all adjacent overlaps are merged, and it is
trivial to add counting the number of merges done (will
add soonish) as well as the number of direction changes
(100-200,20-30 for eg)…

We can then place limits on them, expose/log them, etc...

Re: Fixing Ranges

2011-08-25 Thread Dirk-Willem van Gulik

On 25 Aug 2011, at 15:48, Plüm, Rüdiger, VF-Group wrote:

 +1 for 2.3, -0 for 2.2. I guess for 2.2 we should only detect misuse (the 
 definition of misuse
 needs to configurable) and reply with a 200 if misuse is detected.

Ok - given the patch a good test - and actually - am not sure we need even that 
(much) abuse detection. 

I find it darn hard to kill the server now; and would almost say that we can 
leave it as is. And PERHAPS have an optional cap on the number of ranges (Set 
at unlimited by default -leaving it to LimitRequestFieldSize to cap it off at 
effectively some 1200-1500).

Thanks

Dw

Re: Fixing Ranges

2011-08-25 Thread Dirk-Willem van Gulik

On 25 Aug 2011, at 17:45, Greg Ames wrote:

 On Thu, Aug 25, 2011 at 10:25 AM, Jim Jagielski j...@jagunet.com wrote:
 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow… I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?
 
 I'm no longer convinced that we *need* to merge/optimize the Range header.  
 Stefan's fix to the brigade plumbing has converted O(N^2) memory + CPU use 
 into something scalable.  

Same feeling here. BUT that is based on the FreeBSD platform I care about. And 
we have noticed earlier that Ubuntu (which its default commit pattern in the 
vm) behaves very differently. 

So is it fair to assume this reasoning holds across the board -- or should we 
get some very explicit +1's on this appraoch from key linux, windows and what 
not users ?

Dw

Re: Truly minor inconsistency in mod_rangecnt.c

2011-08-25 Thread Dirk-Willem van Gulik

On 25 Aug 2011, at 15:53, Tom Evans wrote:

 I wasn't sure whether to mail this in, it is inconsequential; the
 module is supposed to count the number of ranges, but it actually
 counts the number of commas between ranges, leading to an off-by-one.
 IE, a request with 6 ranges would not be rejected, where as the code
 has #define MAXRANGEHEADERS (5).

Yup - spot on - that is indeed a bug. And actually - with what we know
now - that number should probably be a 100 or so.

 Its truly minor, but made my test tool to determine whether a server
 is vulnerable to give some false positives, as it was sending 5 ranges
 and expecting a 417.

But lets fix it fixed :)

Thanks!

Dw.


Re: Fixing Ranges

2011-08-25 Thread Tim Bannister
On 25 Aug 2011, at 15:48, Plüm, Rüdiger, VF-Group wrote:

 For 2.3 the last one could be 3 state:
 
 off - Don't do anything about that
 on - reply with 200 if misuse is detected.
 optimize - Do sorts and merges and fill too small chunks between the ranges.
 
 Default for 2.3 would be optimize.

I don't know exactly how the PDF plugins work, but we might expect requests 
like:

HEAD /foo HTTP/1.1
Host: xxx.example

to learn the content length, followed by:

GET /foo HTTP/1.1
Host: xxx.example
If-Range: bar
Range: bytes=0-4095,8384511-8388607,4096-8384510,8384512-

(hoping to read the indexes at the start and end of the document first, then 
fill in the rest).


A default that forces the clients back to seeing only the whole entity seems 
too strong, especially if httpd will now have better code to handle this case. 
Detecting misuse and handling that with a 200 still fine though.

I expect that clients exist which would get confused at having small chunks 
filled in.
For example, a client that expects either a multipart/byte-ranges response or a 
whole-entity 200 (because the server doesn't accept ranges). With the above 
“optimize”, the client instead gets a sorted and merged single-range response. 
Naive coding could have the client believe that it is seeing the whole entity 
rather than just a range.

…yes, such a client is badly written but badly written clients can and do 
exist. If httpd punishes their users unduly, httpd itself may attract some 
blame.

-- 
Tim Bannister – is...@jellybaby.net



Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 2:27 PM, Dirk-Willem van Gulik wrote:

 
 On 25 Aug 2011, at 17:45, Greg Ames wrote:
 
 On Thu, Aug 25, 2011 at 10:25 AM, Jim Jagielski j...@jagunet.com wrote:
 Now that the memory utilz is being fixed, we need to determine
 what sort of usage we want to allow… I'm guessing that people
 are ok with http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
 as the guiding spec?
 
 I'm no longer convinced that we *need* to merge/optimize the Range header.  
 Stefan's fix to the brigade plumbing has converted O(N^2) memory + CPU use 
 into something scalable.  
 
 Same feeling here. BUT that is based on the FreeBSD platform I care about. 
 And we have noticed earlier that Ubuntu (which its default commit pattern in 
 the vm) behaves very differently. 
 
 So is it fair to assume this reasoning holds across the board -- or should we 
 get some very explicit +1's on this appraoch from key linux, windows and what 
 not users ?
 

I just need to create and connect some runtime config directives
for the Range stuff and I think we should be mostly complete ;)



CVE-2011-3192 - NeXT update ?

2011-08-25 Thread Dirk-WIllem van Gulik
Folks,

What is wisdom? We have an updated version at 
people.apache.org/CVE-2011-3192.txt. 

i'd say, let's send this of day if we expect the full patch to take another 24+ 
hours. As there is a need for the i proved mitigations  And otherwise skip it 
and go to final ASAP?

What is your take ?

Thanks,

Dw.
-- 
Dirk-Willem van Gulik.

Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
For us to be able to control the number of overlaps, ranges,
etc will require an API bump to add an entry to server_rec.

Are we sure we want to do that for 2.2 or save that for trunk?

On Aug 25, 2011, at 3:12 PM, Jim Jagielski wrote:
 
 I just need to create and connect some runtime config directives
 for the Range stuff and I think we should be mostly complete ;)
 



Re: Fixing Ranges

2011-08-25 Thread William A. Rowe Jr.
On 8/25/2011 3:05 PM, Jim Jagielski wrote:
 For us to be able to control the number of overlaps, ranges,
 etc will require an API bump to add an entry to server_rec.
 
 Are we sure we want to do that for 2.2 or save that for trunk?

Let's -not- overload server_rec.

First, it's per-dir... it would be entirely sensible for us to
allow backwards-broken clients to access particular sorts of docs
in bizzaro ways (.pdf for example), denying stupid ranges on all
other documents (particularly generated/proxied content).

And there are probably going to be few overrides... set it up
globally and forget it, except for the edge cases.

But I suggest instead of abusing core config, let's create one
config section for byterange, which will not break binary ABI,
and should be infrequently merged.


Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Stefan Fritsch

On Thu, 25 Aug 2011, j...@apache.org wrote:


Author: jim
Date: Thu Aug 25 17:38:19 2011
New Revision: 1161661

URL: http://svn.apache.org/viewvc?rev=1161661view=rev
Log:
Merge in byteranges

Modified:
   httpd/httpd/trunk/modules/http/byterange_filter.c

Modified: httpd/httpd/trunk/modules/http/byterange_filter.c
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/byterange_filter.c?rev=1161661r1=1161660r2=1161661view=diff
==
--- httpd/httpd/trunk/modules/http/byterange_filter.c (original)
+++ httpd/httpd/trunk/modules/http/byterange_filter.c Thu Aug 25 17:38:19 2011




+if (in_merge) {
+continue;
+} else {
+char *nr = apr_psprintf(r-pool, % APR_OFF_T_FMT -% 
APR_OFF_T_FMT,
+ostart, oend);
+merged = apr_pstrcat(r-pool, merged, (num_ranges++ ? , : ), 
nr, NULL);
+}
}

+if (in_merge) {
+char *nr = apr_psprintf(r-pool, % APR_OFF_T_FMT -% APR_OFF_T_FMT,
+ostart, oend);
+merged = apr_pstrcat(r-pool, merged, (num_ranges++ ? , : ), nr, 
NULL);
+}


Is it really a good idea to first parse the range string (involving lots 
of copying with ap_getword), then format it back into a string just to 
have it parsed by parse_byterange again? I would much prefer to have it 
parsed only once into an array of values and then do the merging in that 
array. This is more efficient and I think it would also lead to better 
readability. My original patch for merging/sorting had some code for that 
which we could reuse:


http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox/%3c201108240028.03308...@sfritsch.de%3E



Re: Fixing Ranges

2011-08-25 Thread Jim Jagielski
Using stef's byterange4 test, I'm seeing:

apr_brigade_length (bb=0x7feb00a23200, read_all=1, length=0x7fff6e03e8b0) at 
apr_brigade.c:201
201 if (bkt-length == (apr_size_t)(-1)) {
(gdb) where
#0  apr_brigade_length (bb=0x7feb00a23200, read_all=1, length=0x7fff6e03e8b0) 
at apr_brigade.c:201
#1  0x00010e500851 in copy_brigade_range (bb=0x7feb00a24d90, 
bbout=0x7feb00a23200, start=0, end=200) at byterange_filter.c:306
#2  0x00010e500d1b in ap_byterange_filter (f=0x7feb00a291b0, 
bb=0x7feb00a24d90) at byterange_filter.c:401
#3  0x00010e46a67e in ap_pass_brigade (next=0x7feb00a291b0, 
bb=0x7feb00a24d90) at util_filter.c:533
#4  0x00010e556681 in session_output_filter (f=0x7feb00a24a98, 
in=0x7feb00a24d90) at mod_session.c:453
#5  0x00010e46a67e in ap_pass_brigade (next=0x7feb00a24a98, 
bb=0x7feb00a24d90) at util_filter.c:533
#6  0x00010e4cb387 in bucketeer_out_filter (f=0x7feb00a24ad0, 
bb=0x7feb00a24d48) at mod_bucketeer.c:98
#7  0x00010e46a67e in ap_pass_brigade (next=0x7feb00a24ad0, 
bb=0x7feb00a24d48) at util_filter.c:533
#8  0x00010e47e36f in default_handler (r=0x7feb00a27ea0) at core.c:4208

We seem to be spinning at:

AP_DEBUG_ASSERT(APR_SUCCESS == apr_brigade_length(bbout, 1, pofft));

Cannot recreate in the real world...

Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Joe Schaefer
+1, also has the advantage of not being a quadratic
allocator the way Jim's usage of apr_pstrcat is.






From: Stefan Fritsch s...@sfritsch.de
To: dev@httpd.apache.org
Sent: Thursday, August 25, 2011 4:56 PM
Subject: Re: svn commit: r1161661 - 
/httpd/httpd/trunk/modules/http/byterange_filter.c

On Thu, 25 Aug 2011, j...@apache.org wrote:

 Author: jim
 Date: Thu Aug 25 17:38:19 2011
 New Revision: 1161661
 
 URL: http://svn.apache.org/viewvc?rev=1161661view=rev
 Log:
 Merge in byteranges
 
 Modified:
    httpd/httpd/trunk/modules/http/byterange_filter.c
 
 Modified: httpd/httpd/trunk/modules/http/byterange_filter.c
 URL: 
 http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/http/byterange_filter.c?rev=1161661r1=1161660r2=1161661view=diff
 ==
 --- httpd/httpd/trunk/modules/http/byterange_filter.c (original)
 +++ httpd/httpd/trunk/modules/http/byterange_filter.c Thu Aug 25 17:38:19 
 2011


 +        if (in_merge) {
 +            continue;
 +        } else {
 +            char *nr = apr_psprintf(r-pool, % APR_OFF_T_FMT -% 
 APR_OFF_T_FMT,
 +                                    ostart, oend);
 +            merged = apr_pstrcat(r-pool, merged, (num_ranges++ ? , : 
 ), nr, NULL);
 +        }
     }
 
 +    if (in_merge) {
 +        char *nr = apr_psprintf(r-pool, % APR_OFF_T_FMT -% 
 APR_OFF_T_FMT,
 +                                ostart, oend);
 +        merged = apr_pstrcat(r-pool, merged, (num_ranges++ ? , : ), 
 nr, NULL);
 +    }

Is it really a good idea to first parse the range string (involving lots of 
copying with ap_getword), then format it back into a string just to have it 
parsed by parse_byterange again? I would much prefer to have it parsed only 
once into an array of values and then do the merging in that array. This is 
more efficient and I think it would also lead to better readability. My 
original patch for merging/sorting had some code for that which we could reuse:

http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox/%3c201108240028.03308...@sfritsch.de%3E





Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 4:56 PM, Stefan Fritsch wrote:

 
 Is it really a good idea to first parse the range string (involving lots of 
 copying with ap_getword), then format it back into a string just to have it 
 parsed by parse_byterange again?

It is if we want to be able to create a correct r-range element
with ap_set_byterange()

Certainly that is where we should be creating the canonical
Range: we are using (or figuring out whether to punt it).
 
 I would much prefer to have it parsed only once into an array of values and 
 then do the merging in that array. This is more efficient and I think it 
 would also lead to better readability. My original patch for merging/sorting 
 had some code for that which we could reuse:
 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox/%3c201108240028.03308...@sfritsch.de%3E
 



Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 5:02 PM, Joe Schaefer wrote:

 +1, also has the advantage of not being a quadratic
 allocator the way Jim's usage of apr_pstrcat is.
 

So what, exactly, will ap_set_byterange() do…? It was
my impression that it created our r-range entry...

Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Jim Jagielski

On Aug 25, 2011, at 4:56 PM, Stefan Fritsch wrote:

 
 Is it really a good idea to first parse the range string (involving lots of 
 copying with ap_getword), then format it back into a string just to have it 
 parsed by parse_byterange again? I would much prefer to have it parsed only 
 once into an array of values and then do the merging in that array. This is 
 more efficient and I think it would also lead to better readability. My 
 original patch for merging/sorting had some code for that which we could 
 reuse:
 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201108.mbox/%3c201108240028.03308...@sfritsch.de%3E
 

Be my guest. commit and fix rp's concerns.

Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Joe Schaefer
My criticism has to do with your implementation.
There's no point in fixing exploitable code with
a differently exploitable implementation.  Just
buffer things in an internal array and merge the
string once at the end of the loop, and *not* as
you iterate over the elements of the range header.





From: Jim Jagielski j...@jagunet.com
To: dev@httpd.apache.org
Sent: Thursday, August 25, 2011 5:10 PM
Subject: Re: svn commit: r1161661 - 
/httpd/httpd/trunk/modules/http/byterange_filter.c


On Aug 25, 2011, at 5:02 PM, Joe Schaefer wrote:

 +1, also has the advantage of not being a quadratic
 allocator the way Jim's usage of apr_pstrcat is.
 

So what, exactly, will ap_set_byterange() do…? It was
my impression that it created our r-range entry...



Re: Fixing Ranges

2011-08-25 Thread Roy T. Fielding
On Aug 25, 2011, at 2:02 PM, Jim Jagielski wrote:

 Using stef's byterange4 test, I'm seeing:
 
 apr_brigade_length (bb=0x7feb00a23200, read_all=1, length=0x7fff6e03e8b0) at 
 apr_brigade.c:201
 201   if (bkt-length == (apr_size_t)(-1)) {

apr_size_t is unsigned.  That's borked.

Roy



Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Jim Jagielski
And who said this was the final version? Not me...

My plan was to simply add elements to an array and then
apr_array_pstrcat()...

On Thu, Aug 25, 2011 at 02:14:05PM -0700, Joe Schaefer wrote:
My criticism has to do with your implementation.
There's no point in fixing exploitable code with
a differently exploitable implementation.  Just
buffer things in an internal array and merge the
string once at the end of the loop, and *not* as
you iterate over the elements of the range header.
 
--
 
  From: Jim Jagielski j...@jagunet.com
  To: dev@httpd.apache.org
  Sent: Thursday, August 25, 2011 5:10 PM
  Subject: Re: svn commit: r1161661 -
  /httpd/httpd/trunk/modules/http/byterange_filter.c
 
  On Aug 25, 2011, at 5:02 PM, Joe Schaefer wrote:
 
   +1, also has the advantage of not being a quadratic
   allocator the way Jim's usage of apr_pstrcat is.
  
 
  So what, exactly, will ap_set_byterange() do***? It was
  my impression that it created our r-range entry...

-- 
===
   Jim Jagielski   [|]   j...@jagunet.com   [|]   http://www.jaguNET.com/
Great is the guilt of an unnecessary war  ~ John Adams


Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Joe Schaefer
Look Jim, as someone who has written
a HTTP parsing library with a good track
record of no published security holes,
take my criticism of what you wrote for
what it's worth.  I have no idea what
your plans are, was simply agreeing with
Stefan's critique of your code.





From: Jim Jagielski j...@jagunet.com
To: dev@httpd.apache.org; Joe Schaefer joe_schae...@yahoo.com
Sent: Thursday, August 25, 2011 5:29 PM
Subject: Re: svn commit: r1161661 - 
/httpd/httpd/trunk/modules/http/byterange_filter.c

And who said this was the final version? Not me...

My plan was to simply add elements to an array and then
apr_array_pstrcat()...

On Thu, Aug 25, 2011 at 02:14:05PM -0700, Joe Schaefer wrote:
    My criticism has to do with your implementation.
    There's no point in fixing exploitable code with
    a differently exploitable implementation.  Just
    buffer things in an internal array and merge the
    string once at the end of the loop, and *not* as
    you iterate over the elements of the range header.
 
    --
 
      From: Jim Jagielski j...@jagunet.com
      To: dev@httpd.apache.org
      Sent: Thursday, August 25, 2011 5:10 PM
      Subject: Re: svn commit: r1161661 -
      /httpd/httpd/trunk/modules/http/byterange_filter.c
 
      On Aug 25, 2011, at 5:02 PM, Joe Schaefer wrote:
 
       +1, also has the advantage of not being a quadratic
       allocator the way Jim's usage of apr_pstrcat is.
      
 
      So what, exactly, will ap_set_byterange() do***? It was
      my impression that it created our r-range entry...

-- 
===
   Jim Jagielski   [|]  j...@jagunet.com   [|]  http://www.jaguNET.com/
        Great is the guilt of an unnecessary war  ~ John Adams




Re: Fixing Ranges

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Roy T. Fielding wrote:
 On Aug 25, 2011, at 2:02 PM, Jim Jagielski wrote:
  Using stef's byterange4 test, I'm seeing:
  
  apr_brigade_length (bb=0x7feb00a23200, read_all=1,
  length=0x7fff6e03e8b0) at apr_brigade.c:201 201 if
  (bkt-length == (apr_size_t)(-1)) {
 
 apr_size_t is unsigned.  That's borked.

Still, that's how it's implemented in apr-util:

/** The length of the data in the bucket.  This could have been 
implemented
 *  with a function, but this is an optimization, because the most
 *  common thing to do will be to get the length.  If the length 
is unknown,
 *  the value of this field will be (apr_size_t)(-1).
 */
apr_size_t length;

Maybe it should actually be APR_SIZE_T_MAX...


Re: Fixing Ranges

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Jim Jagielski wrote:
 We seem to be spinning at:
 
   AP_DEBUG_ASSERT(APR_SUCCESS == apr_brigade_length(bbout, 1,
 pofft));
 
 Cannot recreate in the real world...

I have managed to fix that one. The key to recreating is having 
mod_bucketeer configured and requesting 
t/htdocs/apache/chunked/byteranges.txt.

But there are still more failures in byterange4.t. Yesterday's trunk 
passes, so these are new bugs, too. Investigating.


Re: CVE-2011-3192 - NeXT update ?

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Dirk-WIllem van Gulik wrote:
 Folks,
 
 What is wisdom? We have an updated version at
 people.apache.org/CVE-2011-3192.txt.
 
 i'd say, let's send this of day if we expect the full patch to take
 another 24+ hours. As there is a need for the i proved mitigations
  And otherwise skip it and go to final ASAP?
 
 What is your take ?

There are still plenty of bugs in the new code, so I am not confident 
that it will be ready within 24 hours.


Re: svn commit: r1161767 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Greg Ames
On Thu, Aug 25, 2011 at 5:43 PM, s...@apache.org wrote:


 avoid inserting the same bucket into bbout twice, causing an endless loop

 --- httpd/httpd/trunk/modules/http/byterange_filter.c (original)
 +++ httpd/httpd/trunk/modules/http/byterange_filter.c Thu Aug 25 21:43:32
 2011
 @@ -195,8 +195,8 @@ static apr_status_t copy_brigade_range(a
 return rv;
 }

 -APR_BRIGADE_INSERT_TAIL(bbout, copy);
 if (e == first) {
 +APR_BRIGADE_INSERT_TAIL(bbout, copy);
 if (off_first != start64) {
 rv = apr_bucket_split(copy, (apr_size_t)(start64 -
 off_first));
 if (rv == APR_ENOTIMPL) {


?  Isn't copy going to be a new bucket on each pass of the while() loop?
Suppose we have 3 buckets and need to split buckets #1 and #3 to satisfy the
range.  We also need a copy of bucket #2 in the output brigade.  I don't see
where it is happening with this patch.

Greg


Re: svn commit: r1161767 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Stefan Fritsch
On Friday 26 August 2011, Greg Ames wrote:
 ?  Isn't copy going to be a new bucket on each pass of the
 while() loop? Suppose we have 3 buckets and need to split buckets
 #1 and #3 to satisfy the range.  We also need a copy of bucket #2
 in the output brigade.  I don't see where it is happening with
 this patch.

True, that patch is broken, too :-( Will fix in a minute.


Re: svn commit: r1161661 - /httpd/httpd/trunk/modules/http/byterange_filter.c

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Jim Jagielski wrote:
 Be my guest. commit and fix rp's concerns.

OK, done that. Trunk r1161791 passes all tests, including the new 
byterange3.t and byterange4.t.


Re: CVE-2011-3192 - NeXT update ?

2011-08-25 Thread Stefan Fritsch
On Thursday 25 August 2011, Stefan Fritsch wrote:
 On Thursday 25 August 2011, Dirk-WIllem van Gulik wrote:
  Folks,
  
  What is wisdom? We have an updated version at
  people.apache.org/CVE-2011-3192.txt.
  
  i'd say, let's send this of day if we expect the full patch to
  take another 24+ hours. As there is a need for the i proved
  mitigations
  
   And otherwise skip it and go to final ASAP?
  
  What is your take ?
 
 There are still plenty of bugs in the new code, so I am not
 confident that it will be ready within 24 hours.

Looks better now. But I would be even more comfortable if there was a 
test for the apr_bucket_read() parts. Does anybody have an idea how to 
test that?

In any case, I won't continue on this until tomorrow.