Issues with accessing Apache Httpd Modules

[kruthika.krishnan]

Hi ,
I am trying to access the apache HTTPD modules for Apache version 2.4, through 
the URL
http://modules.apache.org/
It shows an error:
Service Unavailable
The server is temporarily unable to service your request due to maintenance 
downtime or capacity problems. Please try again later.
Is there any other way to access the Apache modlues the modules I want to 
access are : mod_dav and mod_dav_fs.

Thanks and Regards,
Kruthika Krishnan



This message is for the designated recipient only and may contain privileged, 
proprietary, or otherwise confidential information. If you have received it in 
error, please notify the sender immediately and delete the original. Any other 
use of the e-mail by you is prohibited. Where allowed by local law, electronic 
communications with Accenture and its affiliates, including e-mail and instant 
messaging (including content), may be scanned by our systems for the purposes 
of information security and assessment of internal compliance with Accenture 
policy.
__

www.accenture.com

Re: Thread-safety of javax.servlet.Servlet#getServletConfig()

[andreas]

 On Fri, 25 Nov 2016 23:02:00 +0930 Péter Gergely Horváth  wrote  
>Hi All, 
> 
>I am wondering why calling javax.servlet.Servlet#getServletConfig() is 
>thread safe: if you check the implementation in 
> javax.servlet.GenericServlet from servlet API 3.0.1, you see the following: 
> 
>package javax.servlet; 
> 
>// lines omitted 
> 
>public abstract class GenericServlet 
> implements Servlet, ServletConfig, java.io.Serializable 
>{ 
> // lines omitted 
> 
> private transient ServletConfig config; 
> 
> // lines omitted 
> 
> public void init(ServletConfig config) throws ServletException { 
> this.config = config; 
> this.init(); 
> } 
> 
> // lines omitted 
> 
> public ServletConfig getServletConfig() { 
> return config; 
> } 
> // lines omitted 
>} 
> 
> 
>The field config is written in init(ServletConfig) without any 
>synchronization, locking or config being volatile, while getServletConfig() 
>could be called anytime from any later worker thread of the servlet 
>container. I took a quick look into Tomcat/Catalina code base, however I 
>see no indication of the servlet container performing any magic, which 
>would guarantee thread safe access to field config through 
>getServletConfig() from e.g. 
>javax.servlet.GenericServlet#service(ServletRequest, ServletResponse) and 
>the corresponding methods in javax.servlet.http.HttpServlet. 
> 
>Am I missing something here? What will guarantee that if Thread A is used 
>to init(ServletConfig) then Thread B calling getServletConfig() will see 
>the correct state of the field config (Java "happens-before"), and not e.g. 
>null? 
> 
>I am asking this because I have seen some legacy code, where a servlet 
>stored something into a field inside the init() method, which was then used 
>from within a javax.servlet.http.HttpServlet#doGet or 
>javax.servlet.http.HttpServlet#doPost method, without synchronization of 
>any kind like this: 
> 
>public class FoobarServlet extends HttpServlet { 
> 
> private FoobarService foobarService; 
> 
> @Override 
> public void init() throws ServletException { 
>this.foobarService = // ... acquire foobarService 
> } 
> protected void doGet(HttpServletRequest request, HttpServletResponse 
>response) throws ServletException, IOException { 
> List foobars = this.foobarService.getFoobars(); // read the 
>field WITHOUT synchronization 
> // ... 
> } 
> // ... 
>Is this approach (having no synchronization, locking or the field being 
>volatile) correct? I assumed it is not, seeing something like that in the 
>servlet API is quite confusing. 
> 
> 
>What do you think? 
> 
>Thanks, 
>Peter 


A Servlet will process requests only if it is fully initialized, i.e. init has 
been executed. The init method gets called only once and the servlet config 
won't change afterwards. I don't think there is need for synchronization. The 
same is probably valid for your own objects. Problems arise when individual 
requests change the state of these objects. 

Andy










-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [SECURITY] CVE-2016-6816 Apache Tomcat Information Disclosure

[Utkarsh Dave]

Please ignore my previous mail. I got the correct one
https://tomcat.apache.org/security-7.html



On Sun, Nov 27, 2016 at 6:41 PM, Utkarsh Dave 
wrote:

> Hi All
>
> This vulnerability (CVE-2016-6816) is said to be "Affects: 9.0.0.M1 to
> 9.0.0.M11" on another url https://tomcat.apache.org/security-9.html.
> But in the mail it says Tomcat 7 is also affected.
> Does this vulnerability affects version 7.0.72
>
> -Regards
> Utkarsh
>
> On Tue, Nov 22, 2016 at 1:42 AM, Mark Thomas  wrote:
>
>> CVE-2016-6816 Apache Tomcat Information Disclosure
>>
>> Severity: Important
>>
>> Vendor: The Apache Software Foundation
>>
>> Versions Affected:
>> Apache Tomcat 9.0.0.M1 to 9.0.0.M11
>> Apache Tomcat 8.5.0 to 8.5.6
>> Apache Tomcat 8.0.0.RC1 to 8.0.38
>> Apache Tomcat 7.0.0 to 7.0.72
>> Apache Tomcat 6.0.0 to 6.0.47
>> Earlier, unsupported versions may also be affected.
>>
>> Description
>> The code that parsed the HTTP request line permitted invalid characters.
>> This could be exploited, in conjunction with a proxy that also permitted
>> the invalid characters but with a different interpretation, to inject
>> data into the HTTP response. By manipulating the HTTP response the
>> attacker could poison a web-cache, perform an XSS attack and/or obtain
>> sensitive information from requests other then their own.
>>
>> Mitigation
>> Users of affected versions should apply one of the following mitigations
>> - Upgrade to Apache Tomcat 9.0.0.M13 or later
>>   (Apache Tomcat 9.0.0.M12 has the fix but was not released)
>> - Upgrade to Apache Tomcat 8.5.8 or later
>>   (Apache Tomcat 8.5.7 has the fix but was not released)
>> - Upgrade to Apache Tomcat 8.0.39 or later
>> - Upgrade to Apache Tomcat 7.0.73 or later
>> - Upgrade to Apache Tomcat 6.0.48 or later
>>
>> Credit:
>> This issue was discovered by Regis Leroy from Makina Corpus.
>>
>> References:
>> [1] http://tomcat.apache.org/security-9.html
>> [2] http://tomcat.apache.org/security-8.html
>> [3] http://tomcat.apache.org/security-7.html
>> [4] http://tomcat.apache.org/security-6.html
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
>

Re: [SECURITY] CVE-2016-6816 Apache Tomcat Information Disclosure

[Utkarsh Dave]

Hi All

This vulnerability (CVE-2016-6816) is said to be "Affects: 9.0.0.M1 to
9.0.0.M11" on another url https://tomcat.apache.org/security-9.html.
But in the mail it says Tomcat 7 is also affected.
Does this vulnerability affects version 7.0.72

-Regards
Utkarsh

On Tue, Nov 22, 2016 at 1:42 AM, Mark Thomas  wrote:

> CVE-2016-6816 Apache Tomcat Information Disclosure
>
> Severity: Important
>
> Vendor: The Apache Software Foundation
>
> Versions Affected:
> Apache Tomcat 9.0.0.M1 to 9.0.0.M11
> Apache Tomcat 8.5.0 to 8.5.6
> Apache Tomcat 8.0.0.RC1 to 8.0.38
> Apache Tomcat 7.0.0 to 7.0.72
> Apache Tomcat 6.0.0 to 6.0.47
> Earlier, unsupported versions may also be affected.
>
> Description
> The code that parsed the HTTP request line permitted invalid characters.
> This could be exploited, in conjunction with a proxy that also permitted
> the invalid characters but with a different interpretation, to inject
> data into the HTTP response. By manipulating the HTTP response the
> attacker could poison a web-cache, perform an XSS attack and/or obtain
> sensitive information from requests other then their own.
>
> Mitigation
> Users of affected versions should apply one of the following mitigations
> - Upgrade to Apache Tomcat 9.0.0.M13 or later
>   (Apache Tomcat 9.0.0.M12 has the fix but was not released)
> - Upgrade to Apache Tomcat 8.5.8 or later
>   (Apache Tomcat 8.5.7 has the fix but was not released)
> - Upgrade to Apache Tomcat 8.0.39 or later
> - Upgrade to Apache Tomcat 7.0.73 or later
> - Upgrade to Apache Tomcat 6.0.48 or later
>
> Credit:
> This issue was discovered by Regis Leroy from Makina Corpus.
>
> References:
> [1] http://tomcat.apache.org/security-9.html
> [2] http://tomcat.apache.org/security-8.html
> [3] http://tomcat.apache.org/security-7.html
> [4] http://tomcat.apache.org/security-6.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Apache/Tomcat vulnerability

[tomcat]

On 27.11.2016 19:03, Jaaz Portal wrote:

2016-11-27 18:30 GMT+01:00 André Warnier (tomcat) :


On 27.11.2016 14:26, Jaaz Portal wrote:


hi,
everything i know so far is just this single log line that appeared in
apache error.log

[Fri Nov 25 13:08:00.647835 2016] [mpm_event:error] [pid 13385:tid
1397934896385
92] AH00484: server reached MaxRequestWorkers setting, consider raising
the
MaxR
equestWorkers setting

there was nothing else, just this strange line



This is not a "strange" line. It is telling you something.
One problem is that you seem convinced in advance, without serious proof,
that there is a "bug" or a vulnerability in httpd or tomcat.
Read the explanation of the httpd parameter, here :
http://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxrequestworkers
and try to understand it.



I understand it very well. We are serving no more that 500 clients per day
so there was no other option that some kind of attack.


About the "bug" or "vulnerability" : a webserver is supposed to serve HTTP

requests from clients.  That is what you expect of it. It has no choice but
to accept any client connection and request, up to the maximum it can
handle considering its configuration and the capacity of the system on
which it runs. That is not a bug, it is a feature.



We have some weeks ago come under attack from some Polish group. It was
first bind that was DoS'ed, we was running on stable Debian so i updated
bind to latest version. It did not helped. They has dos'ed it so we
switched to other dns provider. That has helped.

Then they exploited some well know vulnerability in mod_proxy. We have
updated apache to the latest but again they has exploited it, so we have
switched to mod_jk. And then guess what. They exploited it too so i decided
to write to this list looking for help before trying jetty.




The normal Apache httpd access log, will log a request only when it is
finished.  If the request never finishes, it will not get logged.
That may be why you do not see these requests in the log.
But have a look at this Apache httpd module :
http://httpd.apache.org/docs/2.4/mod/mod_log_forensic.html



ok, thanks, will take care

Note : that is also why I was telling you to enable the mod_jk log, and to

examine it.
Because mod_jk will also log information before the request produces a
response.


and server hanged.

Again, /what/ is "hanged" ? Apache httpd, or tomcat ?



Apache was accepting connection but not processed it. After restart of
tomcat server it worked again.



Also in


access logs there are no clues that it was under any heavy load.

after around hour after discovering that our server hanged-out we have
restarted tomcat server
and it worked again



Yes, because that will close all connections between Apache httpd and
tomcat, and abort all requests that are in the process of being processed
by tomcat. So mod_jk will get an error from tomcat, and will report an
error to httpd, and httpd will communicate that error to the clients, and
close their connection.
It still does not tell you what the problem was.
The only thing that it suggests, is that the "bad" requests actually make
it all the way to tomcat.



correct

i will enable logs that you has pointed out and we will look what i will
catch
however i think we have only one chance, as if the solution we has found
out (connection_timeout + mod_bn)
will work they will stop exploiting it

thank you very much for all the help and explanations
i will report to the list new facts once they will attack us again

best regards,
artur



Ok, but also read this e.g. :
https://www.corero.com/blog/695-going-after-the-people-behind-ddos-attacks.html
Attempts to bring down a site by DoS attacks is a crime, in most places.
You can report it, while at the same time trying to defend yourself against it.

It is also relatively easy, and quite inexpensive in terms of system resources, to run a 
small shell script which takes a list every few seconds of the connections to the port of 
your webserver, and which IPs they are coming *from*.

E.g.
First try the netstat command, to see what it lists, like :
# netstat -n --tcp | more

Then you will want to filter this a bit, to only consider established connections to your 
webserver, for example :

# netstat -n --tcp | grep ":80" | grep "ESTABLISHED"

Then you will want to send this to a logfile, regularly, like this :

# date >> some_file.log
# netstat -n --tcp | grep ":80" | grep "ESTABLISHED" >> some_file.log
(repeat every 3 seconds)

This will not generate GB of logfiles, and it will tell you, when the problem happens, how 
many connections there are exactly to your webserver, and where they are coming from.

Then later you can further analyse this information..





i think that setting connection-timeout and limiting the number of clients
by mod_bd i will
have effect that next time somebody will try this exploit it will block
his
access to the site
for minute or two, pretty good holistic 

Re: Apache/Tomcat vulnerability

[Jaaz Portal]

2016-11-27 18:30 GMT+01:00 André Warnier (tomcat) :

> On 27.11.2016 14:26, Jaaz Portal wrote:
>
>> hi,
>> everything i know so far is just this single log line that appeared in
>> apache error.log
>>
>> [Fri Nov 25 13:08:00.647835 2016] [mpm_event:error] [pid 13385:tid
>> 1397934896385
>> 92] AH00484: server reached MaxRequestWorkers setting, consider raising
>> the
>> MaxR
>> equestWorkers setting
>>
>> there was nothing else, just this strange line
>>
>
> This is not a "strange" line. It is telling you something.
> One problem is that you seem convinced in advance, without serious proof,
> that there is a "bug" or a vulnerability in httpd or tomcat.
> Read the explanation of the httpd parameter, here :
> http://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxrequestworkers
> and try to understand it.
>

I understand it very well. We are serving no more that 500 clients per day
so there was no other option that some kind of attack.


About the "bug" or "vulnerability" : a webserver is supposed to serve HTTP
> requests from clients.  That is what you expect of it. It has no choice but
> to accept any client connection and request, up to the maximum it can
> handle considering its configuration and the capacity of the system on
> which it runs. That is not a bug, it is a feature.
>
>
We have some weeks ago come under attack from some Polish group. It was
first bind that was DoS'ed, we was running on stable Debian so i updated
bind to latest version. It did not helped. They has dos'ed it so we
switched to other dns provider. That has helped.

Then they exploited some well know vulnerability in mod_proxy. We have
updated apache to the latest but again they has exploited it, so we have
switched to mod_jk. And then guess what. They exploited it too so i decided
to write to this list looking for help before trying jetty.


>
> The normal Apache httpd access log, will log a request only when it is
> finished.  If the request never finishes, it will not get logged.
> That may be why you do not see these requests in the log.
> But have a look at this Apache httpd module :
> http://httpd.apache.org/docs/2.4/mod/mod_log_forensic.html
>

ok, thanks, will take care

Note : that is also why I was telling you to enable the mod_jk log, and to
> examine it.
> Because mod_jk will also log information before the request produces a
> response.
>
>
> and server hanged.
>
> Again, /what/ is "hanged" ? Apache httpd, or tomcat ?
>

Apache was accepting connection but not processed it. After restart of
tomcat server it worked again.


> Also in
>
>> access logs there are no clues that it was under any heavy load.
>>
>> after around hour after discovering that our server hanged-out we have
>> restarted tomcat server
>> and it worked again
>>
>
> Yes, because that will close all connections between Apache httpd and
> tomcat, and abort all requests that are in the process of being processed
> by tomcat. So mod_jk will get an error from tomcat, and will report an
> error to httpd, and httpd will communicate that error to the clients, and
> close their connection.
> It still does not tell you what the problem was.
> The only thing that it suggests, is that the "bad" requests actually make
> it all the way to tomcat.
>

correct

i will enable logs that you has pointed out and we will look what i will
catch
however i think we have only one chance, as if the solution we has found
out (connection_timeout + mod_bn)
will work they will stop exploiting it

thank you very much for all the help and explanations
i will report to the list new facts once they will attack us again

best regards,
artur



>> i think that setting connection-timeout and limiting the number of clients
>> by mod_bd i will
>> have effect that next time somebody will try this exploit it will block
>> his
>> access to the site
>> for minute or two, pretty good holistic solution i would say
>>
>> still, it seems that there is serious vulnerability somewhere in apache,
>> mod_jk or tomcat
>> i would like to help find it out but need some hints which debug options
>> enable to catch the bad guys
>> when they will try next time
>>
>> best regards,
>> artur
>>
>> 2016-11-27 13:58 GMT+01:00 André Warnier (tomcat) :
>>
>> On 27.11.2016 13:23, Jaaz Portal wrote:
>>>
>>> hi Andre,
 thank you very much this was very educative but in my case it is little
 bit
 different.
 The server is no flooded, there is maybe dozen of very sophisticated
 connections that somehow hangs apache workers threads


>>> Can you be a bit more specific ?
>>> When you say "apache workers threads", do you mean threads in Apache
>>> httpd, or threads in Apache Tomcat ? (both are Apache webservers, so it
>>> is
>>> difficult to tell what you are talking about, unless you are very
>>> precise).
>>>
>>> Let me give you some additional explanations, and maybe you can figure
>>> out
>>> exactly where it "hangs".
>>>
>>>  From the Apache httpd 

Re: Apache/Tomcat vulnerability

[tomcat]

On 27.11.2016 14:26, Jaaz Portal wrote:

hi,
everything i know so far is just this single log line that appeared in
apache error.log

[Fri Nov 25 13:08:00.647835 2016] [mpm_event:error] [pid 13385:tid
1397934896385
92] AH00484: server reached MaxRequestWorkers setting, consider raising the
MaxR
equestWorkers setting

there was nothing else, just this strange line


This is not a "strange" line. It is telling you something.
One problem is that you seem convinced in advance, without serious proof, that there is a 
"bug" or a vulnerability in httpd or tomcat.

Read the explanation of the httpd parameter, here :
http://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxrequestworkers
and try to understand it.

If Apache httpd is writing that line to the log, it means that it has reached the maximum 
number of requests that it is allowed to serve at the same time, considering your 
configuration. That could be normal - if your server is supposed to be heavily loaded - or 
it could be the sign of a "flood" attack. But you seem to be convinced in advance that 
there is no flood. You also do not really answer the questions which you are asked, which 
makes helping you difficult.


About the "bug" or "vulnerability" : a webserver is supposed to serve HTTP requests from 
clients.  That is what you expect of it. It has no choice but to accept any client 
connection and request, up to the maximum it can handle considering its configuration and 
the capacity of the system on which it runs. That is not a bug, it is a feature.


It does not know in advance that these requests are bad. To find out that they are bad, it 
needs to accept the request and read it. For that, there needs to be a process or a thread 
available to do that.
If it accepts the request, and if the URL of the request is in principle valid, it will 
try to process that request.  If there is an application on your server responding to that 
request, then if it takes a long time or freezes the thread which processes the request, 
it may be a bug in your application.


(Note : there /may/ be an unknown vulnerability in Apache httpd or tomcat; but is not 
necessarily so. And there are millions of Apache webservers on the Internet; so if they 
had a bug like that, you would not be the only one to report it.  So be humble and 
realistic; try your best to find out what really happens, before you cry wolf).


The normal Apache httpd access log, will log a request only when it is finished.  If the 
request never finishes, it will not get logged.

That may be why you do not see these requests in the log.
But have a look at this Apache httpd module : 
http://httpd.apache.org/docs/2.4/mod/mod_log_forensic.html


Note : that is also why I was telling you to enable the mod_jk log, and to 
examine it.
Because mod_jk will also log information before the request produces a response.


and server hanged.

Again, /what/ is "hanged" ? Apache httpd, or tomcat ?

Also in

access logs there are no clues that it was under any heavy load.

after around hour after discovering that our server hanged-out we have
restarted tomcat server
and it worked again


Yes, because that will close all connections between Apache httpd and tomcat, and abort 
all requests that are in the process of being processed by tomcat. So mod_jk will get an 
error from tomcat, and will report an error to httpd, and httpd will communicate that 
error to the clients, and close their connection.

It still does not tell you what the problem was.
The only thing that it suggests, is that the "bad" requests actually make it all the way 
to tomcat.




i think that setting connection-timeout and limiting the number of clients
by mod_bd i will
have effect that next time somebody will try this exploit it will block his
access to the site
for minute or two, pretty good holistic solution i would say

still, it seems that there is serious vulnerability somewhere in apache,
mod_jk or tomcat
i would like to help find it out but need some hints which debug options
enable to catch the bad guys
when they will try next time

best regards,
artur

2016-11-27 13:58 GMT+01:00 André Warnier (tomcat) :


On 27.11.2016 13:23, Jaaz Portal wrote:


hi Andre,
thank you very much this was very educative but in my case it is little
bit
different.
The server is no flooded, there is maybe dozen of very sophisticated
connections that somehow hangs apache workers threads



Can you be a bit more specific ?
When you say "apache workers threads", do you mean threads in Apache
httpd, or threads in Apache Tomcat ? (both are Apache webservers, so it is
difficult to tell what you are talking about, unless you are very precise).

Let me give you some additional explanations, and maybe you can figure out
exactly where it "hangs".

 From the Apache httpd front-end point of view, mod_jk (the connector to
Apache Tomcat) is basically one among other "response generators".  Apache
httpd does not "know" that behind mod_jk, there is one 

Re: Apache/Tomcat vulnerability

[Jaaz Portal]

hi,
everything i know so far is just this single log line that appeared in
apache error.log

[Fri Nov 25 13:08:00.647835 2016] [mpm_event:error] [pid 13385:tid
1397934896385
92] AH00484: server reached MaxRequestWorkers setting, consider raising the
MaxR
equestWorkers setting

there was nothing else, just this strange line and server hanged. Also in
access logs there are no clues that it was under any heavy load.

after around hour after discovering that our server hanged-out we have
restarted tomcat server
and it worked again

i think that setting connection-timeout and limiting the number of clients
by mod_bd i will
have effect that next time somebody will try this exploit it will block his
access to the site
for minute or two, pretty good holistic solution i would say

still, it seems that there is serious vulnerability somewhere in apache,
mod_jk or tomcat
i would like to help find it out but need some hints which debug options
enable to catch the bad guys
when they will try next time

best regards,
artur

2016-11-27 13:58 GMT+01:00 André Warnier (tomcat) :

> On 27.11.2016 13:23, Jaaz Portal wrote:
>
>> hi Andre,
>> thank you very much this was very educative but in my case it is little
>> bit
>> different.
>> The server is no flooded, there is maybe dozen of very sophisticated
>> connections that somehow hangs apache workers threads
>>
>
> Can you be a bit more specific ?
> When you say "apache workers threads", do you mean threads in Apache
> httpd, or threads in Apache Tomcat ? (both are Apache webservers, so it is
> difficult to tell what you are talking about, unless you are very precise).
>
> Let me give you some additional explanations, and maybe you can figure out
> exactly where it "hangs".
>
> From the Apache httpd front-end point of view, mod_jk (the connector to
> Apache Tomcat) is basically one among other "response generators".  Apache
> httpd does not "know" that behind mod_jk, there is one or more Tomcat
> servers.  Apache httpd receives the original client request, and depending
> on the URL of the request, it will pass it to mod_jk or to another response
> generator, to generate the response to the request.
> That mod_jk in the background is using a Tomcat server to actually
> generate the response, is none of the business of Apache httpd, and it does
> not care. All it cares about, is to actually receive the response from
> mod_jk, and pass it back to the client.
>
> If httpd passes a request to mod_jk, it is because you have specified in
> the Apache configuration, the type of URL that it should pass to mod_jk.
> That happens via your "JkMount (URL pattern)" directives in Apache httpd.
>
> Of course Apache httpd will not pass a request to mod_jk, before it has
> received at least the URL of the request, and analysed it to determine *if*
> it should pass it to mod_jk (*).
>
> If the mod_jk logging is enabled, you can see in it, exactly *which*
> requests are passed to mod_jk and to Tomcat.
> Do you know *which* requests, from which clients, cause this "thread
> hanging" symptom ?
> Once you would know this, maybe you can design a strategy to block
> specifically these requests.
>
> and the effect is permanent. Quickly the pool is exhausted
>>
>
> which pool exactly ?
>
> and the only
>
>> solution that works is to restart tomcat.
>>
>> I think it is a bug similar to this one from mod_proxy:
>> https://tools.cisco.com/security/center/viewAlert.x?alertId=34971
>>
>>
> Maybe, maybe not. As long as we do not know what the requests are, that
> block things, we do not know this.
>
>
> I think also that your solution with setting connectionTimeout will solve
>> the problem, at least partially. THANK YOU.
>>
>
> Same thing, we do not know this yet.  It was only giving this explanation,
> to help you think about where the problem may be.
>
>
>> I would like to help you further investigate this issue, as our server
>> comes under such attack once or twice in a week.
>>
>>
> Other than giving you hints, there is not much I or anyone else can do to
> help. You are the one with control of your servers and logfiles, so you
> have to investigate and provide more precise information.
>
>
> (*) actually, to be precise, Apache httpd passes *all* requests to mod_jk,
> to ask it "if it wants that request". mod_jk then accepts or declines,
> depending on the JkMount instructions. If mod_jk declines, then Apache
> httpd will ask the next response generator if it wants this request, etc..
>
>
>
>
>
> best regards,
>> artur
>>
>> 2016-11-27 12:46 GMT+01:00 André Warnier (tomcat) :
>>
>> Hi.
>>>
>>> Have a look that the indicated parameters in the two pages below.
>>> You may be the target of such a variant of DDoS attack : many clients
>>> open
>>> a TCP connection to your server (front-end), but then never sends a HTTP
>>> request on that connection.  In the meantime, the server accepts the TCP
>>> connection, and passes it on to a "child" process or thread for
>>> 

Re: Apache/Tomcat vulnerability

[tomcat]

On 27.11.2016 13:23, Jaaz Portal wrote:

hi Andre,
thank you very much this was very educative but in my case it is little bit
different.
The server is no flooded, there is maybe dozen of very sophisticated
connections that somehow hangs apache workers threads


Can you be a bit more specific ?
When you say "apache workers threads", do you mean threads in Apache httpd, or threads in 
Apache Tomcat ? (both are Apache webservers, so it is difficult to tell what you are 
talking about, unless you are very precise).


Let me give you some additional explanations, and maybe you can figure out exactly where 
it "hangs".


From the Apache httpd front-end point of view, mod_jk (the connector to Apache Tomcat) is 
basically one among other "response generators".  Apache httpd does not "know" that behind 
mod_jk, there is one or more Tomcat servers.  Apache httpd receives the original client 
request, and depending on the URL of the request, it will pass it to mod_jk or to another 
response generator, to generate the response to the request.
That mod_jk in the background is using a Tomcat server to actually generate the response, 
is none of the business of Apache httpd, and it does not care. All it cares about, is to 
actually receive the response from mod_jk, and pass it back to the client.


If httpd passes a request to mod_jk, it is because you have specified in the Apache 
configuration, the type of URL that it should pass to mod_jk.

That happens via your "JkMount (URL pattern)" directives in Apache httpd.

Of course Apache httpd will not pass a request to mod_jk, before it has received at least 
the URL of the request, and analysed it to determine *if* it should pass it to mod_jk (*).


If the mod_jk logging is enabled, you can see in it, exactly *which* requests are passed 
to mod_jk and to Tomcat.

Do you know *which* requests, from which clients, cause this "thread hanging" 
symptom ?
Once you would know this, maybe you can design a strategy to block specifically these 
requests.



and the effect is permanent. Quickly the pool is exhausted


which pool exactly ?

and the only

solution that works is to restart tomcat.

I think it is a bug similar to this one from mod_proxy:
https://tools.cisco.com/security/center/viewAlert.x?alertId=34971



Maybe, maybe not. As long as we do not know what the requests are, that block things, we 
do not know this.




I think also that your solution with setting connectionTimeout will solve
the problem, at least partially. THANK YOU.


Same thing, we do not know this yet.  It was only giving this explanation, to help you 
think about where the problem may be.




I would like to help you further investigate this issue, as our server
comes under such attack once or twice in a week.



Other than giving you hints, there is not much I or anyone else can do to help. You are 
the one with control of your servers and logfiles, so you have to investigate and provide 
more precise information.



(*) actually, to be precise, Apache httpd passes *all* requests to mod_jk, to ask it "if 
it wants that request". mod_jk then accepts or declines, depending on the JkMount 
instructions. If mod_jk declines, then Apache httpd will ask the next response generator 
if it wants this request, etc..






best regards,
artur

2016-11-27 12:46 GMT+01:00 André Warnier (tomcat) :


Hi.

Have a look that the indicated parameters in the two pages below.
You may be the target of such a variant of DDoS attack : many clients open
a TCP connection to your server (front-end), but then never sends a HTTP
request on that connection.  In the meantime, the server accepts the TCP
connection, and passes it on to a "child" process or thread for
processing.  The child then waits for the HTTP request line to arrive on
the connection (during a certain time), but it never arrives. After a
while, this triggers a timeout (see below), but the standard value of that
timeout may be such that in the meantime, a lot of other connections have
been established by other such nefarious clients, so a lot of resources of
the webserver are tied up, waiting for something that will never come.
Since there is never any real request sent on the connection, you would
(probably) not see this in the logs either.

The above is the basic mechanism of such an attack.  There may be
variations, such as the client not "not sending" a request line, but
sending it extremely slowly, thus achieving perhaps similar kinds of
effects.

As someone pointed out, it is quite difficult to do something about this
at the level of the webserver itself, because by the time you would do
something about it, the resources have already been consumed and your
server is probably already overloaded.
There are specialised front-end devices and software available, to detect
and protect against this kind of attack.

You may want to have a look at the following parameters, but make sure to
read the caveats (side-effects, interlocking timeouts 

Re: Apache/Tomcat vulnerability

[Jaaz Portal]

hi Andre,
thank you very much this was very educative but in my case it is little bit
different.
The server is no flooded, there is maybe dozen of very sophisticated
connections that somehow hangs apache workers threads
and the effect is permanent. Quickly the pool is exhausted and the only
solution that works is to restart tomcat.

I think it is a bug similar to this one from mod_proxy:
https://tools.cisco.com/security/center/viewAlert.x?alertId=34971

I think also that your solution with setting connectionTimeout will solve
the problem, at least partially. THANK YOU.

I would like to help you further investigate this issue, as our server
comes under such attack once or twice in a week.

best regards,
artur

2016-11-27 12:46 GMT+01:00 André Warnier (tomcat) :

> Hi.
>
> Have a look that the indicated parameters in the two pages below.
> You may be the target of such a variant of DDoS attack : many clients open
> a TCP connection to your server (front-end), but then never sends a HTTP
> request on that connection.  In the meantime, the server accepts the TCP
> connection, and passes it on to a "child" process or thread for
> processing.  The child then waits for the HTTP request line to arrive on
> the connection (during a certain time), but it never arrives. After a
> while, this triggers a timeout (see below), but the standard value of that
> timeout may be such that in the meantime, a lot of other connections have
> been established by other such nefarious clients, so a lot of resources of
> the webserver are tied up, waiting for something that will never come.
> Since there is never any real request sent on the connection, you would
> (probably) not see this in the logs either.
>
> The above is the basic mechanism of such an attack.  There may be
> variations, such as the client not "not sending" a request line, but
> sending it extremely slowly, thus achieving perhaps similar kinds of
> effects.
>
> As someone pointed out, it is quite difficult to do something about this
> at the level of the webserver itself, because by the time you would do
> something about it, the resources have already been consumed and your
> server is probably already overloaded.
> There are specialised front-end devices and software available, to detect
> and protect against this kind of attack.
>
> You may want to have a look at the following parameters, but make sure to
> read the caveats (side-effects, interlocking timeouts etc.), otherwise you
> may do more harm than good.
>
> Another thing : the settings below are for Apache Tomcat, which in your
> case is the back-end. It would of course be much better to detect and
> eliminate this at the front-end, or even before.  I had a look at the
> Apache httpd documentation, and could not find a corresponding parameter.
> But I am sure that it must exist. You may want to post this same question
> on the Apache httpd user's list for a better response.
>
> Tomcat configuration settings :
>
> AJP Connector : (http://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html#
> Standard_Implementations)
>
> connectionTimeout
>
> The number of milliseconds this Connector will wait, after accepting a
> connection, for the request URI line to be presented. The default value for
> AJP protocol connectors is -1 (i.e. infinite).
>
> (You could for example try to set this to 3000 (milliseconds) or even
> lower. That should be more than enough for any legitimate client so send
> the HTTP request line.  Note however that by doing this at the Tomcat
> level, you will probably move the problem to the Apache httpd/mod_jk
> level.  But at least it might confirm that this is the problem that you are
> seeing.  The mod_jk logfile at the httpd level may give you some hints
> there.)
>
>
> HTTP Connector : (http://tomcat.apache.org/tomc
> at-8.5-doc/config/http.html#Standard_Implementation)
>
> connectionTimeout
>
> The number of milliseconds this Connector will wait, after accepting a
> connection, for the request URI line to be presented. Use a value of -1 to
> indicate no (i.e. infinite) timeout. The default value is 6 (i.e. 60
> seconds) but note that the standard server.xml that ships with Tomcat sets
> this to 2 (i.e. 20 seconds). Unless disableUploadTimeout is set to
> false, this timeout will also be used when reading the request body (if
> any).
>
>
>
> On 26.11.2016 09:57, Jaaz Portal wrote:
>
>> hi,
>> sorry, its mod_jk no jk2, my typo. All at latest versions. We tried with
>> mod proxy too.
>>
>> There is no flood of the server. Nobody is flooding us, they use some
>> specific connections after which pool of apache workers is exhausted and
>> blocked
>> and we need to restart tomcat server.
>> It is some kind of exploit but do not know how to log it to obtain
>> details.
>>
>> i had put a limit on connections per client with hope that this will help
>> but once again, it is not a flood.
>> They open several connections that are not dropped by apache when they
>> 

Re: Apache/Tomcat vulnerability

[tomcat]

Hi.

Have a look that the indicated parameters in the two pages below.
You may be the target of such a variant of DDoS attack : many clients open a TCP 
connection to your server (front-end), but then never sends a HTTP request on that 
connection.  In the meantime, the server accepts the TCP connection, and passes it on to a 
"child" process or thread for processing.  The child then waits for the HTTP request line 
to arrive on the connection (during a certain time), but it never arrives. After a while, 
this triggers a timeout (see below), but the standard value of that timeout may be such 
that in the meantime, a lot of other connections have been established by other such 
nefarious clients, so a lot of resources of the webserver are tied up, waiting for 
something that will never come.
Since there is never any real request sent on the connection, you would (probably) not see 
this in the logs either.


The above is the basic mechanism of such an attack.  There may be variations, such as the 
client not "not sending" a request line, but sending it extremely slowly, thus achieving 
perhaps similar kinds of effects.


As someone pointed out, it is quite difficult to do something about this at the level of 
the webserver itself, because by the time you would do something about it, the resources 
have already been consumed and your server is probably already overloaded.
There are specialised front-end devices and software available, to detect and protect 
against this kind of attack.


You may want to have a look at the following parameters, but make sure to read the caveats 
(side-effects, interlocking timeouts etc.), otherwise you may do more harm than good.


Another thing : the settings below are for Apache Tomcat, which in your case is the 
back-end. It would of course be much better to detect and eliminate this at the front-end, 
or even before.  I had a look at the Apache httpd documentation, and could not find a 
corresponding parameter. But I am sure that it must exist. You may want to post this same 
question on the Apache httpd user's list for a better response.


Tomcat configuration settings :

AJP Connector : 
(http://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html#Standard_Implementations)


connectionTimeout   

The number of milliseconds this Connector will wait, after accepting a connection, for the 
request URI line to be presented. The default value for AJP protocol connectors is -1 
(i.e. infinite).


(You could for example try to set this to 3000 (milliseconds) or even lower. That should 
be more than enough for any legitimate client so send the HTTP request line.  Note however 
that by doing this at the Tomcat level, you will probably move the problem to the Apache 
httpd/mod_jk level.  But at least it might confirm that this is the problem that you are 
seeing.  The mod_jk logfile at the httpd level may give you some hints there.)



HTTP Connector : 
(http://tomcat.apache.org/tomcat-8.5-doc/config/http.html#Standard_Implementation)


connectionTimeout   

The number of milliseconds this Connector will wait, after accepting a connection, for the 
request URI line to be presented. Use a value of -1 to indicate no (i.e. infinite) 
timeout. The default value is 6 (i.e. 60 seconds) but note that the standard 
server.xml that ships with Tomcat sets this to 2 (i.e. 20 seconds). Unless 
disableUploadTimeout is set to false, this timeout will also be used when reading the 
request body (if any).



On 26.11.2016 09:57, Jaaz Portal wrote:

hi,
sorry, its mod_jk no jk2, my typo. All at latest versions. We tried with
mod proxy too.

There is no flood of the server. Nobody is flooding us, they use some
specific connections after which pool of apache workers is exhausted and
blocked
and we need to restart tomcat server.
It is some kind of exploit but do not know how to log it to obtain details.

i had put a limit on connections per client with hope that this will help
but once again, it is not a flood.
They open several connections that are not dropped by apache when they
disconnect. This way whole pool is quickly exhausted and the server broken.

i would like to help you to figure details of this attack but this is
production server so it is impossible to much debugging options

best,
artur

2016-11-25 23:44 GMT+01:00 Niranjan Babu Bommu :


you can find who is flooding site in apache access.log and block them in
firewall.

ex to find the IP:

cat /var/log/apache2/access.log |cut -d' ' -f1 |sort |uniq -c|sort -gr



On Fri, Nov 25, 2016 at 8:42 AM, Jaaz Portal  wrote:


hi,
we are from some weeks struggling with some Polish hackers that are
bringing our server down. After updating apache to latest version

(2.4.23)

and tomcat (8.0.38) available for debian systems we still cannot secure

our

server.

Today it has stopped to respond again and we needed to restart tomcat
process to get it back alive.

There is no too much clues