Re: Invalid characters in request header

2017-09-12 Thread Christopher Schultz 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Martynas,

On 9/9/17 6:29 AM, Martynas Jusevičius wrote:
> Well then you're out of luck. Everything is as expected though, at
> least on your end -- client sends invalid request, gets error
> response. What else do you need?

He's specifically asking for a way to stop logging this error,
possibly by filtering before the error has a chance to be logged.

Yuval, there is no way to do this with Tomcat. If you are fronting
Tomcat with a web server, that's probably the place to do this kind of
filtering

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJZuCs/AAoJEBzwKT+lPKRYvEEP/2saVZo0s0esHSwQHqWXbe6S
bJjyT0vBi3n1q+UK5LK31vCYI2yJfQNMTdeTHL4VUbVs4d8PRrL4Nm4VBCCqc2ri
XckvnthtTwo/hT+e0xWaP4xeF+Y0ZqxSFcm6LMbuiLelXldiZ41/wgCIrKZfxmZZ
LNPCohMrNEEgVDckQAn6HC6rBAbaiikbfehGQUQvE2fd5szlW0iT1NOpZ/14xfxZ
hCdaDZ+MM1ZJ2XDtKJ0zQAPRvJ/83FxvwuL9iI7gevWoKXCr+WhkU+7myiwJauM+
qzhTfYWJBTbF8bMCy5fmZ32gykDWC+Dko0gxU8L/otC5cKy9e6W/LYE4TaI0T1dF
LtGRyxQO49dYXKVJOkUQJ8LW/SB7i/CJMh9kmaC8/ukdf1rYIj7wRH6WCxIpvlPa
nBnc5M6xuxGxXSbuNsk388XgxsnAk9EBxSzJ0n0hKymHZTisf/STIxnqMKcYMsRx
1qN6h1dI5tn5skpaHoin4nRrs6nnOYhgaig9bix8qvkbcJk888fBYJTsE6rvjJ0q
a6IeCokVUeZBrBpXfYTlxgC5f2WfVVtUAm2dIL5VZJBHJFirNjnpb8NJw/9xVQ36
0wwRipcrR9ADFCYZ6XAkq44pDyrTh2a6yacOVUnwx+d6lBFvz34kda3QmcunsbQ2
2QlesHic9gIvPpIC0rO3
=mbeF
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

2017-09-12 Thread Yasser Zamani 
Ouch! maxConnections! You're right. I failed to take account of it 
correctly. Actually I was confused connections like threads (because of 
maxThreads/Connections) and thought they release when their thread 
releases :[ while a connection keeps alive until response and thread 
pool is shared between all of them to service them on demand.

I'm really sorry that I bothered you a lot!

Thanks a tone for your replies!

Regards.

On 9/12/2017 1:51 PM, Mark Thomas wrote:
> On 12/09/17 10:00, Yasser Zamani wrote:
>>
>>
>> On 9/12/2017 1:17 AM, Mark Thomas wrote:
>>> On 07/09/17 23:30, Yasser Zamani wrote:
 Thanks for your attention.

 Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
 it's connector in same way (BIO,20,20,10). I get same result, fortunately 
 :)

 OUTPUT:

 Using CATALINA_BASE:
 "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
 Using CATALINA_HOME:
 "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
 Using CATALINA_TMPDIR:
 "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
 Using JRE_HOME:"E:\jdk1.7.0_79"
 INFO: Server version:Apache Tomcat/7.0.81
 INFO: Server built:  Aug 11 2017 10:21:27 UTC
 INFO: Server number: 7.0.81.0
 INFO: OS Name:   Windows 8.1
 INFO: OS Version:6.3
 INFO: Architecture:  amd64
 INFO: Java Home: E:\jdk1.7.0_79\jre
 INFO: JVM Version:   1.7.0_79-b15
 INFO: JVM Vendor:Oracle Corporation
 INFO: CATALINA_BASE:
 C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2

 Container MAX used threads: 10
>>>
>>> I see similar results.
>>>
>>> There looks to be things going on either in JMeter or at the network
>>> level I don't understand. I had to resort to drawing it out to get my
>>> head around what is happening.
>>>
>>
>> Sorry for bothering you,
>>
>> To examine if things going on either in JMeter or at the network, I
>> tested same config (BIO,20,20,10) on Jetty. All 70 requests returned
>> successfully and response time was ~20 seconds for all, as I expects.
>
> I'm fairly sure Jetty uses NIO, not BIO which would explain the
> differences you are observing.
>
>> Then to make myself sure, I tested same servlet but a sync one (removed
>> my own thread pool and asyncStart etc) on Jetty. Average response time
>> increased to 95s, as I expected.
>>
>>> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
>>> The 21st request is accepted but then the acceptor blocks waiting for
>>> the connection count to reduce below 20 before proceeding.
>>>
>>
>> You have forgot.
>
> No I haven't.
>
>> My configuration is maxThreads=maxConnections=20 (not
>> 10) and acceptCount=10. As it prints "Container MAX used threads: 10" so
>> it never reaches maxThreads. So why acceptor blocks ?!
>
> After the first 20 requests you have 20 connections so you hit the
> maxConnection limit. That is why the acceptor blocks and subsequent
> requests go into the accept queue.
>
>> I think although
>> I have an async servlet and own thread pool, and although time consuming
>> process (Thread.sleep) is inside my own thread pool, but Tomcat's
>> container thread wrongly does not back to thread pool
>
> Incorrect. The container thread does return to the container thread
> pool. It does so almost immediately. Given that there are 0.5 seconds
> between requests and that the time taken to process an incoming request,
> dispatch the request to your thread pool and return the container thread
> to the container thread pool is almost certainly less than 0.5 it is
> very likely that there is never more than one container thread active at
> any one point.
>
>> and fails to> satisfy Servlet 3's async API!
>
> Also incorrect.
>
>>> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
>>> the test and the first request accepted won't finish processing for
>>> another 4.5 seconds.
>>>
>>> Requests 32 to 40 are dropped since the request queue is full. We are
>>> now 20s into the test and the first request is about to complete
>>> processing. Oddly, JMeter doesn't report these as failed until some 35
>>> seconds later.
>>>
>>> Request 1 completes. This allows request 21 to proceed. The acceptor
>>> takes a connection from the accept queue (this appears to be FIFO).
>>> Request 41 enters the accept queue.
>>>
>>> The continues until request 10 completes, 30 starts processing and 50
>>> enters the accept queue.
>>>
>>> Next 11 completes, 41 starts processing and 51 enters the accept queue.
>>> This continues until 20 completes, 50 starts processing and 60 enters
>>> the accept queue.
>>>
>>> At this point there are 20 threads processing, 10 in the accept queue
>>> and no thread due to complete for anther 10s.
>>>
>>> I'd expected requests

Re: How to prevent tomcat from deleting a webapp on restart

2017-09-12 Thread James H. H. Lampert 

On 9/11/17, 6:00 PM, Aryeh Friedman wrote:

I have setup a webapp (main app) that is uploaded with a war file and
second one ("images") that is just a dir in the webapp dir, no war file.
Sometimes (not always) on tomcat restart/system reboot images gets
deleted.   How can I prevent this.


Disable auto-deploy.

We learned that a long time ago, as our webapp product communicates with 
a proprietary server running -- like the Tomcat server -- on a 
customer-owned box (which may or may not be the same one running 
Tomcat), and may or may not be listening on the port the webapp expects 
it to be listening on, thus requiring post-deployment settings changes.


--
JHHL

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

2017-09-12 Thread Mark Thomas 
On 12/09/17 10:00, Yasser Zamani wrote:
> 
> 
> On 9/12/2017 1:17 AM, Mark Thomas wrote:
>> On 07/09/17 23:30, Yasser Zamani wrote:
>>> Thanks for your attention.
>>>
>>> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
>>> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>>>
>>> OUTPUT:
>>>
>>> Using CATALINA_BASE:
>>> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
>>> Using CATALINA_HOME:
>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
>>> Using CATALINA_TMPDIR:
>>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
>>> Using JRE_HOME:"E:\jdk1.7.0_79"
>>> INFO: Server version:Apache Tomcat/7.0.81
>>> INFO: Server built:  Aug 11 2017 10:21:27 UTC
>>> INFO: Server number: 7.0.81.0
>>> INFO: OS Name:   Windows 8.1
>>> INFO: OS Version:6.3
>>> INFO: Architecture:  amd64
>>> INFO: Java Home: E:\jdk1.7.0_79\jre
>>> INFO: JVM Version:   1.7.0_79-b15
>>> INFO: JVM Vendor:Oracle Corporation
>>> INFO: CATALINA_BASE:
>>> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>>>
>>> Container MAX used threads: 10
>>
>> I see similar results.
>>
>> There looks to be things going on either in JMeter or at the network
>> level I don't understand. I had to resort to drawing it out to get my
>> head around what is happening.
>>
> 
> Sorry for bothering you,
> 
> To examine if things going on either in JMeter or at the network, I 
> tested same config (BIO,20,20,10) on Jetty. All 70 requests returned 
> successfully and response time was ~20 seconds for all, as I expects.

I'm fairly sure Jetty uses NIO, not BIO which would explain the
differences you are observing.

> Then to make myself sure, I tested same servlet but a sync one (removed 
> my own thread pool and asyncStart etc) on Jetty. Average response time 
> increased to 95s, as I expected.
> 
>> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
>> The 21st request is accepted but then the acceptor blocks waiting for
>> the connection count to reduce below 20 before proceeding.
>>
> 
> You have forgot.

No I haven't.

> My configuration is maxThreads=maxConnections=20 (not 
> 10) and acceptCount=10. As it prints "Container MAX used threads: 10" so 
> it never reaches maxThreads. So why acceptor blocks ?!

After the first 20 requests you have 20 connections so you hit the
maxConnection limit. That is why the acceptor blocks and subsequent
requests go into the accept queue.

> I think although 
> I have an async servlet and own thread pool, and although time consuming 
> process (Thread.sleep) is inside my own thread pool, but Tomcat's 
> container thread wrongly does not back to thread pool

Incorrect. The container thread does return to the container thread
pool. It does so almost immediately. Given that there are 0.5 seconds
between requests and that the time taken to process an incoming request,
dispatch the request to your thread pool and return the container thread
to the container thread pool is almost certainly less than 0.5 it is
very likely that there is never more than one container thread active at
any one point.

> and fails to> satisfy Servlet 3's async API!

Also incorrect.

>> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
>> the test and the first request accepted won't finish processing for
>> another 4.5 seconds.
>>
>> Requests 32 to 40 are dropped since the request queue is full. We are
>> now 20s into the test and the first request is about to complete
>> processing. Oddly, JMeter doesn't report these as failed until some 35
>> seconds later.
>>
>> Request 1 completes. This allows request 21 to proceed. The acceptor
>> takes a connection from the accept queue (this appears to be FIFO).
>> Request 41 enters the accept queue.
>>
>> The continues until request 10 completes, 30 starts processing and 50
>> enters the accept queue.
>>
>> Next 11 completes, 41 starts processing and 51 enters the accept queue.
>> This continues until 20 completes, 50 starts processing and 60 enters
>> the accept queue.
>>
>> At this point there are 20 threads processing, 10 in the accept queue
>> and no thread due to complete for anther 10s.
>>
>> I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
>> processed. It looks like there is some sort of timeout for acceptance or
>> rejection in the accept queue.
>>
>> That explains the rejected requests.
>>
> 
> I'm not such smart to understand your analysis :) I just understand when 
> my servlet is completely async, then I should not have any rejected 
> requests (specially in such low load) like Jetty result.

You are failing to take account of the maxConnections configuration.
With a new request being made every 0.5 seconds and a wait time of 20
seconds Tomcat needs to be able to

Re: BIO: Async servlet with it's own thread pool; but get connection refused!

2017-09-12 Thread Yasser Zamani 


On 9/12/2017 1:17 AM, Mark Thomas wrote:
> On 07/09/17 23:30, Yasser Zamani wrote:
>> Thanks for your attention.
>>
>> Now I downloaded a fresh apache-tomcat-7.0.81-windows-x64 and chenged
>> it's connector in same way (BIO,20,20,10). I get same result, fortunately :)
>>
>> OUTPUT:
>>
>> Using CATALINA_BASE:
>> "C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2"
>> Using CATALINA_HOME:
>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81"
>> Using CATALINA_TMPDIR:
>> "C:\Users\user\Downloads\apache-tomcat-7.0.81-windows-x64-IJ\apache-tomcat-7.0.81\temp"
>> Using JRE_HOME:"E:\jdk1.7.0_79"
>> INFO: Server version:Apache Tomcat/7.0.81
>> INFO: Server built:  Aug 11 2017 10:21:27 UTC
>> INFO: Server number: 7.0.81.0
>> INFO: OS Name:   Windows 8.1
>> INFO: OS Version:6.3
>> INFO: Architecture:  amd64
>> INFO: Java Home: E:\jdk1.7.0_79\jre
>> INFO: JVM Version:   1.7.0_79-b15
>> INFO: JVM Vendor:Oracle Corporation
>> INFO: CATALINA_BASE:
>> C:\Users\user\.IntelliJIdea2016.3\system\tomcat\Unnamed_Async-Servlet-Example_2
>>
>> Container MAX used threads: 10
>
> I see similar results.
>
> There looks to be things going on either in JMeter or at the network
> level I don't understand. I had to resort to drawing it out to get my
> head around what is happening.
>

Sorry for bothering you,

To examine if things going on either in JMeter or at the network, I 
tested same config (BIO,20,20,10) on Jetty. All 70 requests returned 
successfully and response time was ~20 seconds for all, as I expects.

Then to make myself sure, I tested same servlet but a sync one (removed 
my own thread pool and asyncStart etc) on Jetty. Average response time 
increased to 95s, as I expected.

> The first 20 requests (10 seconds) are accepted and processed by Tomcat.
> The 21st request is accepted but then the acceptor blocks waiting for
> the connection count to reduce below 20 before proceeding.
>

You have forgot. My configuration is maxThreads=maxConnections=20 (not 
10) and acceptCount=10. As it prints "Container MAX used threads: 10" so 
it never reaches maxThreads. So why acceptor blocks ?! I think although 
I have an async servlet and own thread pool, and although time consuming 
process (Thread.sleep) is inside my own thread pool, but Tomcat's 
container thread wrongly does not back to thread pool and fails to 
satisfy Servlet 3's async API!

> Requests 22 to 31 are placed in the accept queue. We are now 15.5s into
> the test and the first request accepted won't finish processing for
> another 4.5 seconds.
>
> Requests 32 to 40 are dropped since the request queue is full. We are
> now 20s into the test and the first request is about to complete
> processing. Oddly, JMeter doesn't report these as failed until some 35
> seconds later.
>
> Request 1 completes. This allows request 21 to proceed. The acceptor
> takes a connection from the accept queue (this appears to be FIFO).
> Request 41 enters the accept queue.
>
> The continues until request 10 completes, 30 starts processing and 50
> enters the accept queue.
>
> Next 11 completes, 41 starts processing and 51 enters the accept queue.
> This continues until 20 completes, 50 starts processing and 60 enters
> the accept queue.
>
> At this point there are 20 threads processing, 10 in the accept queue
> and no thread due to complete for anther 10s.
>
> I'd expected requests 61 to 70 to be rejected. However, 65 to 70 are
> processed. It looks like there is some sort of timeout for acceptance or
> rejection in the accept queue.
>
> That explains the rejected requests.
>

I'm not such smart to understand your analysis :) I just understand when 
my servlet is completely async, then I should not have any rejected 
requests (specially in such low load) like Jetty result. As I said, max 
Tomcat container's threads used by my APP is 10 (half of initial size 
20) so why I see rejected requests while thread pool has 10 free threads 
to accept new requests ?!

> The other question is why maxThreads is reported as it is.
>
> The answer is that the thread pool never grows beyond its initial size
> of 10. A request comes in, it is processed by a container thread,
> dispatched to an async thread and then the container thread is returned
> to the pool to await the next request. Tomcat is able to do this because
> the container doesn't perform any I/O on the connection once it enters
> async mode until it is dispatched back to the container. The default
> thread pool implementation cycles through the threads so the max value
> you see is 10 which is the initial size.
>

My config was maxThreads=20 (not 10).

Sincerely Yours,
Yasser.

> HTH,
>
> Mark
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail:

Re: Setting PropertySourceVault programatically in o.a.t.util.digester.Digester

2017-09-12 Thread Rémy Maucherat 
On Mon, Sep 11, 2017 at 10:56 PM, Mark Thomas  wrote:

> On 11/09/17 21:07, Coty Sutherland wrote:
> > On Mon, Sep 11, 2017 at 2:43 PM, Mark Thomas  wrote:
> >> On 11/09/17 19:07, Coty Sutherland wrote:
> >>
> >>> So, my questions comes down to "Can I configure a PropertySource
> >>> instance and pass that to tomcat's Digester before tomcat starts?".
> >>
> >> No, because of the way it is currently coded.
> >
> > I thought so.
> >
> >>> Is this a silly question? If so, why?
> >>
> >> No. N/A.
> >>
> >> I can see the benefit in making it configurable.
> >>
> >> It looks like you'd need something along the lines of an
> >> addSource(IntrospectionUtils.PropertySource) method that added to the
> >> source array.
> >
> > Hm. So I took a look at the Digester and having a way to add the
> > source there would be ideal, but I think that only solves the issue of
> > having tomcat's internal classes configure it, right? I don't see a
> > way to do something like tomcat.setDigester(myDigester) anywhere. As
> > far as the digester that loads the server.xml I just see
> > o.a.c.startup.Catalina.createStartDigester(). I've only had a second
> > to look at it, but I'd love to figure this out. Hints appreciated :)
>
> I don't think I looked at it for much longer.
>
> You don't want that Digester instance - the Tomcat class doesn't parse
> server.xml. You want the equivalent for the Context which is set in
> ContextConfig. (or is this for web.xml?)
>
> Take a look at the difference between addWebApp and addContext.
>
> It sounds like you are using addWebApp. In which case maybe a property
> source setter on ContextConfig that is then injected into that Digester
> instance?
>
> > I guess I should create an enhancement BZ for this?
>
> I think so. There are a few different digester instances. It would be
> good to have a solution that handled all likely variations.
>

Right now the property source comes from a system property. So the
equivalent would be to use a static method to set some.

Rémy