Re: AW: AW: Suppress or replace WWW-Authorization header

2015-10-28 Thread Andy Wang



On 10/28/2015 12:04 PM, André Warnier (tomcat) wrote:

The server supports two kinds of deployment: Standalone with an
embedded Jetty-server and as war-file for app-servers (most of them
are tomcat-server). I try to suppress the browser BASIC-login-dialog
for the REST-service-calls from AngularJS.
On Jetty I modify the 401-responses and replace the "WWW-Authenticate"
header by anything else than "BASIC" and that works, now I try to
find a solution for the deployment on tomcat servers.


[.. snipped ..]


1) if the server sends to the client an authentication header saying
HTTP Basic, then the client will popup a builtin HTTP Basic dialog
(which you do not want)
2) if the server sends to the client an authentication header saying
something else, then the client cannot handle it

1 + 2 = solution impossible

You mentioned before that with another server than Tomcat, you solved
this apparently impossible problem.  Can you tell us how ?

Or else, can you tell us which authentication methods, /apart/ from HTTP
Basic, the client does support ?


I've seen the behavior that Torsten is attempting to achieve, but I 
think the issue is a fundamental lack of understanding of the protocol 
on his part.  This is assuming I'm understanding what he's trying to get at.


He's saying that in a regular http basic 401 exchange, if you remove the 
WWW-Authenticate header, it will actually prevent the some clients from 
popping up a dialog. I've seen this before with some clients.  I'm not 
sure if all clients react this way.  The hold HTTPClient (not the apache 
commons one, but one hosted on a .cz domain?  something like that.  This 
is along long time ago) had that behavior.  I admit I've never seen that 
behavior in a browser but never tried.


RFC 2617 states this:
The 401 (Unauthorized) response message is used by an origin server
   to challenge the authorization of a user agent. This response MUST
   include a WWW-Authenticate header field containing at least one
   challenge applicable to the requested resource.

I don't think it's unreasonable to assume state that tomcat (or just 
about any standards compliant web server) when serving up HTTP Basic 
authentication will provide a complete and valid response and nothing else.


I may be misunderstanding, but my interpretation is Torsten simply wants 
the client handling the RESTful service to never have to be accidentally 
prompted to authenticate.  If it screwed up and didn't send an 
Authorization header to begin with, then skip the www-authenticate 
header and hide the failure from the user.


If this is the case, I know of no way to do this short of
a) an intermediate proxy doing the work
b) creating your own authentication handler in tomcat to detect your 
user-agent and spit back custom 401 responses depending on the agent.


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Chunked transfer delay with httpd 2.4 + mod_jk 1.2.41 on Windows.

2015-10-19 Thread Andy Wang

Hi all,
I'm seeing a weird problem that I'm running out of ideas on.  I'm going 
to send this email to both the apache httpd users list and the tomcat 
users list (separate messages) but hoping for any ideas at all.


The issue is currently reproduced using Apache httpd 2.4.16, mod_jk 
1.2.41 and tomcat 8.0.28.


I've created a very very simple JSP page that does nothing but print a 
small string, but I've tried changing the jsp page to print a very very 
large string (1+ characters) and no difference.


If I POST to this JSP page, and something like mod_deflate is in place 
to force a chunked transfer the TCP packet capture looks like this:


No. Time   Source  Destination   Protocol Length Info
   1850 4827.762721000 client  serverTCP  66 54131→2280 
[SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
   1851 4827.764976000 server  clientTCP  66 2280→54131 
[SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
   1852 4827.765053000 client  serverTCP  54 54131→2280 
[ACK] Seq=1 Ack=1 Win=131328 Len=0
   1853 4827.765315000 client  serverHTTP 791POST 
/JSPtoPostTo HTTP/1.1
   1854 4827.777981000 server  clientTCP  466[TCP 
segment of a reassembled PDU]
   1855 4827.982961000 client  serverTCP  54 54131→2280 
[ACK] Seq=738 Ack=413 Win=130816 Len=0
   1856 4832.770458000 server  clientHTTP 74 HTTP/1.1 
200 OK  (text/html)
   1857 4832.770459000 server  clientTCP  60 2280→54131 
[FIN, ACK] Seq=433 Ack=738 Win=65536 Len=0
   1858 4832.770555000 client  serverTCP  54 54131→2280 
[ACK] Seq=738 Ack=434 Win=130816 Len=0
   1859 4832.770904000 client  serverTCP  54 54131→2280 
[FIN, ACK] Seq=738 Ack=434 Win=130816 Len=0
   1860 4832.77420 server  clientTCP  60 2280→54131 
[ACK] Seq=434 Ack=739 Win=65536 Len=0


Spdficially, note the 5 second delay between the first segment (No. 
1854) and the second data segment (1856).
The first segment contains the headers, the second segment contains the 
mod_deflate compressed contents.


Here's where it gets sort of interesting.
I can only reproduce this with one client.  We have a custom NPAPI 
plugin that this problem occurs on with all versions of firefox.  I 
cannot reproduce it with any other client.  I've used HttpRequester 
extension on firefox to construct an arbitrary POST.  I've copied the 
raw HTTP request from the wireshark capture:


POST 
/Windchill/wtcore/jsp/wvs/mkannio.jsp?CSRF_NONCE=gXFm878tFEjetz6Y6DgPscd5WiSqzlzxuRYjnsx1Igq6hwio7ToBpeV8dgCk9nH95UEEn%2B9IQXXkhgqstENexI0dJ3HohATr8BkqwtAaTiW9hg6puEcum9UfUjqpigM%3D=1=test123123123=0=OR:wt.viewmarkup.DerivedImage:52607=.ast 
HTTP/1.1

Host: host:2280
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 
Firefox/41.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: JSESSIONID=F3B0524C87BD94396CCDE40FF837D2F8.tomcat1
Authorization: Basic d2NhZG1pbjp3Y2FkbWlu
Connection: keep-alive
Content-length: 0
Content-Type: text/plain

and used ncat and curl to attempt to simulate the request and no luck 
reproducing it.


This does not occur with Apache httpd 2.2.31 and mod_jk 1.2.41.  And so 
far, it only occurs when the server is on Windows.


I can't come up with any scenario where I can explain how the client 
could possibly be the factor.


I've enabled mod_jk's trace logging and according to mod_jk the ajp 
request was handled and processed in less than a second.


The timing really seems to imply that this is all somwhere in 
mod_deflate.  But I've tried something similar by creating an html file 
and request it via a POST instead of a GET and the problem doesn't occur 
either.


Hoping for any ideas on where to go with this.

Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Chunked transfer delay with httpd 2.4 + mod_jk 1.2.41 on Windows.

2015-10-19 Thread Andy Wang



On 10/19/2015 06:04 PM, Konstantin Kolinko wrote:request.


Is the below a capture between your client and HTTPD?  (as opposed to
one between HTTPD and Tomcat)



The capture is between client and httpd



Note that Basic auth sends password in plain text (encoded in base64).
So you password is written on the above line.



Yeah, I know.  If you decode it yu'll see it's throwaway credentials :)


Connection: keep-alive
Content-length: 0
Content-Type: text/plain


Content-Length of 0 means that this POST request has no body.



Yup.  This is just to simulate the issue.

The issue is purely on the response, not on the requestion.


BTW, a delay of several seconds may be there is a client expects a
100-continue response from the server. If there is no HTTP status 100
response it may proceed sending the body.


There is no 100-continue behavior here.

As i also posted this to the users@httpd mailing list, they've responded 
over there.  It looks like the pipelining feature is resulting in a 
delay before flushing if the payload isn't a full packet size.


So that looks like a likely candidate here.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Chunked transfer delay with httpd 2.4 + mod_jk 1.2.41 on Windows.

2015-10-19 Thread Andy Wang
I should add that we have a custom CompressionFilter that also 
reproduces the problem without mod_deflate. Which seems to point to 
something in Apache httpd and chunked transfers being the hold up, but I 
don't know for sure.


I'm currently exclusively testing with mod_deflate as this is OOTB 
behavior rather than introduce the confusion of a custom ServletFilter.


Andy


On 10/19/2015 04:45 PM, Andy Wang wrote:

Hi all,
I'm seeing a weird problem that I'm running out of ideas on.  I'm going
to send this email to both the apache httpd users list and the tomcat
users list (separate messages) but hoping for any ideas at all.

The issue is currently reproduced using Apache httpd 2.4.16, mod_jk
1.2.41 and tomcat 8.0.28.

I've created a very very simple JSP page that does nothing but print a
small string, but I've tried changing the jsp page to print a very very
large string (1+ characters) and no difference.

If I POST to this JSP page, and something like mod_deflate is in place
to force a chunked transfer the TCP packet capture looks like this:

No. Time   Source  Destination   Protocol Length Info
1850 4827.762721000 client  serverTCP  66 54131→2280
[SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
1851 4827.764976000 server  clientTCP  66 2280→54131
[SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460 WS=256 SACK_PERM=1
1852 4827.765053000 client  serverTCP  54 54131→2280
[ACK] Seq=1 Ack=1 Win=131328 Len=0
1853 4827.765315000 client  serverHTTP 791POST
/JSPtoPostTo HTTP/1.1
1854 4827.777981000 server  clientTCP  466[TCP
segment of a reassembled PDU]
1855 4827.982961000 client  serverTCP  54 54131→2280
[ACK] Seq=738 Ack=413 Win=130816 Len=0
1856 4832.770458000 server  clientHTTP 74 HTTP/1.1
200 OK  (text/html)
1857 4832.770459000 server  clientTCP  60 2280→54131
[FIN, ACK] Seq=433 Ack=738 Win=65536 Len=0
1858 4832.770555000 client  serverTCP  54 54131→2280
[ACK] Seq=738 Ack=434 Win=130816 Len=0
1859 4832.770904000 client  serverTCP  54 54131→2280
[FIN, ACK] Seq=738 Ack=434 Win=130816 Len=0
1860 4832.77420 server  clientTCP  60 2280→54131
[ACK] Seq=434 Ack=739 Win=65536 Len=0

Spdficially, note the 5 second delay between the first segment (No.
1854) and the second data segment (1856).
The first segment contains the headers, the second segment contains the
mod_deflate compressed contents.

Here's where it gets sort of interesting.
I can only reproduce this with one client.  We have a custom NPAPI
plugin that this problem occurs on with all versions of firefox.  I
cannot reproduce it with any other client.  I've used HttpRequester
extension on firefox to construct an arbitrary POST.  I've copied the
raw HTTP request from the wireshark capture:

POST
/Windchill/wtcore/jsp/wvs/mkannio.jsp?CSRF_NONCE=gXFm878tFEjetz6Y6DgPscd5WiSqzlzxuRYjnsx1Igq6hwio7ToBpeV8dgCk9nH95UEEn%2B9IQXXkhgqstENexI0dJ3HohATr8BkqwtAaTiW9hg6puEcum9UfUjqpigM%3D=1=test123123123=0=OR:wt.viewmarkup.DerivedImage:52607=.ast
HTTP/1.1
Host: host:2280
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101
Firefox/41.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: JSESSIONID=F3B0524C87BD94396CCDE40FF837D2F8.tomcat1
Authorization: Basic d2NhZG1pbjp3Y2FkbWlu
Connection: keep-alive
Content-length: 0
Content-Type: text/plain

and used ncat and curl to attempt to simulate the request and no luck
reproducing it.

This does not occur with Apache httpd 2.2.31 and mod_jk 1.2.41.  And so
far, it only occurs when the server is on Windows.

I can't come up with any scenario where I can explain how the client
could possibly be the factor.

I've enabled mod_jk's trace logging and according to mod_jk the ajp
request was handled and processed in less than a second.

The timing really seems to imply that this is all somwhere in
mod_deflate.  But I've tried something similar by creating an html file
and request it via a POST instead of a GET and the problem doesn't occur
either.

Hoping for any ideas on where to go with this.

Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: building 64-bit isapi_redirect.dll

2015-08-14 Thread Andy Wang



On 08/14/2015 11:57 AM, Christopher Schultz wrote:


You might want to reach-out to the Apache Lounge folks, too... they
seem to have a good process for building ASF binaries that they might
be willing to share with you.



That was one of the first things I thought about and checked.  This is 
what they have to say about it:


Be sure that you have installed the Visual C++ 2010 SP1 Redistributable 
Package: Win64 vcredist_x64.exe, Win32 vcredist_x86.exe


:)  Pretty much where I'm at right now.

Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: building 64-bit isapi_redirect.dll

2015-08-13 Thread Andy Wang



On 08/13/2015 04:46 PM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Andy,

On 8/13/15 5:34 PM, Andy Wang wrote:

I was hoping to find out how the official isapi_redirect.dlls are
built. Specifically the x64 version.  Reason I ask is that 1) the
documentation still documents using MS VC 6.0 which I think can't
build 64-bit 2) the official binaries seem to link against
msvcrt.dll and not msvcr[version].dll

I've been using VisualStudio 2010 to build it, and have been
needing to install the vcredist installer to get msvcr100.dll.

Trying to figure out how the official binaries are built such that
this dependency isn't required?

Doing some googling it looks like it can be done with a combination
of the Windows SDK and DDK.  Is this how this was done?


I'm no expert.

In the past, I tried to figure-out my own process for building
win32/win64-based builds of various Tomcat-related things. For me, it
was tcnative.

What I *do* know is that building against msvcrt.dll is a trick that
is required in order to get a universal build that will work under any
environment. I believe you are required to use the Driver Development
Kit for that.

If you are building for your own environment, you can simply build
against msvcrtXXX.dll and be perfectly happy, since your environment
by definition has that library.

Take a look at markt's instructions for building tcnative on win32;
they may be helpful in getting the mod_jk build working for you:

http://wiki.apache.org/tomcat/BuildTcNativeWin


It does, and it's the process I thought about using, but I've also read 
that you shouldn't rely on the DDK for building as the msvcrt.dll gets 
updated regularly with windows releases and you could potentially end up 
with dll incompatibilities as the ABIs change.


That said, perhaps the connector's use of msvcrt.dll is less susceptible 
to this, since I imagine it's simply using pretty standard C calls.


I actually do redistribute my bulids for others to use, which is why I'm 
trying to decide if I should concern myself with the extra prerequisite 
of install the vcredist (or bundling the dll files myself).  The key is, 
I'm not familiar with how IIS loads dlls, so I'm not sure if including 
msvcr100.dll in the same directory as isapi_redirect.dll will work or if 
I'd have to put it in the path somewhere.


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



building 64-bit isapi_redirect.dll

2015-08-13 Thread Andy Wang
I was hoping to find out how the official isapi_redirect.dlls are built. 
 Specifically the x64 version.  Reason I ask is that
1) the documentation still documents using MS VC 6.0 which I think can't 
build 64-bit
2) the official binaries seem to link against msvcrt.dll and not 
msvcr[version].dll


I've been using VisualStudio 2010 to build it, and have been needing to 
install the vcredist installer to get msvcr100.dll.


Trying to figure out how the official binaries are built such that this 
dependency isn't required?


Doing some googling it looks like it can be done with a combination of 
the Windows SDK and DDK.  Is this how this was done?


Thanks,
Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat, REMOTE_USER, getRemoteUser()

2015-07-28 Thread Andy Wang



On 07/28/2015 02:03 PM, Christopher Schultz wrote:

On 7/28/15 2:29 PM, John Baker wrote:

Hello,


I'm not sure how long ago that was, but I don't live in the
Windows world. I would have thought that someone at Apache Lounge
would have balked if a release was broken. Were you building a
release version, or trunk?


I downloaded a release. This was a few years ago now. I suspect
mod_jk on Windows is no longer well used.


I'm guessing it's less popular than on *NIX. George Stanchev (a Tomcat
community member) is certainly using it. He filed a bug and patch
against the ISAPI module that coincidentally has to do with
REMOTE_USER handling.

https://bz.apache.org/bugzilla/show_bug.cgi?id=57836

Windows is not being ignored.



I've been building and shipping and supporting an apache based web
server for windows, linux, aix, and solaris for well over a decade now.
  I can't recall the last time i had a problem with mod_jk on windows
that didn't also exist on other platforms.  I've also been building and 
maintaining 32 and 64-bit isapi_redirect.dlls.  I can't give numbers, 
but I'm confident to say i have a very good sample set.


The only Windows-specific problem I can think of is the ISAPI
remote_user =  that was being described on a bugzilla report that I
ran into years ago and we assumed it was a flaw in IIS/ISAPI so never
reported it to tomcat folk :) [ reading the thread, i'm not convinced it 
isn't a flaw with ISAPI ]


I'm mostly a lurker on these mailing lists (and have been on them for
over a decade as well) primarly because I don't have alot of problems,
and usually when I do run across one, someone already has reported it
and it's already been fixed.  This is true on all of the platforms I
have to support including Windows.


I'm suggesting that [REMOTE_USER via HTTP header] support should be
  more integrated to the getRemoteUser() call, as has been
implemented with AJP.


Agreed. I just don't like your proposed patch. Note that the AJP
connector *does not* do it like your proposal did. AJP uses a
side-channel that does not involve HTTP headers.


So, your proposed implementation is incorrect and represents
a security vulnerability.


It does not represent a security vulnerability.


Sure it does: a client can supply a forged REMOTE_USER header
quite easily.


Yes, and that's precisely the problem with mod_jk. Anyone with
local access to the host can do the same as a
Apache/mod_proxy_http/REMOTE_USER solution. I'm amazed the wider
security world hasn't picked up on the widely abused mod_jk
security hole.


It's rare to have port 8009 open to the world, but you're right:
anyone on localhost can own you. But if they have localhost, you are
already owned anyway, right?



I constantly have to explain to people why a header cannot simply be
blindly trusted for secure data :)

In terms of the localhost issue, you have the combination of securing
the host, and also potentially file security combined with an AJP
secret.  That's one additional mechanism


So, how is Tomcat supposed to know that the request has been
properly-sanitized? At least with AJP, one expects that a nearby
web server is making the request and that the user knows what
they are doing. Nobody sets up a one-box-wonder in production
with the AJP port publicly available. But people do that all the
time with port 80 available publicly.


Well, for a start, I'm not sure the AJP connector comes bound to
localhost (tomcat 8.x download):

!-- Define an AJP 1.3 Connector on port 8009 -- Connector
port=8009 protocol=AJP/1.3 redirectPort=8443 /

Next, no-one makes any effort to secure AJP to Apache. And whilst
I'm sympathetic to the sanitisation of HTTP headers (by Apache)
belief, I'm not sure Tomcat's now decade old HTTP connector - that
has had lots of attention/work - needs the protection once offered
by Apache HTTPd. It would certainly be interesting to get the two
pen-tested.


I'm just suggesting that if Apache httpd is going to be trading
authentication headers with Tomcat, it should do it properly. It's
easy to overlook the Header unset REMOTE_USER configuration that
would be required to first sanitize the incoming HTTP request.

This has nothing to do with whether or not httpd makes sense out in
front of Tomcat (and there are still some good reasons for that): your
use-case pre-supposes that setup.



I'm not sure there is a better way to do this.  REMOTE_USER is the 
standards based mechanism of communicating this from a trusted web 
server to a consumer of that data.


AJP is intended to be for this purpose.  There's no RFC/official IETF 
spec for AJP, but the documentation for AJP:

https://tomcat.apache.org/connectors-doc/ajp/ajpv13a.html
According to email from Gal Shachor to the jakarta-dev mailing list, 
the original goals of JK (and thus ajp13) were to extend mod_jserv and 
ajp12 by (I am only including the goals which relate to communication 
between the web server and the servlet container):


So the primary goal of this was 

Re: Tomcat, REMOTE_USER, getRemoteUser()

2015-07-28 Thread Andy Wang



On 07/28/2015 03:02 PM, Andy Wang wrote:

I'd also like a better way and after discussing with some
security-geeks, we were wondering if there's some way we can
implement a Valve that takes a username and a signature using a
shared secret. The problem is signing in Apache: I've not looked
too hard for a module to do this but maybe one exists? If one does
exist, then the mod_jk module could use the same strategy to ensure
Tomcat only trusts a username + valid signature.


Take a look at the RemoteIpValve.  One could make something similar, say
a RemoteAuthenticationValve, would be my guess.  Given that you're using
a shared secret already, I'm not sure what signing will buy you.  If
some malicious entity gets the shared secret, the signature/encryption
is going to do nothing for you.  If you're concerned about the message
secured by the shared secret being in plain text across a remote
http-tomcat configuration, I imagine stunnel would help solve the problem



clarification - given that you're already talking a shared secret (not 
using - since you're discussing a hypothetical mechanism - i.e. 
non-existent).



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT] Random Form Resubmissions

2015-06-17 Thread Andy Wang



On 06/17/2015 12:43 PM, Caldarale, Charles R wrote:

From: Jerry Malcolm [mailto:techst...@malcolms.com]
Subject: OT: Random Form Resubmissions



I have written defensive code in my webapp to detect this situation and
handle it.  So it's not a critical problem now. But it just frustrates
me that I have no clue what is going on.  And I'm curious if the users
are seeing something strange as this is occurring.  It appears that the
client's browser is holding onto the form and just randomly resending it
the server without the user's knowledge.  And it finally stops when they
close their browser or reboot their computer.  I know this makes zero sense.


This sounds like a lot of fun.


HAH!! I was thinking the exact same thing :)


Can you tell from the logs (or doing some traceroute runs) if there's anything 
in common about the clients?  Things such as a small set of origins (e.g., PRC, 
NK, AOL, NSA) or client environment (iOS, Android, IE7) might give a clue.

I wonder if some intermediary box thinks it's being useful by trying to get 
responses to unacknowledged requests and keeps replaying them, but the real 
client lost interest and disconnected a long time ago.



The comment:
And it finally stops when they close their browser or reboot their 
computer.
has me thinking, proxy/internet security software on the system doing 
it?  Or a browser extension.  But why and what eludes me.



  - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: TCP connections reuse

2015-06-12 Thread Andy Wang

Could this be as simple as the default keepaliveTimeout = 15000 (i.e. 15s)

Andy

On 06/12/2015 11:20 AM, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Maxim,

On 6/12/15 1:53 AM, Maxim Neshcheret wrote:

According to
http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepali

ve.html




connections in HTTP 1.1

http://docs.oracle.com/javase/6/docs/technotes/guides/net/http-keepal

ive.html%20connections%20in%20HTTP%201.1




are persistent by default.


But if it is incorrect behavior in Java then maybe there is some
way or configuration in Tomcat to put Keep-alive parameter in the
response header?

I do call HttpURLConnection.disconnect() only when client is
stopped. Keep-alive is used for sure.


I would expect the connection to be kept open unless you call
disconnect. But you are creating a new HttpURLConnection each time and
then discarding it. I'm not sure how the connection pooling works
under the hood within the JVM. It used to be difficult to get the JVM
to release a connection and actually terminate the TCP/IP connection.

http://docs.oracle.com/javase/8/docs/technotes/guides/net/http-keepalive
.html

Pay close attention to the remarks about stream-handling.

Tomcat may close the connection under certain circumstances, usually
error-related. If you are getting 200 OK responses from Tomcat, then
I'm not sure why the TCP/IP connection would be dropped, unless Java
thinks that 30 seconds is too long of an idle time to keep those
connections open.

Thanks,
- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org

iQIcBAEBCAAGBQJVewa7AAoJEBzwKT+lPKRYke0QALlr4G7IZueNeG90NCsMhY4Y
iU4RsiXDFRBgb3Wd9X305yyYiULzRB2u7TXAirB2vXttJiBmXR6v9cKqxW2DdgDZ
cRRd+ymQbDyhhO3RDodVDQp8OAf3YlXoTH+zIvSYYFnzo148iuHxLyLrXyyHkfv2
wf5zwVjLVH0Pu7ndTbu23kJ6zhMKrOzzgKa1t/iWOa9LqxE6vht9d24HYzyluZoQ
xnmkovYf2bNo4pVw4xcg9QNeeIDAnRBBcusNr4qUjif14pA7fmbliTPBtzFTyi9/
AuFlNdIJtz5zLXhTopjLwkrNDyx3AFwo0UQrlpoaoURAvVhDri5PlphDM94TyLBx
VRPeChq1Tj0wnAP/j1wqr6VEyC30AZ/w/z89zwy1SpTE7ywUwmJmamFcSVoUTOjL
U9J78+29pzlzcFj1OR0lh5xXMjL0yXmcLXSmKNWp0AwbVQacV5PSz5QqM32zM9Bn
1sOndI24BXcl3VyXPai9JdmxgoowGCszis+xCn+yuDwE5moeBV7xmeQtdagnhFex
oBzGNDH/K/Up/Kh2bUxPXM0Ij0ksG6L8s8WTuyu1Tctly0NG5piW2xDwjs9nbomc
qKQvfjnd5zI3ps6CyE/ZTa0LacyolaRgWVxXoZZ9En4bUVVexOHLllzc7e7uban4
C4Tgxbwn1lMQ8+GLYYrd
=AbYE
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: SSL Handshake Exceptions

2015-05-11 Thread Andy Wang



On 05/11/2015 01:24 PM, jairaj kamal wrote:

javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target


This usually means that the ssl client (the client that's originating 
the direct connection to the ssl server) is unable to construct a proper 
certificate trust path for the server.


As you noted, you used a self-signed cert.  This means that you need to 
import the certificate into the jssecacerts truststore (or if your 
client has it's own truststore, it needs to be imported there).


Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: SSL Handshake Exceptions

2015-05-11 Thread Andy Wang
Honestly, I'm going to be a little purposefully obtuse here. 
Manipulating your trust store is a security step.  You really need to 
understand what you're doing and why, so I'd suggest you do some google 
searches to read up on it using keywords pulled out of my original response.


I will add one more thing.  Your original stack trace showed the 
webserver to be some com.redwood.r2w class.  Quick googling finds that 
this is some commercial product.  You might want to try the support 
channels from your vendor as they may have special instructions for 
trusting self-signed certs.


Andy


On 05/11/2015 02:30 PM, jairaj kamal wrote:

Hi,

Can you share the steps to import the certificate into the jssecacerts
truststore, my client is webserver.

*Jairaj Kamal*


On Mon, May 11, 2015 at 2:16 PM, Andy Wang aw...@ptc.com wrote:




On 05/11/2015 01:24 PM, jairaj kamal wrote:


javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target



This usually means that the ssl client (the client that's originating the
direct connection to the ssl server) is unable to construct a proper
certificate trust path for the server.

As you noted, you used a self-signed cert.  This means that you need to
import the certificate into the jssecacerts truststore (or if your client
has it's own truststore, it needs to be imported there).

Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org






-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Configure Tomcat 7 using Apache 2.4.6

2015-04-08 Thread Andy Wang



On 04/08/2015 12:43 PM, Leggio, Andrew wrote:

I am trying to get tomcat to work under Apache.  I have verified that
tomcat is listening on port 8009.

I tried doing the following:

*Apache Web Server Settings*

Add the following to the */etc/httpd/conf.d/proxy_ajp.conf* file or if
that file does not exist you can add it to the end of the
*/etc/httpd/conf/httpd.conf* file instead:

IfModule mod_proxy_ajp.c

   ProxyPass / ajp://localhost:8009/

/IfModule



Did you actually load the mod_proxy_ajp module?
Given the directory structure of the configuration files you're likely 
using a distribution bundled apache httpd.  Given that you'd probably 
get a bit more help from the resources for that distribution.  But it's 
most likely that you need to ensure that the module is loaded.


Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: AJP Connector : question on mod_proxy_ajp

2015-03-31 Thread Andy Wang

On 03/31/2015 04:11 PM, Rainer Jung wrote:

Am 31.03.2015 um 22:47 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

André,

On 3/31/15 3:41 PM, André Warnier wrote:

I have a question of my own.


??!


+1


Tomcat 6.x/7.x/8.x.

Until now, we have been using mostly the Apache httpd mod_jk
connector to Tomcat. And we have been using the Tomcat AJP
Connector's 'tomcatAuthentication=false' setting, to propagate
the authenticated user from httpd to Tomcat.

Now we have a case where we must use the Apache httpd
mod_proxy_ajp connector instead of mod_jk, and I want to make sure
that the working of the AJP Connector's attribute
tomcatAuthentication remains the same in that context. Does it ?


Not automatically IIRC.


Mostly the same.


And do we have to specify anything special in the httpd
configuration for mod_proxy_ajp to make it so, or is the
authenticated user always passed to Tomcat by default, as seems to
be implied here :

http://httpd.apache.org/docs/2.2/mod/mod_proxy_ajp.html Attributes
?remote_user0x03String


mod_jk sends the user's authentication information over in the
REMOTE_USER field, so you'd just make sure that the REMOTE_USER field
is being sent using mod_proxy_ajp. I'm not exactly sure what the
incantation is for telling mod_proxy_ajp to send that field.

You could try to see if you get a REMOTE_USER without any additional
configuration on the httpd side. Note that REMOTE_USER will not show
up if you call request.getAttributeNames -- you have to explicitly
call request.getAttribute(REMOTE_USER) if you want to get it back.


mod_jk and mod_proxy_ajp by default forward the user member of the
request_rec struct as the AJP attribute SC_A_REMOTE_USER to Tomcat. If
tomcatAuthentication is false, Tomcat uses this attribute instead of
doing its own authentication.

With mod_jk you are a bit more flexible, because you can e.g. configure
it to take the user name from an Apache environment variable you choose
instead of where Apache puts it when its builtin authentication
procedures are used. But for the typical setup it should work for both
modules out of the box.


We've maintained mod_proxy_ajp and mod_jk configuration in our 
application for a while now.  We only officially support mod_jk, but 
yeah for basic web server protocol based authentication in Apache it 
just works ootb with pretty much the same with both mod_jk and 
mod_proxy_ajp.  I just did a test now and the only difference in 
configuration is JkMount for mod_jk or the Proxy* directives for 
mod_proxy_ajp.  Authentication just automatically worked between the two.


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Problem with nsapi_redirect.so (1.2.37) on iPlanet 7.0.15 and Solaris 11

2013-02-21 Thread Andy Wang

On 02/19/2013 11:13 AM, Rainer Jung wrote:

It will be tedious, but if we want to check whether the OS disallows
some syscalls when running as suid under root, then truss should provide
insight.

So run iPlanet (the iPlanet start script) under truss -f -o
/some/path/tr.out once in the working config and once in the non-working
one and try to find differences w.r.t. to syscalls that return an error.

Once you know what you are looking after, the additional truss flags -v
all -w all -r all will provide aditional insight (and a huge volume of
output).


I was hoping to avoid truss and that someone else had experienced this, 
or had an idea of what might be new in Solaris 11 that would cause this :)


This is in the case where it works:
8435/2: read(7, 0xFC1FDF50, 4096) Err#11 EAGAIN
8435/3: lwp_create()(returning as new lwp ...)  = 0
8435/3: setustack(0xFC8D0AA0)
8435/3: schedctl()  = 0xFC975020
8435/3: open(/etc/netconfig, O_RDONLY)= 13
8435/3: fstat64(13, 0xFC07F720) = 0
8435/3: fstat64(13, 0xFC07F630) = 0

thread 3 was created and that's what does the systhread_start stuff.

In the case where it doesn't work:
8207/2: read(8, 0xFC1FDF50, 4096) Err#11 EAGAIN
8205:   pollsys(0x080B9008, 1, 0x080471D8, 0x)  = 1
8205:   accept(4, 0x, 0x, SOV_DEFAULT)  = 5
8192:   waitid(P_ALL, 0, 0x080461D0, 
WEXITED|WTRAPPED|WSTOPPED|WCONTINUED) (sleeping...)

8203:   sigsuspend(0x08047220)  (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x)  = 0
8207/2: pollsys(0xFC1FDEB0, 1, 0xFC1FDE78, 0x) (sleeping...)
8205:   pollsys(0x080B9008, 2, 0x080471D8, 0x) (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x) (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x)  = 0
8207/1: pollsys(0x, 0, 0x080427F8, 0x) (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x)  = 0
8207/1: pollsys(0x, 0, 0x080427F8, 0x) (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x)  = 0
8207/1: pollsys(0x, 0, 0x080427F8, 0x) (sleeping...)
8207/1: pollsys(0x, 0, 0x080427F8, 0x)  = 0
8207/2: pollsys(0xFC1FDEB0, 1, 0xFC1FDE78, 0x)  = 0

So something is definitely preventing the systhread_start call to even 
get as far as the system call to lwp_create.


Unfortunately, this is now quite a bit beyond me.   Might have to figure 
out our partner contacts at oracle to see what's up.


Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Problem with nsapi_redirect.so (1.2.37) on iPlanet 7.0.15 and Solaris 11

2013-02-19 Thread Andy Wang

On 02/19/2013 12:11 AM, Mladen Turk wrote:

On 02/18/2013 10:47 PM, Andy Wang wrote:


If I execute startserv as the non-privileged user rather than root or 
do this on Solaris 10, no problems.
Any ideas why systhread_start (this is an iPlanet NSAPI function) 
would fail here as root?




Did you tried to check the ulimit.
Seems like webservd once when switched to non privileged user cannot 
create threads

either because of some security settings or lack of resources.


Yeah, sorry, I should have mentioned it.
-u is 29995
-n is 1024
both are identical for the root role or the webservd user.

I'm not that familiar with Solaris 11 and what they did to the root as a 
role instead of a regular user so I wasn't sure what other resource 
configuration to look at.  What does confuse me though is the thread 
pool stuff in the webserver itself (as well as the built-in servlet 
engine) seem fully functional so this issue seems specific to the 
jk_init call to systhread_start.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Problem with nsapi_redirect.so (1.2.37) on iPlanet 7.0.15 and Solaris 11

2013-02-19 Thread Andy Wang

On 02/19/2013 12:11 AM, Mladen Turk wrote:


Did you tried to check the ulimit.
Seems like webservd once when switched to non privileged user cannot 
create threads

either because of some security settings or lack of resources.




I'm not so sure that this sequence is what's occurring.
When the jk_init function is called, the non-privileged webservd process 
hasn't actually been spawned yet.  As far as I can tell, it's still 
running within the root owned webservd parent process.


So at this point, I don't think it's switched to the non-privileged user 
yet.


Of course that still doesn't make sense why the root user wouldn't be 
able to create a thread with systhread_start.


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Problem with nsapi_redirect.so (1.2.37) on iPlanet 7.0.15 and Solaris 11

2013-02-18 Thread Andy Wang
I'm having some problems getting the nsapi_redirect.dll working with 
iPlanet 7.0.15 on solaris 11.  The problem seems specifically related to 
Solaris 11 and only if I try to install/run the server as root (using 
webservd as the non-privileged user).


When I do so (and after enabling debug jk loglevel) during jk_init I see 
the following messages in the jk log:


[Mon Feb 18 22:36:25.588 2013] [20669:1] [debug] 
jk_init::jk_nsapi_plugin.c (337): jk_init, a second passed


repeated a bunch of times

and in the iPlanet error log the following:
[18/Feb/2013:22:37:24] failure (20669): CORE2254: Error running Init 
function jk_init


Looking at the code:
s = systhread_start(SYSTHREAD_DEFAULT_PRIORITY,
0, init_workers_on_other_threads, 
init_map);

for (sleep_cnt = 0; sleep_cnt  60; sleep_cnt++) {
systhread_sleep(1000);
jk_log(logger, JK_LOG_DEBUG, jk_init, a second passed);
if (init_on_other_thread_is_done) {
break;
}
}

I can find no evidence in the jklog that the 
init_workers_on_other_threads thread actually started properly.


Here's the cute thing.  It only happens on Solaris 11 with the server 
run as root, and configured to run as a non-privileged user (any 
non-privileged user has this issue)


If I execute startserv as the non-privileged user rather than root or do 
this on Solaris 10, no problems.


Any ideas why systhread_start (this is an iPlanet NSAPI function) would 
fail here as root?


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-14 Thread Andy Wang

On 05/11/2012 04:51 AM, Mladen Turk wrote:


I was following this tread and was hoping that someone will say:
Do not use workstation grade software for server applications
but no. XP (Win7 falls in the same category) has good network stack
but focused on client applications.
There is a good reason why MS charges extra $$ for server versions
since they make sure your server application don't perform well on
workstation software.

The problem is, we've seen the behaviour reported sporadically from 
customers on windows 2003 server.  The only case I've been able to find 
to reproduce it was this XP system.


I'm not sure what to make of the customer reports of the slow downs 
but at this point, I'm going to have to ask them to use something 
like ab.exe to do the downloads instead of Internet Explorer (most of 
them use IE to do it). Maybe there's some stupidity

with IE.



IE tries to open several concurrent connection at once (RFC says that 
number is max 3)

I'm sure you can simulate that with 'ab -c 10 -k -n 1 ...'

I'm not sure how this would be relevant though as the behavior is 
reproducible with a single connection to download a large file (no 
concurrenct connections to this site or any other site.


There is nothing wrong with tomcat, mod_jk or httpd.
It's just your use case.


I'm not convinced that there isn't something odd going on but for now 
I'll ignore it.  I may have a stupid use case, but we have had reports 
out in the field of what appear to be perfectly sane use cases that 
don't perform.  i just can't reproduce them.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-10 Thread Andy Wang
I have solid numbers that I will e-mail in a follow up  by itself so 
it's not lossed in the shuffle.


Some answers to the comments inline.

Thanks,
Andy



Do you mean that Tomcat performance appears to be the same regardless
of version? That's both good and bad... I thought there were some
performance improvements to the connectors from 5.5-  6.0. Maybe that
was 4.x-5.5.

Yeah, the download performance through AJP and through HTTP connectors 
is fairly consistent from 5.5-6.0-7.0

Tomcat7 is using JDK 1.7 and this is interesting.  The benchmarks
with tomcat7+jdk1.7 vary widely across the board (both through ajp
and direct http to tomcat) from 30s-40sMB/s. Java 1.6 seems alot
more consistent.  Not sure why yet.

That is interesting. On the other hand, the server /is/ on a virtual
machine, and you never know what other processes are stealing focus.
Many VMs are notorious for bad IO throughput (I'm looking at you, 
OpenVZ).


I'm using vmware 5 esxi servers on a pretty beefy system.  8 2.8ghz xeon 
cores, 32gb memory and I'm only running a single VM at a time.  Each VM 
is allocated 4GB of memory and 2 cores.  Should be more than enough to 
handle serving up a 450MB iso file.

Have you tried bare hardware?


No, unfortunately I don't have access to bare hardware, but as I 
mentioned, the vmware 5 esxi servers are pretty beefy.


How much control do you have over the VM server?  Seems to me that 
resources/performance could be not only affected by other VMs but also 
intentionally throttled.


I have pretty much full control over the VM servers.  There is no 
throttling going on, no resource limits and as I said, I shut down all 
the other VMs just to rule out contention with them.

Some tips for this kind of testing:

1. Don't run ab on localhost: all the numbers will be worthless
2. Run ab with a range of concurrencies, including c=1
3. Make /lots/ of requests. IMO, 5 requests is really a pinhole
analysis. I would make as many requests as you can over 10 minutes and
see what the throughput ends up being.

I actually ran localhost just out of curiosity and while the numbers are 
faster, the performance difference is consistent with a remote host.  I 
don't want to make alot of requests.  I want to see throughput of large 
files becuase this is where we're seeing the slow downs.  Concurrency is 
not an issue.

I'll compare Windows XP performance and Windows 2008 performance
and after that I'll do the same on a Linux VM to get a better
comparison.

It will be good to see.

I decided to just do windows.  The numbers I got are enough for me to 
not care at this point.

I plan on doing both local ab requests as well as remote.  The
problem with remote is that our network is busy, so it may account
for some variations but I don't think I can get our IT to segment
me anything for this purpose :(.

Just get a crossover cable and use static IP addresses.

Unfortunately no access to additional hardware.



I'm not so concerned about a 25% hit.  I'm really more concerned
with the drop to 4-5MB/s over time that seems to happen.

Does this happen locally or only remotely? I wonder if you're hitting
some kind of traffic-shaping or QOS rules on your own internal network.



This type of hit occurs for our customers both locally and remotely.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-10 Thread Andy Wang
So I cannot reproduce the slow down to 4-5MB/s on the same VM I was able 
to reproduce it on once I copied the VM to an adequate vmware server.  
But I do see some neat numbers in case people care.


I ran with ab -5 directly against apache, against a url mapped to ajp as 
well as direct to the http connectors.
The numbers were consistent between tomcat 5.0, 5.5, 6.0 and 7.0 so I'm 
only going to post one set of numbers for tomcat.


This is against a windows XP SP3 with no sendbuffersize, tcpbuffersize 
or scaling window tuning:

Direct to apache http:
Transfer rate:  21925.90 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01   0.3  1   1
Processing: 19593 20077 474.0  20045   20855
Waiting:12   0.6  2   3
Total:  19593 20078 474.1  20046   20856

Through AJP:
Transfer rate:  36732.95 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01   0.2  1   1
Processing: 10662 11984 879.6  12227   12975
Waiting:45   0.5  5   5
Total:  10663 11984 879.7  12227   12976

Direct to tomcat http:
Transfer rate:  30968.31 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01   1.6  0   4
Processing: 11326 14214 2655.1  15565   16952
Waiting:35   1.6  5   7
Total:  11326 14215 2654.3  15565   16952


Note how much better the both the tomcat results are than direct 
apache.  Most interestingly, note how much better AJP is than direct 
tomcat HTTP connector.  That was quite unexpected.


Here are the results from a Windows 2008 system on the same vm host:
Direct to apache http:
Transfer rate:  57968.69 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01   0.1  1   1
Processing:  7453 7594 181.8   75757890
Waiting:2   12  18.5  6  45
Total:   7453 7594 181.8   75767890

Through AJP:
Transfer rate:  31532.82 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:01   0.2  1   1
Processing: 10723 13960 2813.3  15795   16409
Waiting:35   3.1  4  10
Total:  10724 13961 2813.4  15795   16410

Direct to Tomcat http:
Transfer rate:  37742.45 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:00   0.1  0   1
Processing: 10974 11664 452.1  11812   12192
Waiting:2   14  25.2  4  59
Total:  10974 11664 452.2  11813   12192

Tomcat http averaged better times, BUT ajp was able to perform faster at 
times.  Direct HTTP to apache is way faster though but I think that's to 
be expected.


So realistically, I think the 2008 numbers make sense to me and the XP 
numbers show that XPs tcp stack is a piece of crap (which I think alot 
of people already know).


I'm not sure what to make of the customer reports of the slow downs but 
at this point, I'm going to have to ask them to use something like 
ab.exe to do the downloads instead of Internet Explorer (most of them 
use IE to do it).  Maybe there's some stupidity with IE.


Anyways, I'm closing the book on this (with a bookmark just in case) but 
wanted to provide the numbers in case people were curious what I got.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-08 Thread Andy Wang

On 05/07/2012 06:50 PM, Andy Wang wrote:

On 05/07/2012 06:06 PM, André Warnier wrote:


Considering your setup, it should not be too hard to set up a 
download of the same file file directly from Tomcat (through its HTTP 
Connector), to compare that with your two previous ways.  This way, 
you could make sure if it is Tomcat, or the mod_jk/AJP link which is 
the issue.


Also, still considering your setup, it should be possible to 
configure things so that these file downloads are handled directly by 
Apache httpd, since that seems to satisfy your expectations.  mod_jk 
JkMount/JkUnMount rules (*) should make that possible, no ?
Have to be a bit careful not to introduce security holes, and I am 
assuming that the files are static (which may be wrong here).


(*) or the Location .. + setHandler jakarta-servlet configuration 
variation


Thanks for the http connector idea.  I forgot about that.  The primary 
reason why i'm using tomcat to download a static file is really for 
testing purposes to confirm performance between mod_jk and direct 
apache.  we have servlets that stream content files that see the same 
massive performance hit so in our actual use case it's not static 
files :(.  I'm thinking this would be a valid test to help at least 
tweak mod_jk to it's potential.


We've checked and double checked the buffering code of the servlets 
and it all looks fine AND the performance is fine on Linux and the 
speed characteristics are identical to serving static files through 
tomcat + mod_jk so I'm hoping that it's an apples to apples comparison.


Andy



Through the HTTP connector the performance is similar to apache direct.  
30mb/s


So there's something interesting going on specifically ajp.

Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-08 Thread Andy Wang

Which connector are you using? If you have APR available, AJP should use
the APR connector by default. Do you know if you are using APR/native?
If not, consider trying it, and use sendFile=true. I'm not sure if it
will improve anything because the real problem might be the actual
buffering between Tomcat and httpd.
Unfortunately, we found some odd problems with APR/native a long time 
ago and haven't ever gone back to it.  I don't have it available for the 
systems I need this working on to try

Wireshark should allow you to sniff traffic on localhost.


http://wiki.wireshark.org/CaptureSetup/Loopback
It doesn't work on Windows.  I've tried the loopback adapter piece, but 
it's quite obnoxious.

What am I saying, Windows is quite obnoxious :)

Andy


Re: Slow downloads through mod_jk on Windows XP

2012-05-08 Thread Andy Wang

After playing with this a bit more (testing this time against tomcat 7.0.27)
the ajpPacketSize has zero effect on the speed.

Downloading a large file through mod_jk to tomcat looks like this:
2012-05-08 16:01:22 (15.0 MB/s) - “sol-11--text-x86.iso.8” saved 
[450799616/450799616]


Downloading the same large file directly through apache looks like:
2012-05-08 16:01:58 (19.3 MB/s) - “sol-11--text-x86.iso.11” saved 
[450799616/450799616]



the numbers are pretty consistent. So apache still beats tomcat by a 
good chunk but it's much much better than with tomcat 5.0 where the 
through tomcat numbers were about half that.


I'm still wondering if there's something that can be tweaked in the MS 
TCP/IP stack to bring the two together closer.


Andy

On 05/08/2012 02:06 PM, Andy Wang wrote:

On 05/07/2012 06:50 PM, Andy Wang wrote:

On 05/07/2012 06:06 PM, André Warnier wrote:


Considering your setup, it should not be too hard to set up a 
download of the same file file directly from Tomcat (through its 
HTTP Connector), to compare that with your two previous ways. This 
way, you could make sure if it is Tomcat, or the mod_jk/AJP link 
which is the issue.


Also, still considering your setup, it should be possible to 
configure things so that these file downloads are handled directly 
by Apache httpd, since that seems to satisfy your expectations. 
mod_jk JkMount/JkUnMount rules (*) should make that possible, no ?
Have to be a bit careful not to introduce security holes, and I am 
assuming that the files are static (which may be wrong here).


(*) or the Location .. + setHandler jakarta-servlet 
configuration variation


Thanks for the http connector idea. I forgot about that. The primary 
reason why i'm using tomcat to download a static file is really for 
testing purposes to confirm performance between mod_jk and direct 
apache. we have servlets that stream content files that see the same 
massive performance hit so in our actual use case it's not static 
files :(. I'm thinking this would be a valid test to help at least 
tweak mod_jk to it's potential.


We've checked and double checked the buffering code of the servlets 
and it all looks fine AND the performance is fine on Linux and the 
speed characteristics are identical to serving static files through 
tomcat + mod_jk so I'm hoping that it's an apples to apples comparison.


Andy



Through the HTTP connector the performance is similar to apache 
direct. 30mb/s


So there's something interesting going on specifically ajp.

Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org






-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-08 Thread Andy Wang


He did that previously, and the result seemed to be that Tomcat alone 
was comparable to httpd alone, and both were better than httpd/mod_jk 
+ Tomcat; which is indeed physically to be expected : more tubing, 
less throughput (excepting quantum tunelling effects of course).

The question is more : how much of a degradation ?
15.0/19.3 = 0.77 = 23% less throughput.  I don't know if this is a 
good chunk or the to be expected kind of degradation.


According to the (looking seriously outdated) AJP protocol 
documentation at 
http://tomcat.apache.org/connectors-doc/ajp/ajpv13a.html, it would 
seem that the maximum data size chunk which AJP can send back from 
Tomcat to the front-end httpd is 8K at a time.  So AJP might not be 
very well suited, when it comes to send back big blobs of data.


Rainer would need to confirm if that is still the case now.

One earlier message seemed to indicate that this httpd/mod_jk+tomcat 
deficit only happened under Windows though, and not under Linux.  If 
that is confirmed, maybe there is some subtle difference in how the 
TCP/IP stack is being used under the one vs the other ?

Thanks for that summary,
That's about what I'm seeing.  I just created a directory containing 
Apache configured to serve up the iso file directly as well as through 
three tomcats (tomcat5.5, tomcat6, and tomcat7) to see if the behavior 
is related to tomcat that I can easily copy between different Windows 
systems.  Initial benchmarks seem to show that the behavior between 
tomcats is not an issue.Tomcat7 is using JDK 1.7 and this is 
interesting.  The benchmarks with tomcat7+jdk1.7 vary widely across the 
board (both through ajp and direct http to tomcat) from 30s-40sMB/s.  
Java 1.6 seems alot more consistent.  Not sure why yet.


I've also moved off the crappy Windows XP VM I was provided to a more 
recent Windows 2008 VM as well as a fresh Windows XP SP3 VM.  In past 
experience it seems windows XP and windows 2003 were the worst of the 
bunch with the ajp downloads dropping as low as 4-5MB/s over time.


I'm going to run a barrage of tests and provide the numbers.  Do you 
think ab -n 5 and allowing ab to average the values of 5 hits for the 
~440MB iso is a sound average?


I'll compare Windows XP performance and Windows 2008 performance and 
after that I'll do the same on a Linux VM to get a better comparison.


I also did bump up the ajpPacket size to 64K with no noticeable change 
to the benchmark numbers.  So while 8k seems crappy it doesn't seem to 
be an issue.  Given that apache and tomcat are both local I wouldn't 
expect that to be a big problem with 8k chunks given the near 
non-existent latency of local connections.


I plan on doing both local ab requests as well as remote.  The problem 
with remote is that our network is busy, so it may account for some 
variations but I don't think I can get our IT to segment me anything for 
this purpose :(.


I'm not so concerned about a 25% hit.  I'm really more concerned with 
the drop to 4-5MB/s over time that seems to happen.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Slow downloads through mod_jk on Windows XP

2012-05-07 Thread Andy Wang

Hi all,
We've had a number of cases of people reporting to us that file 
downloads are slow when passed through tomcat and I've not been able to 
reproduce the problem on Linux but finally was provided a windows XP VM 
that was able to reproduce the problem.


This is Apache 2.2.22 and mod_jk 1.2.32 with tomcat 5.0.30 (yeah, I know 
this is ancient.  I'll try with something newer tomorrow).


I have two URLs configured both with an identical iso file (roughly 
450MB) in size.
When I go through the URL configured to run directly through Apache the 
download speeds on the file are around 30MB/s which is reasonable for 
what I would expect on the network connection between the client and the 
web server.


The second URL is JkMount'ed through to tomcat and download the same 
file drops down to 5-6 MB/s (starts at 10-ish MB/s).


The one thing I haven't tried configuring is upping the ajp packet size 
because the version of tomcat I was provided doesn't support that, but 
otherwise the operating system is tuned so the TcpWindowSize is 131072.


Unfortunately, this is windows, and both apache and tomcat are local on 
the server so I can't sniff the packets between the two systems with 
wireshark.


I don't see anything on the mod_jk workers.properties configuration that 
deals with buffer sizes.


Hoping someone has some ideas on what to look at that might help tweak 
this thing to perform similarly to Apache direct downloads.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Slow downloads through mod_jk on Windows XP

2012-05-07 Thread Andy Wang

On 05/07/2012 06:06 PM, André Warnier wrote:

Andy Wang wrote:

Hi all,
We've had a number of cases of people reporting to us that file 
downloads are slow when passed through tomcat and I've not been able 
to reproduce the problem on Linux but finally was provided a windows 
XP VM that was able to reproduce the problem.


This is Apache 2.2.22 and mod_jk 1.2.32 with tomcat 5.0.30 (yeah, I 
know this is ancient.  I'll try with something newer tomorrow).


I have two URLs configured both with an identical iso file (roughly 
450MB) in size.
When I go through the URL configured to run directly through Apache 
the download speeds on the file are around 30MB/s which is reasonable 
for what I would expect on the network connection between the client 
and the web server.


The second URL is JkMount'ed through to tomcat and download the same 
file drops down to 5-6 MB/s (starts at 10-ish MB/s).


The one thing I haven't tried configuring is upping the ajp packet 
size because the version of tomcat I was provided doesn't support 
that, but otherwise the operating system is tuned so the 
TcpWindowSize is 131072.


Unfortunately, this is windows, and both apache and tomcat are local 
on the server so I can't sniff the packets between the two systems 
with wireshark.


I don't see anything on the mod_jk workers.properties configuration 
that deals with buffer sizes.


Hoping someone has some ideas on what to look at that might help 
tweak this thing to perform similarly to Apache direct downloads.




Considering your setup, it should not be too hard to set up a download 
of the same file file directly from Tomcat (through its HTTP 
Connector), to compare that with your two previous ways.  This way, 
you could make sure if it is Tomcat, or the mod_jk/AJP link which is 
the issue.


Also, still considering your setup, it should be possible to configure 
things so that these file downloads are handled directly by Apache 
httpd, since that seems to satisfy your expectations.  mod_jk 
JkMount/JkUnMount rules (*) should make that possible, no ?
Have to be a bit careful not to introduce security holes, and I am 
assuming that the files are static (which may be wrong here).


(*) or the Location .. + setHandler jakarta-servlet configuration 
variation


Thanks for the http connector idea.  I forgot about that.  The primary 
reason why i'm using tomcat to download a static file is really for 
testing purposes to confirm performance between mod_jk and direct 
apache.  we have servlets that stream content files that see the same 
massive performance hit so in our actual use case it's not static files 
:(.  I'm thinking this would be a valid test to help at least tweak 
mod_jk to it's potential.


We've checked and double checked the buffering code of the servlets and 
it all looks fine AND the performance is fine on Linux and the speed 
characteristics are identical to serving static files through tomcat + 
mod_jk so I'm hoping that it's an apples to apples comparison.


Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Possible race condition with mod_jk + multiple workers in recovery mode

2011-01-21 Thread Andy Wang
On 01/17/2011 04:53 AM, Rainer Jung wrote:
 On 13.01.2011 00:36, Andy Wang wrote:
 Aahh, having the maintenance thread do a periodic probe would be
 awesome.

 I see what you mean about parallel probing delaying request handling,
 but what would you think about modifying the loop so that after going
 through all the JK_WORKER_USABLE() workers to retry the PROBING
 workers.  At that point if none of the workers are up, then it seems
 like parallel probing wouldn't be a bad idea would it?

 If none of the workers are up, then normally forced recovery kicks
 in, i.e. the workers that erred will be put into mode
 JK_LB_STATE_FORCE, which is this worker is in error, but use it
 nevertheless, because there is no good worker left.

 Regards,

 Rainer
I'm having a really hard time reproducing it, but the logs from one of
the times I saw it seemed to imply that the worker being tried by the
first thread was put into PROBE state, and then another thread came in,
and completed, with an error state because none of the other workers are
alive before the first thread's worker completed and it skipped the
worker the first thread was using becaause it was put into PROBE state. 
So the first worker thread's worker was never used by the second
thread's worker resulting in the second thread.

There was never a chance for the first worker to go into FORCE mode.  It
was set into PROBE mode, and somehow thread 2 snuck in and before the
1st thread finished.

Andy

Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Possible race condition with mod_jk + multiple workers in recovery mode

2011-01-12 Thread Andy Wang
On 01/12/2011 03:36 PM, Rainer Jung wrote:

 That was meant as an improvement. Recovery is only used, when a
 worker has been in error state for enough time (by default 60 seconds)
 and we want to find out whether it is still in error or not. mod_jk
 has no active probing with test URLs, so if the 60 seconds passed it
 marks the worker/instance a recoverable. Then it waits for the next
 request which is eligible to be handled by that instance (either
 carrying an old session iformation pointing to that instance or being
 freely balancable), and sends it there, marking the worker as being
 probed. If it succeeds, fine we can set the instance to OK. If not the
 instance goes back to error and the request will be sent to some other
 OK instance.

 Most of the time an instance will be in error for longer time, so
 probing in parallel is not a good idea, e.g. if the error state leads
 to delays in request handling. OTOH if the worker is fine now, it
 should ususally respond very quickly, so the time window where request
 could have been handled by the worker but weren't is very small. This
 seems fine to me.

 All this will be nicer once we add a probing feature to the
 maintenance thread which will probe the erroneous workers via a
 background thread and recover the worker as soon as the probing succeeds.


Aahh, having the maintenance thread do a periodic probe would be awesome.

I see what you mean about parallel probing delaying request handling,
but what would you think about modifying the loop so that after going
through all the JK_WORKER_USABLE() workers to retry the PROBING
workers.  At that point if none of the workers are up, then it seems
like parallel probing wouldn't be a bad idea would it?

Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Possible race condition with mod_jk + multiple workers in recovery mode

2011-01-11 Thread Andy Wang
I'm still digging, but I thought I'd send this along to the mailing list
while I try to decipher the mod_jk code.

We're using tomcat-connector 1.2.31 on apache 2.2.17 on a Linux system.

Log file is at this URL:
http://www.moonteeth.com/~dopey/tomcat/mod_jk.log
http://www.moonteeth.com/%7Edopey/tomcat/mod_jk.log

The worker configuration consists of a load balanced worker with 9
workers (tomcat1-9).  At any given time, usually only one tomcat is in
use.  In the case of the log, all the workers are in recovery state
(apache started before tomcat, and someone hit the page so the tomcats
are all down).

In the log file the requests
5356:1146444096
/Windchill/servlet/WindchillAuthGW/wt.httpgw.HTTPAuthentication/login
and
5357:1138932032 /Windchill/servlet/WindchillGW/wt.httpgw.HTTPServer/ping
come in almost simultaneously.   5357:1138932032 completes first and
picks up tomcat1, recovers it and uses it.

However 5356:1146444096 never tries worker 1.  5357:1138932032 grabbed
worker 1 before 5356:1146444096 does, and by the time  5356:1146444096
gets the worker list, it never bothers to try tomcat1, just tries all
the other workers.

As I said, I'm still digging at the mod_jk code to try to find out
what's going on, but hoping that maybe someone else has also seen this
problem.

I can't reproduce this on my system unfortunately, and it's quite
intermittent.

Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Possible race condition with mod_jk + multiple workers in recovery mode

2011-01-11 Thread Andy Wang
I'm not sure, but it looks like the service() function in jk_lb_worker.c
calls puts a recovering worker into the JK_LB_STATE_PROBE state and then
doesn't set it to JK_LB_STATE_OK until after the end-service() call.

I think this allows a second thread to come in, and since
JK_WORKER_USABLE() returns false because of the JK_LB_STATE_PROBE state
it never tries to use that worker and the second request thread
completes, then the first request completes and finally marks the worker
as JK_LB_STATE_OK.

Still working on a reproducible state to debug this in, but does this
sound like a possible problem or am I mis-reading what the
end-service() call does here:
service_stat = end-service(end, s, l, is_service_error);

Thanks,
Andy

On 01/11/2011 12:08 PM, Andy Wang wrote:
 I'm still digging, but I thought I'd send this along to the mailing list
 while I try to decipher the mod_jk code.

 We're using tomcat-connector 1.2.31 on apache 2.2.17 on a Linux system.

 Log file is at this URL:
 http://www.moonteeth.com/~dopey/tomcat/mod_jk.log
 http://www.moonteeth.com/%7Edopey/tomcat/mod_jk.log

 The worker configuration consists of a load balanced worker with 9
 workers (tomcat1-9).  At any given time, usually only one tomcat is in
 use.  In the case of the log, all the workers are in recovery state
 (apache started before tomcat, and someone hit the page so the tomcats
 are all down).

 In the log file the requests
 5356:1146444096
 /Windchill/servlet/WindchillAuthGW/wt.httpgw.HTTPAuthentication/login
 and
 5357:1138932032 /Windchill/servlet/WindchillGW/wt.httpgw.HTTPServer/ping
 come in almost simultaneously.   5357:1138932032 completes first and
 picks up tomcat1, recovers it and uses it.

 However 5356:1146444096 never tries worker 1.  5357:1138932032 grabbed
 worker 1 before 5356:1146444096 does, and by the time  5356:1146444096
 gets the worker list, it never bothers to try tomcat1, just tries allaf
 the other workers.

 As I said, I'm still digging at the mod_jk code to try to find out
 what's going on, but hoping that maybe someone else has also seen this
 problem.

 I can't reproduce this on my system unfortunately, and it's quite
 intermittent.

 Andy

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



tomcat connector issue with Sun Web Server 7.0.

2010-10-05 Thread Andy Wang
 Hi all,
We have a configuration where we use the tomcat connector with the Sun
Java System Web Server and I'm noticing an unusual behavior that I've
not noticed with Apache and mod_jk.

When making a request to a URL such as
http://host/webapp/dir/

In apache, the underlying request is made as:
REQ:GET /webapp/dir/

In the Sun Java System Web Server, the underlying request to tomcat gets
the default directory index file added to it, so the AJP get looks like:
REQ:GET /webapp/dir/index.html

Anyone have any idea if it's possible make the SJSWS web server NOT
attempt to add the index.html on to the end of resource when passing the
request over to tomcat via AJP?

Thanks,
Andy



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: JVM goes away

2010-01-11 Thread Andy Wang
dmesg
check if the linux out of memory kill struck you :)

Andy

On 01/11/2010 04:37 PM, Carl wrote:
 This is a new server, a Dell T110 with a Xeon 3440 processor and 4GB memory.  
 I have turned off both the turbo mode and hyperthreading.

 The environment:

 64 bit Slackware Linux

 java version 1.6.0_17
 Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
 Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)

 Tomcat: apache-tomcat-6.0.20

 JAVA_OPTS=-Xms2400m -Xmx2400m -XX:PermSize=512m -XX:MaxPermSize=512m

 I have watched observed the memory usage and general performance with Java 
 VisualVM and have seen nothing strange.  GC seems to be performing well and 
 the memory rarely gets anywhere near the max.

 The server runs well, idling along at 2-5% load, serving jsp's, etc. at a 
 reasonable speed.  Without warning and with no tracks in any log (Tomcat or 
 system) or to the console, the JVM will just go away, disappear.  Sometimes, 
 the system will run for a week, sometimes for only several hours.  Initially, 
 I thought the problem was the turbo or hyperthreading but, no, the problem 
 persists.

 When the JVM goes away, the memory that it held is still being held (as seen 
 from top) but it is nowhere near the machine physical memory.

 The application has been running on an older server (Dell 600SC, 32 bit 
 Slackware, 2GB memory) for several years and, while the application will 
 throw exceptions now and then, it never crashed the JVM.  This leads me to 
 believe the problem has something to do with the 64 bit JVM but, with errors, 
 I can't be certain and don't know what I can do about it except go back to 32 
 bit.

 I plan to reinstall Java tonight but, it would seem if the JVM were 
 corrupted, it simply would not run.

 Any ideas are welcome.

 TIA,

 Carl


   


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: JVM goes away

2010-01-11 Thread Andy Wang
I assume $GB means 4GB :)
With that kind of memory use it doesn't sound entirely like the OOM
killer.  Have you looked around the filesystem for hs_err[pid].pid
files?  This usually is written to the cwd of the java process.  That
might give you ideas if it's a native crash.  If so, it'll have the java
stack, and other native information that might shed some light.

Otherwise, if the Tomcat JVM isn't running as a daemon, is it nohup'ed?

Andy

On 01/11/2010 05:33 PM, Carl wrote:
 Peter and Andy,

 Thanks for your quick responses.

 Memory:  Physical - $GB
Used - 2.4GB to 3.0 GB (according to top... have never
 seen it above 3GB)
Swap - 19GB, none ever used (or, at least I have never
 seen any used.)

 The above are all from top.

 The 2.4GB is after the JVM just crashed (after running less than an
 hour after having run for five days with nary a blip) and I just
 restarted Tomcat (customers are running right now) so it is a little
 higher than normal because it has perhaps .5GB+ unrecovered from the
 point at which the JVM crashed.

 I checked dmsg but saw nothing that looked out of the ordinary.

 I will cut back on the heap and permgen tonight (gonna be a long one.)

 Any ideas are welcome.

 Thanks,

 Carl


 - Original Message - From: Peter Crowther
 peter.crowt...@melandra.com
 To: Tomcat Users List users@tomcat.apache.org
 Sent: Monday, January 11, 2010 6:06 PM
 Subject: Re: JVM goes away


 2010/1/11 Carl c...@etrak-plus.com:
 This is a new server, a Dell T110 with a Xeon 3440 processor and 4GB
 memory. I have turned off both the turbo mode and hyperthreading.

 The environment:

 64 bit Slackware Linux

 java version 1.6.0_17
 Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
 Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)

 Tomcat: apache-tomcat-6.0.20

 JAVA_OPTS=-Xms2400m -Xmx2400m -XX:PermSize=512m -XX:MaxPermSize=512m

 I have watched observed the memory usage and general performance with
 Java VisualVM and have seen nothing strange. GC seems to be
 performing well and the memory rarely gets anywhere near the max.

 The server runs well, idling along at 2-5% load, serving jsp's, etc.
 at a reasonable speed. Without warning and with no tracks in any log
 (Tomcat or system) or to the console, the JVM will just go away,
 disappear. Sometimes, the system will run for a week, sometimes for
 only several hours. Initially, I thought the problem was the turbo or
 hyperthreading but, no, the problem persists.

 When the JVM goes away, the memory that it held is still being held
 (as seen from top) but it is nowhere near the machine physical memory.

 The application has been running on an older server (Dell 600SC, 32
 bit Slackware, 2GB memory) for several years and, while the
 application will throw exceptions now and then, it never crashed the
 JVM. This leads me to believe the problem has something to do with
 the 64 bit JVM but, with errors, I can't be certain and don't know
 what I can do about it except go back to 32 bit.

 I plan to reinstall Java tonight but, it would seem if the JVM were
 corrupted, it simply would not run.

 Any ideas are welcome.

 I'm with Andy: the Linux OOM killer would show those symptoms.  With
 those settings, you're not leaving a lot of memory for the OS.  How
 much swap do you have, and does the same thing happen if you reduce
 the Java heap and permgen space?

 - Peter

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org



 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: IIS isapi_redirect.dll chunked encoding option

2009-09-02 Thread Andy Wang

Rainer Jung wrote:

Difficult to answer. My feeling is, that the code is fine and the
original contributor Tim Whittington seemed to have used basically the
same code for quite some time.

On the other hand only having it in a separate binary will still prevent
most people to use it, so it might still not be used broadly out in the
field.

At least we are not aware of any problem with the chunked encoding code.

Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

  

Ranier,
Thanks for the generally positive answer :).
We're going to try to enable it out of the box on our IIS configurations 
and see how things go, so pretty soon, there'll be alot more machines 
potentially running with the chunked encoding option.


I haven't looked that closely at the code, but was there enough worry 
that it would cause side effects to make it a compile time AND a 
configuration option?


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



IIS isapi_redirect.dll chunked encoding option

2009-08-21 Thread Andy Wang
What are the general thoughts on the stability of the 
enable_chunked_encoding option for the IIS isapi redirector for tomcat 
and IIS?


We're running into a scenario where something is causing IIS to not send 
down the complete response.  Haven't figured out exactly the cause yet, 
but the symptoms are the browser never acknowledges a response was 
complete even though tomcat has fully written the response out through 
to the web server.  Wireshark captures show the last few packets not 
being sent so the response is just a little short of what's to be expected.


One thing we noticed was setting enable_chunked_encoding (with a 
redirectory built for chunked encoding of course) made everything work 
fine so I wanted to get a feel for just how experimental this really is 
and what the general consensus is on it's stability.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: IIS isapi_redirect.dll chunked encoding option

2009-08-21 Thread Andy Wang
Sorry, forgot to mention that.  We're at the latest and greatest 
tomcat-connector version: 1.2.28.


Thanks,
Andy

Peter Crowther wrote:

2009/8/21 Andy Wang aw...@ptc.com:
  

What are the general thoughts on the stability of the
enable_chunked_encoding option for the IIS isapi redirector for tomcat and
IIS?



I suspect it depends on the version ;-).  What are you using?

- Peter

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


  




Re: tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

2009-06-08 Thread Andy Wang

Mladen Turk wrote:

All higher MSVCRT versions has dependency on MSVCRT.dll so if you
build against that you have assurance that CRT functions will come
from the same CRT library regardless of the version used.
I know that stdio functions have problems, so if like mod_jk the
module uses them for logging or reading config files you might
get into trouble. We observed that with mod_jk and it manifested with
weired looking log files.
For example we use VS6 for building Tomcat Native, and it can
work in both JDK5 (compiled with VS6) and JDK6 (compiled with VS2003)
because it depends on MSVCRT.dll only.

In general MSVCRT.dll + MSVCRTxx.dll is OK, however
MSVCRTxx.dll + MSVCRTyy.dll is a very bad idea.



I just googled what it would take to get VS2005 to link against MSVCRT.dll.
It's doable, but not pretty at all.

I can see why you use VS6 to build stuff.

If only VS6 was still available on MSDN.

Thanks to both you and Rainer for filling in the blanks for me.  And I 
thought Linux glibc compatibility was confusing but this Microsoft 
MSVCRT stuff makes glibc problems look easy to solve  :)


Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

2009-06-05 Thread Andy Wang

Mladen, Rainer,
Thanks for pointing out the crlf thing, I discovered that myself last 
night and was going to update this thread this morning but you beat me 
to it.  :)


I just automatically, use the tarballs since I'm normally a Unix guy.  I 
figure this out when I inadvertently used the Apache tarball instead of 
win zip and wondered why I got the same error trying to load the .dsps.


Mladen, I am not anywhere near remotely a Windows guy.  This is my first 
experience compiling stuff in a Microsoft world since using Turbo C++ 
back in college, and that was DOS :).
With that in mind, can you elaborate on what the VS2005 MSVCRT71 issues 
might be?  The problem is, Visual Studio 6 is not available from MSDN, 
and we don't have it readily available here.  It appears to be a product 
circa 1998, so I thought a newer compiler might not be a bad idea.


I don't know if you recall an e-mail from Jess Holle regarding a 
quieter logging patch that he asked for comments on.  Until we have 
time to look into Rainer's idea of stopping the nodes via the mod_jk 
status worker, we're using our patch for now, thus the need to build our 
own mod_jk.


Thanks,
Andy

Mladen Turk wrote:

Andy Wang wrote:

Hi all,
I was able to get mod_jk building fine using Makefile.vc, but 
couldn't get the .dsp file loaded into Visual Studio 2005.


If you are using HTTPD binaries from ASF use the Visual Studio 6
and Platform SDK (Windows 2003 R2 inclusive)

VS 2005 will force usage of MSVCRT71 while, so you'll have
multiple MSVCRT versions compiled in, which might cause
some nasty logging issues.

BTW, what's wrong with official mod_jk binaries?

Regards

--
^TM


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

2009-06-05 Thread Andy Wang

Rainer Jung wrote:

Mladen might like to elaborate more, but in short: MS binaries need a
Microsoft Visual C++ Runtime library to run with. Those are called
msvcrt. The library version used is determined during the dynamic
linking done with visual studio.

Now if you use an extensible application like Apache, and the web server
and the modules you want to load are compiled with different Visual
Studio versions, multiple and possibly incompatible version of the
msvcrt libs will be loaded during runtime.

Until now, the official downloads of the Apache web server are compiled
with VS6, so our mod_jk Windows binaries are also compiled with VS6 in
order to circumvent possible havoc.

I did see mod_jk compiled with newer VS version running in the standard
Windows Apache from the web server download page, but you *might* run
into trouble.

Regards,

Rainer

  
Ahh.  I see, I think maybe I was misunderstanding Mladen then.  I 
thought that just building with VS2005 would result in some level of 
multiple msvcrt library confusion.
If, we build our own Apache, and our own mod_jk, and our own everything 
else up the toolchain (zlib, openssl, etc etc), we shouldn't have 
problems with multiple msvcrt dependencies right?


But, that brings up a really interesting point, considering that our 
customers may have their own modules and who knows how they build them.  
I'll have to consider that.


Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

2009-06-04 Thread Andy Wang
I had no dependency issues on libhttpd, I was nmake -f Makefile.vc just 
worked for me once the proper environment variables were set.  I just 
can't load the .dsp file and convert it to a vproj file.


libhttpd should have been built with Apache, not the tomcat-connectors, 
so I guess I'm not sure what you're running into.


Andy


Martin Gainty wrote:

Andy

there is a dependency for libhttpd 
but when you try to build libhttpd you are displayed this error


Configuration: libhttpd - Win32 Release
Creating include/os.h
Generating test_char.h from gen_test_char.exe
'.\server\gen_test_char.exe' is not recognized as an internal or external 
command,
operable program or batch file.
Error executing c:\windows\system32\cmd.exe.

libhttpd.dll - 1 error(s), 0 warning(s)

it appears we the project libhttpd cannot be built

which prevents mod_jk from building 


advice?
Martin Gainty 
__ 
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
 
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.

Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le 
destinataire prévu, nous te demandons avec bonté que pour satisfaire informez 
l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est 
interdite. Ce message sert à l'information seulement et n'aura pas n'importe 
quel effet légalement obligatoire. Étant donné que les email peuvent facilement 
être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité 
pour le contenu fourni.




  

Date: Wed, 3 Jun 2009 18:33:22 -0500
From: aw...@ptc.com
To: users@tomcat.apache.org
Subject: tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

Hi all,
I was able to get mod_jk building fine using Makefile.vc, but couldn't 
get the .dsp file loaded into Visual Studio 2005.  Anyone know if 
there's a trick to this, or should I just not care (it does build and 
seem to work fine with the Makefile).


When Visual Studio 2005 tries to convert mod_jk.dsp to the newer format 
it complains with a

Cannot load the project due to a corrupt project file

popup.

Does anyone really care about this or should I just ignore it?

Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




_
Insert movie times and more without leaving Hotmail®. 
http://windowslive.com/Tutorial/Hotmail/QuickAdd?ocid=TXT_TAGLM_WL_HM_Tutorial_QuickAdd_062009
  




tomcat-connectors mod_jk.dsp file will not load in Visual Studio 2005

2009-06-03 Thread Andy Wang

Hi all,
I was able to get mod_jk building fine using Makefile.vc, but couldn't 
get the .dsp file loaded into Visual Studio 2005.  Anyone know if 
there's a trick to this, or should I just not care (it does build and 
seem to work fine with the Makefile).


When Visual Studio 2005 tries to convert mod_jk.dsp to the newer format 
it complains with a

Cannot load the project due to a corrupt project file

popup.

Does anyone really care about this or should I just ignore it?

Thanks,
Andy

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



nsapi_redirector with Sun Java System Web Server 7.0?

2009-04-03 Thread Andy Wang
Has anyone used the nsapi redirectory to connect SJSWS 7.0 with Tomcat? 
I noted that the documentation all still refers to 6.0, but the README
on the binaries page is somewhat encouraging:

# nsapi_redirector-1.2.28-sjsws6.1sp11.so is for Sun Java System Web
Server (aka Netscape Enterprise Server).
It has been compiled against version 6.1 SP 11 and should work also with
7.0 SP 1.
# nsapi_redirector-1.2.28-sjsws6.1sp8-64bit.so is for 64 Bit Sun Java
System Web Server (aka Netscape Enterprise Server).
It has been compiled against version 6.1 SP 8 and should work also with
7.0 SP 1.

I'll be toying with it a bit now, but wanted to ask if there anyone has
first hand experience.

Thanks,
Andy


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org