Re: How to set tomcat 8.0.20 container character encoding of request and response to UTF-8 intead of ISO-8859-1

[Akshat Tandon]

Thanks Chris for the response,

Basically we don't want to do coding around the filter as it will also
bring us an additional overhead of maintaining the code and follow the
quality process , Ideally we want say some setting to say in server.xml to
set request and response character encoding




*Thanks and RegardsAkshat Tandon*

On Tue, Feb 16, 2016 at 8:13 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> Akshat,
>
> On 2/16/16 7:08 AM, Akshat Tandon wrote:
> > We need to set tomcat 8.0.20 container character encoding of request and
> > response to UTF-8 intead of ISO-8859-1 ,
> >
> >  What is the setting for the same ?
> >
> >  We tried setting as mentioned below ,
> > https://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q1 But that
> requires
> > creating filter etc .
> >
> >  Is there any elegant way where we can just change some configuration set
> > to make it applicable at container level ?
>
> What, you don't like the filter already bundled with Tomcat that you can
> simply configure? It really can't get any easier than that.
>
> -chris
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Does Tomcat need a non-ssl connector?

[msh]

On Tue, Feb 16, 2016 at 08:52:50AM -0500, Christopher Schultz wrote:
> 
> 
> If jsvc is working for you, what's the port number you chose for SSL?
> 8080? (That's odd, most people use 8443). Why not just use the standard
> port for HTTPS?

I want to use 443. I don't want proxies or to see the port number appended to 
the URI.

> 
> -chris
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Ty]

> Exactly which versions were you using when you ran your tests?

According to my notes:  7.0.59, which should've been the most recent
version available at the time (about this time last year).

To clarify:  that was the version we did a heap dump analysis of, and noted
the memory increase related to Manifest objects.  The version used for the
test procedure in this thread's original message -- last performed last
week -- was 7.0.67 and we did not do a heap analysis of it (only noting
that at a glance it resulted in approximately the same amount of heap
increase correlating with JAR scanning as we saw with last year's version).



On Tue, Feb 16, 2016 at 3:55 PM Mark Thomas  wrote:

> On 16/02/2016 20:14, Ty wrote:
> >> JAR scanning should be transient. What was it that was causing the 10-20
> >> fold increase?
> >
> > From memory (this was a while ago), it was related to
> > java.util.jar.Manifest objects, or what they contained/referenced.  I'd
> be
> > glad to do another heap analysis and summarize it if it would help, but
> > even more glad to just skip Tomcat 7 altogether.
>
> There was an issue related to caching of Manifest instances but that
> should have been fixed. Exactly which versions were you using when you
> ran your tests?
>
> Mark
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Tim Watts]

On Tue, 2016-02-16 at 14:31 -0500, Christopher Schultz wrote:

> On 2/16/16 11:50 AM, Dougherty, Gregory T., M.S. wrote:
> > I completely and totally trust my servlet with my data.  I do not 
> > in the least bit trust any other servlet running on that Tomcat 
> > instance.
> 
> Then those servlets shouldn't be deployed into the same application.
> Possibly not even within the same JVM. But okay, let's ignore that
> lack of safety for the time being. We're ignoring all kinds of other
> things as well, so one more doesn't matter.
> 

If I may interject: I think part of the problem here is the OP is using
"servlet" and "webapp" interchangably.  He originally used the term
webapp but then switched to servlet around the time he laid out this
authentication solution.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Mark Thomas]

On 16/02/2016 20:14, Ty wrote:
>> JAR scanning should be transient. What was it that was causing the 10-20
>> fold increase?
> 
> From memory (this was a while ago), it was related to
> java.util.jar.Manifest objects, or what they contained/referenced.  I'd be
> glad to do another heap analysis and summarize it if it would help, but
> even more glad to just skip Tomcat 7 altogether.

There was an issue related to caching of Manifest instances but that
should have been fixed. Exactly which versions were you using when you
ran your tests?

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Ty]

>  if Tomcat had the option to ignore META-INF/resources in JAR files

Perhaps interesting:  of the 27 JAR files in my dummy application, not a
single one has a META-INF/resources.

On Tue, Feb 16, 2016 at 3:40 PM Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Ty,
>
> On 2/16/16 3:11 PM, Ty wrote:
> > George, thanks for sharing your experience, good to know I'm not
> > the first down this path.  I'd be interested to know if you've
> > tested at all with Tomcat 8 and what your findings were, if so.
> > Based on my experience I'd expect you to be in a similar boat to
> > us.
> >
> > Mark, thanks for the reply.  I've uploaded my "dummy" webapp here:
> >
> > http://filebin.ca/2XCPi8oAdy84/test.war
> >
> > As for the JAR scanning / startup times, with Tomcat 8's jarsToScan
> > setting I'm confident we'd be able to manage that, and am happy to
> > leapfrog over Tomcat 7. It's the memory utilization in Tomcat 8
> > specifically, independent of JAR scanning, that's the show-stopper
> > for us.  Let me know if I can provide any other info, and thanks
> > again.
>
> I'm not sure if others will agree, but if Tomcat had the option to
> ignore META-INF/resources in JAR files, you could probably save some
> memory. Since you are currently using Tomcat 6, this setting wouldn't
> affect you at all, since Tomcat 6 didn't expose those files in the way
> Mark describes. On the other hand, since it's a spec-requirement,
> there may be some pushback amongst the team here as to whether or not
> we should implement such a workaround.
>
> We have lots of other settings that can technically be used to violate
> spec-defined behavior, so it's not like there isn't precedent for such
> things.
>
> - -chris
> -BEGIN PGP SIGNATURE-
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlbDlzAACgkQ9CaO5/Lv0PBWDgCfV3efH9stmGMVPFtch9unNeJl
> Dq4AoJwlx4NbRa9NfHzu10XHLV86FBGf
> =hafd
> -END PGP SIGNATURE-
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: tomcat 7/modcluster ssl issue

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Qadeer,

On 2/14/16 1:50 PM, Qadeer Khan wrote:
> 
> Hello,
> 
> I am trying to hookup ssl connection between mod_Cluster load
> balancer and tomcat 7 and seems to be following everything but when
> I start tomcat server, I am getting the following error in my
> catalina.out. My password is for the keystore is "tomcat". I have
> checked and rechecked. Any thoughts???
> 
> java.lang.IllegalStateException: java.io.IOException: Keystore was
> tampered with, or password was incorrect at
> org.jboss.modcluster.mcmp.impl.JSSESocketFactory.(JSSESocketFact
ory.java:113)
>
> 
at
org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$Proxy.(DefaultMC
MPHandler.java:790)
> at
> org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.add(DefaultMCMPHandl
er.java:196)
>
> 
at
org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.init(DefaultMCMPHandle
r.java:143)
> at
> org.jboss.modcluster.ModClusterService.init(ModClusterService.java:142
)
>
> 
at
org.jboss.modcluster.container.catalina.CatalinaEventHandlerAdapter.init
(CatalinaEventHandlerAdapter.java:272)
> at
> org.jboss.modcluster.container.catalina.CatalinaEventHandlerAdapter.li
fecycleEvent(CatalinaEventHandlerAdapter.java:210)
>
> 
at
org.jboss.modcluster.container.catalina.standalone.ModClusterListener.li
fecycleEvent(ModClusterListener.java:136)
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(Lifecycle
Support.java:117)
>
> 
at
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.
java:90)
> at
> org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.
java:402)
>
> 
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:110)
> at org.apache.catalina.startup.Catalina.load(Catalina.java:576) at
> org.apache.catalina.startup.Catalina.load(Catalina.java:599) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
ava:62)
>
> 
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor
Impl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497) at
> org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310) at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
> 
> 
> Here is relevant info from my server.xml
> 
>  className="org.jboss.modcluster.container.catalina.standalone.ModClust
erListener"
> stickySession="true" stickySessionForce="false"
> stickySessionRemove="true" advertise="false"
> proxyList="localhost:" ssl="true" sslKeyStorePass="tomcat" 
> sslKeyStore="keystore/keystore.jks" sslKeyStoreType="JKS" 
> sslProtocol="TLS"
> 
>  protocol="org.apache.coyote.http11.Http11NioProtocol"
> maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
> clientAuth="true" keystoreFile="keystore/keystore.jks" 
> keystorePass="tomcat" keystoreType="JKS" sslProtocol="TLS" />

This looks like it might be a JBoss issue, since everything involved
is provided by JBoss.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbDl5cACgkQ9CaO5/Lv0PBGsQCfd7lU3g6rr4XvIFDM/EI4z2CM
taUAn1GUWU4hREoeScVgJyMvbJZ7Cy8q
=cYRw
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ty,

On 2/16/16 3:11 PM, Ty wrote:
> George, thanks for sharing your experience, good to know I'm not
> the first down this path.  I'd be interested to know if you've
> tested at all with Tomcat 8 and what your findings were, if so.
> Based on my experience I'd expect you to be in a similar boat to
> us.
> 
> Mark, thanks for the reply.  I've uploaded my "dummy" webapp here:
> 
> http://filebin.ca/2XCPi8oAdy84/test.war
> 
> As for the JAR scanning / startup times, with Tomcat 8's jarsToScan
> setting I'm confident we'd be able to manage that, and am happy to
> leapfrog over Tomcat 7. It's the memory utilization in Tomcat 8
> specifically, independent of JAR scanning, that's the show-stopper
> for us.  Let me know if I can provide any other info, and thanks
> again.

I'm not sure if others will agree, but if Tomcat had the option to
ignore META-INF/resources in JAR files, you could probably save some
memory. Since you are currently using Tomcat 6, this setting wouldn't
affect you at all, since Tomcat 6 didn't expose those files in the way
Mark describes. On the other hand, since it's a spec-requirement,
there may be some pushback amongst the team here as to whether or not
we should implement such a workaround.

We have lots of other settings that can technically be used to violate
spec-defined behavior, so it's not like there isn't precedent for such
things.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbDlzAACgkQ9CaO5/Lv0PBWDgCfV3efH9stmGMVPFtch9unNeJl
Dq4AoJwlx4NbRa9NfHzu10XHLV86FBGf
=hafd
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[George Sexton]

On 2/16/2016 1:11 PM, Ty wrote:

George, thanks for sharing your experience, good to know I'm not the first
down this path.  I'd be interested to know if you've tested at all with
Tomcat 8 and what your findings were, if so.  Based on my experience I'd
expect you to be in a similar boat to us.


I was doing some work for someone, and I thought about upgrading to 
Tomcat 8, but it doesn't offer anything I need. I'm just going to wait 
for Tomcat 9, which will have something compelling for me (SNI support).





Mark, thanks for the reply.  I've uploaded my "dummy" webapp here:

http://filebin.ca/2XCPi8oAdy84/test.war

As for the JAR scanning / startup times, with Tomcat 8's jarsToScan setting
I'm confident we'd be able to manage that, and am happy to leapfrog over
Tomcat 7. It's the memory utilization in Tomcat 8 specifically, independent
of JAR scanning, that's the show-stopper for us.  Let me know if I can
provide any other info, and thanks again.



On Tue, Feb 16, 2016 at 12:58 PM Mark Thomas  wrote:


On 16/02/2016 16:13, Ty wrote:

Summary:

Because of our use case (hundreds of webapps per instance), we cannot
feasibly upgrade our Tomcat 6 containers to Tomcat 7 or Tomcat 8, due to
the massive increase in memory the upgrade causes.

Background:

We’ve been running Tomcat 6 for several years, on perhaps an unusual

scale:

several dozen Tomcat instances, each running up to a *few hundred

webapps*

in a single container.  (Picture a cloud service provider with several
products and hundreds of customers, with one dedicated webapp
per-customer-per-product, resulting in many thousands of web
applications.  It’s
not ideal but it’s how the software was designed).  Typical max heap

sizes

per container are 1GB to 4GB depending on utilization, and max container
startup times are in the neighborhood of 5 to 10 minutes.

· When Tomcat 7 was released, our testing showed a 10-to-20-fold
increase in memory consumption and a 3-to-5-fold increase in startup time
for our test case (100 deployed webapps).  After some digging we

determined

that these increases were related to JAR scanning
(tomcat.util.scan.DefaultJarScanner).

JAR scanning should be transient. What was it that was causing the 10-20
fold increase?


· When Tomcat 8 was released, we noted that it included a
“whitelist” version of the jarsToSkip Property, jarsToScan.  Our hopes

were

high that we could skip from Tomcat 6 to Tomcat 8, but our test case

(again

100 deployed webapps) quickly deflated those hopes:  similar startup

times

but at least a *30-fold increase in memory consumption*, regardless of

how

we configure jarsToScan or jarsToSkip.

· A heap dump analysis showed that the Tomcat 8 memory increase

was

largely due to the size/count of
org.apache.catalina.webresources.JarResourceSet objects.

Those are only going to be created when a JAR is found that contains
META-INF/resources. The spec requires Tomcat to expose those as static
resources and that requires plumbing. Note Tomcat 6 will just ignore
these because the spec is to old for this feature.


We attempted to
reduce this by setting the Context/Resources@cachingAllowed atttribute

to

“false” and also tried to tune the “cacheMaxSize” and

“cacheObjectMaxSize”

attributes.

Those attributes should have no impact on the number or size of the
JarResourceSet instances.


The only effect that came from these changes was a change in
the object that took up all the space:  instead of
org.apache.catalina.webresources.JarResourceSet objects, they are
org.apache.catalina.webresources.StandardResourceSet objects, taking up
roughly the same amount of space.

Strange. Very strange.


Question:

Is there anything else we can adjust to make Tomcat 8’s memory

consumption

closer to that of Tomcat 6’s, for our use case?

It is hard to see exactly what is going on here. The symptoms you
describe and the resulting changes with configuration are not consistent
with how I would expect Tomcat to behave nor can I see how it might
behave in the way described.


If not, we are faced with
either running Tomcat 6 past its EOL, or possibly maintaining a very

large

jarsToSkip blacklist in Tomcat 7 (until its EOL).  It is not economically
feasible for us to increase the physical memory of our servers by 30x so

we

can run Tomcat 8.

Test procedure:


- Create a "dummy" webapp (using default Maven archetype) named

test.war

- Add several popular Maven dependencies such that the total size of

the

JARs in WEB-INF/lib is about 30MB

Can you provide the sample project? It makes sense to ensure we are
working from the same baseline. I wonder if the libraries themselves are
a factor here.


- Download the latest versions of Tomcat 6, 7, and 8.
- Set Java environment variables for a JMX listener and a 3GB max heap
- Make 100 copies of test.war (test1.war, test2.war, …, test100.war)

and

drop them in /webapps.
- Start the container, 

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Ty]

George, thanks for sharing your experience, good to know I'm not the first
down this path.  I'd be interested to know if you've tested at all with
Tomcat 8 and what your findings were, if so.  Based on my experience I'd
expect you to be in a similar boat to us.

Mark, thanks for the reply.  I've uploaded my "dummy" webapp here:

http://filebin.ca/2XCPi8oAdy84/test.war

As for the JAR scanning / startup times, with Tomcat 8's jarsToScan setting
I'm confident we'd be able to manage that, and am happy to leapfrog over
Tomcat 7. It's the memory utilization in Tomcat 8 specifically, independent
of JAR scanning, that's the show-stopper for us.  Let me know if I can
provide any other info, and thanks again.



On Tue, Feb 16, 2016 at 12:58 PM Mark Thomas  wrote:

> On 16/02/2016 16:13, Ty wrote:
> > Summary:
> >
> > Because of our use case (hundreds of webapps per instance), we cannot
> > feasibly upgrade our Tomcat 6 containers to Tomcat 7 or Tomcat 8, due to
> > the massive increase in memory the upgrade causes.
> >
> > Background:
> >
> > We’ve been running Tomcat 6 for several years, on perhaps an unusual
> scale:
> > several dozen Tomcat instances, each running up to a *few hundred
> webapps*
> > in a single container.  (Picture a cloud service provider with several
> > products and hundreds of customers, with one dedicated webapp
> > per-customer-per-product, resulting in many thousands of web
> > applications.  It’s
> > not ideal but it’s how the software was designed).  Typical max heap
> sizes
> > per container are 1GB to 4GB depending on utilization, and max container
> > startup times are in the neighborhood of 5 to 10 minutes.
> >
> > · When Tomcat 7 was released, our testing showed a 10-to-20-fold
> > increase in memory consumption and a 3-to-5-fold increase in startup time
> > for our test case (100 deployed webapps).  After some digging we
> determined
> > that these increases were related to JAR scanning
> > (tomcat.util.scan.DefaultJarScanner).
>
> JAR scanning should be transient. What was it that was causing the 10-20
> fold increase?
>
> > · When Tomcat 8 was released, we noted that it included a
> > “whitelist” version of the jarsToSkip Property, jarsToScan.  Our hopes
> were
> > high that we could skip from Tomcat 6 to Tomcat 8, but our test case
> (again
> > 100 deployed webapps) quickly deflated those hopes:  similar startup
> times
> > but at least a *30-fold increase in memory consumption*, regardless of
> how
> > we configure jarsToScan or jarsToSkip.
> >
> > · A heap dump analysis showed that the Tomcat 8 memory increase
> was
> > largely due to the size/count of
> > org.apache.catalina.webresources.JarResourceSet objects.
>
> Those are only going to be created when a JAR is found that contains
> META-INF/resources. The spec requires Tomcat to expose those as static
> resources and that requires plumbing. Note Tomcat 6 will just ignore
> these because the spec is to old for this feature.
>
> > We attempted to
> > reduce this by setting the Context/Resources@cachingAllowed atttribute
> to
> > “false” and also tried to tune the “cacheMaxSize” and
> “cacheObjectMaxSize”
> > attributes.
>
> Those attributes should have no impact on the number or size of the
> JarResourceSet instances.
>
> > The only effect that came from these changes was a change in
> > the object that took up all the space:  instead of
> > org.apache.catalina.webresources.JarResourceSet objects, they are
> > org.apache.catalina.webresources.StandardResourceSet objects, taking up
> > roughly the same amount of space.
>
> Strange. Very strange.
>
> > Question:
> >
> > Is there anything else we can adjust to make Tomcat 8’s memory
> consumption
> > closer to that of Tomcat 6’s, for our use case?
>
> It is hard to see exactly what is going on here. The symptoms you
> describe and the resulting changes with configuration are not consistent
> with how I would expect Tomcat to behave nor can I see how it might
> behave in the way described.
>
> > If not, we are faced with
> > either running Tomcat 6 past its EOL, or possibly maintaining a very
> large
> > jarsToSkip blacklist in Tomcat 7 (until its EOL).  It is not economically
> > feasible for us to increase the physical memory of our servers by 30x so
> we
> > can run Tomcat 8.
> >
> > Test procedure:
> >
> >
> >- Create a "dummy" webapp (using default Maven archetype) named
> test.war
> >- Add several popular Maven dependencies such that the total size of
> the
> >JARs in WEB-INF/lib is about 30MB
>
> Can you provide the sample project? It makes sense to ensure we are
> working from the same baseline. I wonder if the libraries themselves are
> a factor here.
>
> >- Download the latest versions of Tomcat 6, 7, and 8.
> >- Set Java environment variables for a JMX listener and a 3GB max heap
> >- Make 100 copies of test.war (test1.war, test2.war, …, test100.war)
> and
> >drop them in /webapps.
> >- Start 

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Ty]

> JAR scanning should be transient. What was it that was causing the 10-20
> fold increase?

>From memory (this was a while ago), it was related to
java.util.jar.Manifest objects, or what they contained/referenced.  I'd be
glad to do another heap analysis and summarize it if it would help, but
even more glad to just skip Tomcat 7 altogether.

On Tue, Feb 16, 2016 at 2:11 PM Ty  wrote:

> George, thanks for sharing your experience, good to know I'm not the first
> down this path.  I'd be interested to know if you've tested at all with
> Tomcat 8 and what your findings were, if so.  Based on my experience I'd
> expect you to be in a similar boat to us.
>
> Mark, thanks for the reply.  I've uploaded my "dummy" webapp here:
>
> http://filebin.ca/2XCPi8oAdy84/test.war
>
> As for the JAR scanning / startup times, with Tomcat 8's jarsToScan
> setting I'm confident we'd be able to manage that, and am happy to leapfrog
> over Tomcat 7. It's the memory utilization in Tomcat 8 specifically,
> independent of JAR scanning, that's the show-stopper for us.  Let me know
> if I can provide any other info, and thanks again.
>
>
>
> On Tue, Feb 16, 2016 at 12:58 PM Mark Thomas  wrote:
>
>> On 16/02/2016 16:13, Ty wrote:
>> > Summary:
>> >
>> > Because of our use case (hundreds of webapps per instance), we cannot
>> > feasibly upgrade our Tomcat 6 containers to Tomcat 7 or Tomcat 8, due to
>> > the massive increase in memory the upgrade causes.
>> >
>> > Background:
>> >
>> > We’ve been running Tomcat 6 for several years, on perhaps an unusual
>> scale:
>> > several dozen Tomcat instances, each running up to a *few hundred
>> webapps*
>> > in a single container.  (Picture a cloud service provider with several
>> > products and hundreds of customers, with one dedicated webapp
>> > per-customer-per-product, resulting in many thousands of web
>> > applications.  It’s
>> > not ideal but it’s how the software was designed).  Typical max heap
>> sizes
>> > per container are 1GB to 4GB depending on utilization, and max container
>> > startup times are in the neighborhood of 5 to 10 minutes.
>> >
>> > · When Tomcat 7 was released, our testing showed a 10-to-20-fold
>> > increase in memory consumption and a 3-to-5-fold increase in startup
>> time
>> > for our test case (100 deployed webapps).  After some digging we
>> determined
>> > that these increases were related to JAR scanning
>> > (tomcat.util.scan.DefaultJarScanner).
>>
>> JAR scanning should be transient. What was it that was causing the 10-20
>> fold increase?
>>
>> > · When Tomcat 8 was released, we noted that it included a
>> > “whitelist” version of the jarsToSkip Property, jarsToScan.  Our hopes
>> were
>> > high that we could skip from Tomcat 6 to Tomcat 8, but our test case
>> (again
>> > 100 deployed webapps) quickly deflated those hopes:  similar startup
>> times
>> > but at least a *30-fold increase in memory consumption*, regardless of
>> how
>> > we configure jarsToScan or jarsToSkip.
>> >
>> > · A heap dump analysis showed that the Tomcat 8 memory increase
>> was
>> > largely due to the size/count of
>> > org.apache.catalina.webresources.JarResourceSet objects.
>>
>> Those are only going to be created when a JAR is found that contains
>> META-INF/resources. The spec requires Tomcat to expose those as static
>> resources and that requires plumbing. Note Tomcat 6 will just ignore
>> these because the spec is to old for this feature.
>>
>> > We attempted to
>> > reduce this by setting the Context/Resources@cachingAllowed atttribute
>> to
>> > “false” and also tried to tune the “cacheMaxSize” and
>> “cacheObjectMaxSize”
>> > attributes.
>>
>> Those attributes should have no impact on the number or size of the
>> JarResourceSet instances.
>>
>> > The only effect that came from these changes was a change in
>> > the object that took up all the space:  instead of
>> > org.apache.catalina.webresources.JarResourceSet objects, they are
>> > org.apache.catalina.webresources.StandardResourceSet objects, taking up
>> > roughly the same amount of space.
>>
>> Strange. Very strange.
>>
>> > Question:
>> >
>> > Is there anything else we can adjust to make Tomcat 8’s memory
>> consumption
>> > closer to that of Tomcat 6’s, for our use case?
>>
>> It is hard to see exactly what is going on here. The symptoms you
>> describe and the resulting changes with configuration are not consistent
>> with how I would expect Tomcat to behave nor can I see how it might
>> behave in the way described.
>>
>> > If not, we are faced with
>> > either running Tomcat 6 past its EOL, or possibly maintaining a very
>> large
>> > jarsToSkip blacklist in Tomcat 7 (until its EOL).  It is not
>> economically
>> > feasible for us to increase the physical memory of our servers by 30x
>> so we
>> > can run Tomcat 8.
>> >
>> > Test procedure:
>> >
>> >
>> >- Create a "dummy" webapp (using default Maven archetype) named
>> test.war
>> >- Add 

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Gregory,

On 2/16/16 11:50 AM, Dougherty, Gregory T., M.S. wrote:
> On 2/16/16, 9:13 AM, "Christopher Schultz" 
>  wrote:
> 
> 
>> Gregory,
>> 
>> On 2/15/16 11:18 AM, Dougherty, Gregory T., M.S. wrote:
>>> How to trade information with a specified URL calling java
>>> code in tomcat/lib:
>>> 
>>> 1: Servlet calls Decryptor.start (String whoIAm, int
>>> validator, int xOr)
>> 
>> What prevents the servlet from specifying whoIAm="someoneElse"?
> 
> Absolutely nothing.  That’s why the decrypt code does an HTTP get 
> against whoIAm, to validate that claim
> 
>>> 2: Code calls Servlet http get with id=randomly generated int
>> 
>> Which code calls Servlet HTTP?
> 
> HttpURLConnection, called from within the decrypt code
> 
> 
>> 
>>> 3: Servlet returns validator in response to the http get call. 
>>> 0 is not a valid validator, it¹s what the Servlet returns if
>>> it doesn¹t have an outstanding call to Decryptor.start
>> 
>> But the whole point is that you don't trust the servlet, right? 
>> What is the servlet always returns the same value? Any servlet 
>> could impersonate any other one.
> 
> I trust each servlet to act in its own best interest.
> 
> An honest servlet will 1: Pass in real random numbers for
> validator and xOr, 2: React correctly to this http get call. An
> incompetently written or dishonest servlet can only give away its
> own passwords by doing the wrong thing.  That’s their problem, not
> mine.
> 
>>> 4: Having received the random number from the http get call, 
>>> Servlet xors it with xOr, and calls Decryptor.decrypt (String 
>>> whoIAm, String password, int xOred)
>> 
>> So, the servlet makes the HTTP request and manages the checking? 
>> I really don't get it.
> 
> Servlet (calles Decrypt.start): Sends whoIAm, validator, xOr Code 
> (calls http get on whoIAm): Sends cValid Servlet (response to HTTP 
> get): Sends validator (same as was send in call to Decrypt.start). 
> This validates whoIAm Servlet (calls Decrypt.decode): Sends
> whoIAm, password (encrypted), xOred (= xOr ^ cValid) Code (return
> value from Decrypt.decode): password (decrypted)

No, I followed all that. I still just don't "get it".

Your initial question was how to verify that the calling code was a
specific application. Your proposed solution allows the calling code
to *authenticate itself*. Thus, no security is being added.
Complexity, yes. Security, no.

Then we went off on the tangent about how administrators shouldn't
have access to the passwords. If the admins could have the passwords,
there wouldn't be any problems, here (I think).

So you're asking for a key-escrow system where just about every
component involved can have access to the password except for the
people who don't need the passwords, because they are administrators.

I suppose you could have separate admins for app servers versus DBs,
but anything the code running on a machine can do, an administrator of
that machine can do, too.

>>> 5: Code checks xOred against whoIAm.  If gets a match decrypts 
>>> the password and returns it from Decryptor.decrypt.
>> 
>> After all that, it's okay for the servlet to be able to read its 
>> own database password? Why not just give the servlet access to a 
>> DataSource with the password already set inside it?
> 
> Because it’s the Servlet’s password?  How many times do I have to 
> save that before you’ll believe it?
> 
> The data belongs to the servlet.  The servlet is working with the 
> decryption code so that the data can be safely stored with the 
> servlet.

If the data (and password protecting it) really belongs to the
servlet, why are you making it so hard for the servlet to get the data?

> Every servlet trusts itself.  None of them should be required to 
> trust any other servlet running on that instance of Tomcat.  None 
> of them should be required to trust that data saved in the source 
> control system won’t be abused, if it can be abused.
> 
>>> But so long as I can make the following two calls from my 
>>> code:
>>> 
>>> URL theURL = new URL (urlString); HttpURLConnection uc = 
>>> (HttpURLConnection) theURL.openConnection ();
>>> 
>>> I think I¹m good.
>>> 
>>> Flaws?
>> 
>> Let's see:
>> 
>> Constructed dubious use case? CHECK
> Wrong.  This use case is my current work situation.

Just because it's the problem you are facing doesn't make it your
use-case. You have decided that this is your use-case based upon the
requirements you are trying to fulfill.

>> Rolled your own security code? CHECK Used highly-secure XOR 
>> algorithm? CHECK Complicated enough to seem plausible? CHECK 
>> Confused objective leads to trivial attacks? CHECK
>> 
>> I think you've got all the bases covered, here.
>> 
>> I still don't understand what's being protected from whom, here. 
>> It looks like the code is very complicated in order to keep the 
>> password from the code, but then the code is given the password 
>> anyway. If you trust 

Re: ***UNCHECKED*** Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

All,

Sorry about this. It seems Enigmail is ruining messages.

- -chris

On 2/16/16 2:09 PM, Christopher Schultz wrote:
> 
> The information contained in this email and its attachments may be
> confidential. If you have received this email in error, please
> notify the sender by return email, delete this email and destroy
> any copy.
> 
> Any advice contained in this email has been prepared without taking
> into account your objectives, financial situation or needs. Before
> acting on any advice in this email, National Australia Bank Limited
> (NAB) recommends that you consider whether it is appropriate for
> your circumstances. If this email contains reference to any
> financial products, NAB recommends you consider the Product
> Disclosure Statement (PDS) or other disclosure document available
> from NAB, before making any decisions regarding any products.
> 
> If this email contains any promotional content that you do not wish
> to receive, please reply to the original sender and write "Don't
> email promotional material" in the subject.
> 
> 
> 
> 
> -
>
> 
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbDeS4ACgkQ9CaO5/Lv0PAT0gCeJphTjt9Zmq1ByxC1ljY2Deul
+HQAoJe7Y681TgsFr0adsrz3xXTI8J6X
=ipRC
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

***UNCHECKED*** Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

The information contained in this email and its attachments may be confidential.
If you have received this email in error, please notify the sender by return 
email,
delete this email and destroy any copy.

Any advice contained in this email has been prepared without taking into
account your objectives, financial situation or needs. Before acting on any
advice in this email, National Australia Bank Limited (NAB) recommends that
you consider whether it is appropriate for your circumstances.
If this email contains reference to any financial products, NAB recommends
you consider the Product Disclosure Statement (PDS) or other disclosure
document available from NAB, before making any decisions regarding any
products.

If this email contains any promotional content that you do not wish to receive,
please reply to the original sender and write "Don't email promotional
material" in the subject.

--- Begin Message ---


PGPMIME version identification
Description: PGPMIME version identification
--- End Message ---

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

***UNCHECKED*** Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

The information contained in this email and its attachments may be confidential.
If you have received this email in error, please notify the sender by return 
email,
delete this email and destroy any copy.

Any advice contained in this email has been prepared without taking into
account your objectives, financial situation or needs. Before acting on any
advice in this email, National Australia Bank Limited (NAB) recommends that
you consider whether it is appropriate for your circumstances.
If this email contains reference to any financial products, NAB recommends
you consider the Product Disclosure Statement (PDS) or other disclosure
document available from NAB, before making any decisions regarding any
products.

If this email contains any promotional content that you do not wish to receive,
please reply to the original sender and write "Don't email promotional
material" in the subject.

--- Begin Message ---


PGPMIME version identification
Description: PGPMIME version identification
--- End Message ---

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

***UNCHECKED*** Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

binrvxcJ0gxtC.bin
Description: PGP/MIME version identification

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[Mark Thomas]

On 16/02/2016 16:13, Ty wrote:
> Summary:
> 
> Because of our use case (hundreds of webapps per instance), we cannot
> feasibly upgrade our Tomcat 6 containers to Tomcat 7 or Tomcat 8, due to
> the massive increase in memory the upgrade causes.
> 
> Background:
> 
> We’ve been running Tomcat 6 for several years, on perhaps an unusual scale:
> several dozen Tomcat instances, each running up to a *few hundred webapps*
> in a single container.  (Picture a cloud service provider with several
> products and hundreds of customers, with one dedicated webapp
> per-customer-per-product, resulting in many thousands of web
> applications.  It’s
> not ideal but it’s how the software was designed).  Typical max heap sizes
> per container are 1GB to 4GB depending on utilization, and max container
> startup times are in the neighborhood of 5 to 10 minutes.
> 
> · When Tomcat 7 was released, our testing showed a 10-to-20-fold
> increase in memory consumption and a 3-to-5-fold increase in startup time
> for our test case (100 deployed webapps).  After some digging we determined
> that these increases were related to JAR scanning
> (tomcat.util.scan.DefaultJarScanner).

JAR scanning should be transient. What was it that was causing the 10-20
fold increase?

> · When Tomcat 8 was released, we noted that it included a
> “whitelist” version of the jarsToSkip Property, jarsToScan.  Our hopes were
> high that we could skip from Tomcat 6 to Tomcat 8, but our test case (again
> 100 deployed webapps) quickly deflated those hopes:  similar startup times
> but at least a *30-fold increase in memory consumption*, regardless of how
> we configure jarsToScan or jarsToSkip.
> 
> · A heap dump analysis showed that the Tomcat 8 memory increase was
> largely due to the size/count of
> org.apache.catalina.webresources.JarResourceSet objects.

Those are only going to be created when a JAR is found that contains
META-INF/resources. The spec requires Tomcat to expose those as static
resources and that requires plumbing. Note Tomcat 6 will just ignore
these because the spec is to old for this feature.

> We attempted to
> reduce this by setting the Context/Resources@cachingAllowed atttribute to
> “false” and also tried to tune the “cacheMaxSize” and “cacheObjectMaxSize”
> attributes.

Those attributes should have no impact on the number or size of the
JarResourceSet instances.

> The only effect that came from these changes was a change in
> the object that took up all the space:  instead of
> org.apache.catalina.webresources.JarResourceSet objects, they are
> org.apache.catalina.webresources.StandardResourceSet objects, taking up
> roughly the same amount of space.

Strange. Very strange.

> Question:
> 
> Is there anything else we can adjust to make Tomcat 8’s memory consumption
> closer to that of Tomcat 6’s, for our use case?

It is hard to see exactly what is going on here. The symptoms you
describe and the resulting changes with configuration are not consistent
with how I would expect Tomcat to behave nor can I see how it might
behave in the way described.

> If not, we are faced with
> either running Tomcat 6 past its EOL, or possibly maintaining a very large
> jarsToSkip blacklist in Tomcat 7 (until its EOL).  It is not economically
> feasible for us to increase the physical memory of our servers by 30x so we
> can run Tomcat 8.
> 
> Test procedure:
> 
> 
>- Create a "dummy" webapp (using default Maven archetype) named test.war
>- Add several popular Maven dependencies such that the total size of the
>JARs in WEB-INF/lib is about 30MB

Can you provide the sample project? It makes sense to ensure we are
working from the same baseline. I wonder if the libraries themselves are
a factor here.

>- Download the latest versions of Tomcat 6, 7, and 8.
>- Set Java environment variables for a JMX listener and a 3GB max heap
>- Make 100 copies of test.war (test1.war, test2.war, …, test100.war) and
>drop them in /webapps.
>- Start the container, note the reported startup time, perform an
>explicit major GC, and note the heap utilization.
>- Run each test several times and average the results

That is a nice simple test we should be able to repeat.

> Test results:
> 
> Case 1 (baseline):
> 
> 
> 
> +-+--+---+
> | version | startup time | heap usage after major GC |
> +-+--+---+
> | tomcat6 | 36,711ms | 21,163,288|
> | tomcat7 | 104,517ms| 489,992,264   |
> | tomcat8 | 156,094ms| 1,010,512,568 |
> +-+--+---+
> 
> 
> Case 2 (Tomcat 7 and 8 with “jarsToSkip=*”)
> 
> +-+--+---+
> | version | startup time | heap usage after major GC |
> +-+--+---+
> | tomcat6 | 36,711ms | 21,163,288|
> | tomcat7 | 

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Leo Donahue]

On Feb 16, 2016 10:50 AM, "Dougherty, Gregory T., M.S." <
dougherty.greg...@mayo.edu> wrote:
>
> On 2/16/16, 9:13 AM, "Christopher Schultz" 
> wrote:
>
>
> >Gregory,
> >
> >On 2/15/16 11:18 AM, Dougherty, Gregory T., M.S. wrote:
> >> How to trade information with a specified URL calling java code in
> >> tomcat/lib:
> >>
> >> 1: Servlet calls Decryptor.start (String whoIAm, int validator, int
xOr)
> >
> >What prevents the servlet from specifying whoIAm="someoneElse"?
>
> Absolutely nothing.  That’s why the decrypt code does an HTTP get against
> whoIAm, to validate that claim
>
> >> 2: Code calls Servlet http get with id=randomly generated int
> >
> >Which code calls Servlet HTTP?
>
> HttpURLConnection, called from within the decrypt code
>
>
> >
> >> 3: Servlet returns validator in response to the http get call.  0 is
> >>not a
> >> valid validator, it¹s what the Servlet returns if it doesn¹t have an
> >> outstanding call to Decryptor.start
> >
> >But the whole point is that you don't trust the servlet, right? What is
> >the servlet always returns the same value? Any servlet could impersonate
> >any other one.
>
> I trust each servlet to act in its own best interest.
>
> An honest servlet will 1: Pass in real random numbers for validator and
> xOr, 2: React correctly to this http get call.
> An incompetently written or dishonest servlet can only give away its own
> passwords by doing the wrong thing.

Give them away?  Are you logging the passwords used?

That’s their problem, not mine.
>
> >> 4: Having received the random number from the http get call, Servlet
> >>xors
> >> it with xOr, and calls Decryptor.decrypt (String whoIAm, String
> >>password,
> >> int xOred)
> >
> >So, the servlet makes the HTTP request and manages the checking? I
> >really don't get it.
>
> Servlet (calles Decrypt.start): Sends whoIAm, validator, xOr
> Code (calls http get on whoIAm): Sends cValid
> Servlet (response to HTTP get): Sends validator (same as was send in call
> to Decrypt.start). This validates whoIAm
> Servlet (calls Decrypt.decode): Sends whoIAm, password (encrypted), xOred
> (= xOr ^ cValid)
> Code (return value from Decrypt.decode): password (decrypted)
>
> >
> >> 5: Code checks xOred against whoIAm.  If gets a match decrypts the
> >> password and returns it from Decryptor.decrypt.
> >
> >After all that, it's okay for the servlet to be able to read its own
> >database password? Why not just give the servlet access to a DataSource
> >with the password already set inside it?
>
> Because it’s the Servlet’s password?  How many times do I have to save
> that before you’ll believe it?
>
> The data belongs to the servlet.  The servlet is working with the
> decryption code so that the data can be safely stored with the servlet.
>
> Every servlet trusts itself.  None of them should be required to trust any
> other servlet running on that instance of Tomcat.  None of them should be
> required to trust that data saved in the source control system won’t be
> abused, if it can be abused.
>
> >>But so long as I can make the following two calls from my code:
> >>
> >> URL theURL = new URL (urlString);
> >>  HttpURLConnection   uc = (HttpURLConnection) theURL.openConnection ();
> >>
> >> I think I¹m good.
> >>
> >> Flaws?
> >
> >Let's see:
> >
> >  Constructed dubious use case? CHECK
> Wrong.  This use case is my current work situation.
>
> >  Rolled your own security code? CHECK
> >  Used highly-secure XOR algorithm? CHECK
> >  Complicated enough to seem plausible? CHECK
> >  Confused objective leads to trivial attacks? CHECK
> >
> >I think you've got all the bases covered, here.
> >
> >I still don't understand what's being protected from whom, here. It
> >looks like the code is very complicated in order to keep the password
> >from the code, but then the code is given the password anyway. If you
> >trust the code but not the admin, then why bother with the complex code?
>
> The problem, Chris, is that you don’t understand the situation, and rather
> than try to understand the situation you’re spending all your time and
> effort sniping at things you don’t understand.  Let’s try one more time:
>
>

If you led with the information below in your original post, it would have
helped.

> My servlet is running on a Tomcat instance that is hosting 5 other
Servlets
>
> I completely and totally trust my servlet with my data.  I do not in the
> least bit trust any  other servlet running on that Tomcat instance
>
> I am willing to trust a Jar file that I’ve written / reviewed, and the
> administrators have added to tomcat/lib
>
> I have data that my servlet needs to use.  I am not going to try to go to
> a page on my servlet and personally type in that data every time it runs.
> Therefore I’m going to have to save that data.  I’m not going to save that
> data somewhere on the server hosting the Tomcat instance, because I just
> don’t’ trust the administrators that much.
>
> This means I have to save the data 

Re: Cannot upgrade past Tomcat 6 due to massive memory increase

[George Sexton]

On 2/16/2016 9:13 AM, Ty wrote:

· When Tomcat 7 was released, our testing showed a 10-to-20-fold
increase in memory consumption and a 3-to-5-fold increase in startup time
for our test case (100 deployed webapps).  After some digging we determined
that these increases were related to JAR scanning
(tomcat.util.scan.DefaultJarScanner).  When setting jarsToSkip=*, the
memory consumption was decreased to something somewhat more reasonable
(2-3x increase over Tomcat 6) and the startup time went back to what we
were used to with Tomcat 6.  However at that time we opted to stay on
Tomcat 6, because our apps were not functional with jarsToSkip=*, and we
deemed it impractical to manage a very large “blacklist” in the jarsToSkip
Property.




I'll just throw in my experiences. I'm running around 800+ copies of a 
web app on tomcat 7.


First, to get Tomcat 7 to start in under a century, I had to do 
jarsToSkip. It hasn't been a big problem to maintain since. Fortunately 
for me as well, my app doesn't use annotations in any significant way so 
I'm pretty lucky there.


Second, I moved all of the generic jars into ${catalina.base}/lib. My 
web app's jar uses statics, so each webapp has perhaps a single 1.5MB 
jar in the context WEB-INF/lib. This is bad because now I pretty much 
have to upgrade all instances in lock-step otherwise I run into version 
dependencies with ${catalina.base}/lib files. Still, it's an acceptable 
tradeoff. If this were possible for you, I'd definitely give this a go.


My startup times are around 30 seconds. A huge part of my remaining 
startup times are caused by loading each instance's unique database. 
When I put the databases on SSD, things were lots better :)


Right now, I have 48GB allocated to the JVM. Really, it's using 16GB of 
eden space, and it's got around 32GB dedicated to Old Gen, but it's 
really only using 5GB of space in Old Gen. Survivor space is under 
256MB. I suppose I could do some tuning with various parameters to force 
more allocation to eden space, and less to old gen, but performance is 
really good, and I don't have any motivation to do it.



--
George Sexton
*MH Software, Inc.*
Voice: 303 438 9585
http://www.mhsoftware.com

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Dougherty, Gregory T., M.S.]

On 2/16/16, 9:13 AM, "Christopher Schultz" 
wrote:


>Gregory,
>
>On 2/15/16 11:18 AM, Dougherty, Gregory T., M.S. wrote:
>> How to trade information with a specified URL calling java code in
>> tomcat/lib:
>> 
>> 1: Servlet calls Decryptor.start (String whoIAm, int validator, int xOr)
>
>What prevents the servlet from specifying whoIAm="someoneElse"?

Absolutely nothing.  That’s why the decrypt code does an HTTP get against
whoIAm, to validate that claim

>> 2: Code calls Servlet http get with id=randomly generated int
>
>Which code calls Servlet HTTP?

HttpURLConnection, called from within the decrypt code


>
>> 3: Servlet returns validator in response to the http get call.  0 is
>>not a
>> valid validator, it¹s what the Servlet returns if it doesn¹t have an
>> outstanding call to Decryptor.start
>
>But the whole point is that you don't trust the servlet, right? What is
>the servlet always returns the same value? Any servlet could impersonate
>any other one.

I trust each servlet to act in its own best interest.

An honest servlet will 1: Pass in real random numbers for validator and
xOr, 2: React correctly to this http get call.
An incompetently written or dishonest servlet can only give away its own
passwords by doing the wrong thing.  That’s their problem, not mine.

>> 4: Having received the random number from the http get call, Servlet
>>xors
>> it with xOr, and calls Decryptor.decrypt (String whoIAm, String
>>password,
>> int xOred)
>
>So, the servlet makes the HTTP request and manages the checking? I
>really don't get it.

Servlet (calles Decrypt.start): Sends whoIAm, validator, xOr
Code (calls http get on whoIAm): Sends cValid
Servlet (response to HTTP get): Sends validator (same as was send in call
to Decrypt.start). This validates whoIAm
Servlet (calls Decrypt.decode): Sends whoIAm, password (encrypted), xOred
(= xOr ^ cValid)
Code (return value from Decrypt.decode): password (decrypted)

>
>> 5: Code checks xOred against whoIAm.  If gets a match decrypts the
>> password and returns it from Decryptor.decrypt.
>
>After all that, it's okay for the servlet to be able to read its own
>database password? Why not just give the servlet access to a DataSource
>with the password already set inside it?

Because it’s the Servlet’s password?  How many times do I have to save
that before you’ll believe it?

The data belongs to the servlet.  The servlet is working with the
decryption code so that the data can be safely stored with the servlet.

Every servlet trusts itself.  None of them should be required to trust any
other servlet running on that instance of Tomcat.  None of them should be
required to trust that data saved in the source control system won’t be
abused, if it can be abused.

>>But so long as I can make the following two calls from my code:
>> 
>> URL theURL = new URL (urlString);
>>  HttpURLConnection   uc = (HttpURLConnection) theURL.openConnection ();
>> 
>> I think I¹m good.
>> 
>> Flaws?
>
>Let's see:
>
>  Constructed dubious use case? CHECK
Wrong.  This use case is my current work situation.

>  Rolled your own security code? CHECK
>  Used highly-secure XOR algorithm? CHECK
>  Complicated enough to seem plausible? CHECK
>  Confused objective leads to trivial attacks? CHECK
>
>I think you've got all the bases covered, here.
>
>I still don't understand what's being protected from whom, here. It
>looks like the code is very complicated in order to keep the password
>from the code, but then the code is given the password anyway. If you
>trust the code but not the admin, then why bother with the complex code?

The problem, Chris, is that you don’t understand the situation, and rather
than try to understand the situation you’re spending all your time and
effort sniping at things you don’t understand.  Let’s try one more time:


My servlet is running on a Tomcat instance that is hosting 5 other Servlets

I completely and totally trust my servlet with my data.  I do not in the
least bit trust any  other servlet running on that Tomcat instance

I am willing to trust a Jar file that I’ve written / reviewed, and the
administrators have added to tomcat/lib

I have data that my servlet needs to use.  I am not going to try to go to
a page on my servlet and personally type in that data every time it runs.
Therefore I’m going to have to save that data.  I’m not going to save that
data somewhere on the server hosting the Tomcat instance, because I just
don’t’ trust the administrators that much.

This means I have to save the data with my web app.  Everything saved with
my app is available to a pool of people, large enough that I can not trust
that the data is secure.


This is the current situation.  Neither you nor I get to change any of the
above parameters.  Is anything about the above unclear (other than why
this is so)?

When I say “I do not trust the Servlet calling the jar code” what I mean
is that I absolutely trust my own servlet when it’s calling the jar code,

Cannot upgrade past Tomcat 6 due to massive memory increase

[Ty]

Summary:

Because of our use case (hundreds of webapps per instance), we cannot
feasibly upgrade our Tomcat 6 containers to Tomcat 7 or Tomcat 8, due to
the massive increase in memory the upgrade causes.

Background:

We’ve been running Tomcat 6 for several years, on perhaps an unusual scale:
several dozen Tomcat instances, each running up to a *few hundred webapps*
in a single container.  (Picture a cloud service provider with several
products and hundreds of customers, with one dedicated webapp
per-customer-per-product, resulting in many thousands of web
applications.  It’s
not ideal but it’s how the software was designed).  Typical max heap sizes
per container are 1GB to 4GB depending on utilization, and max container
startup times are in the neighborhood of 5 to 10 minutes.

· When Tomcat 7 was released, our testing showed a 10-to-20-fold
increase in memory consumption and a 3-to-5-fold increase in startup time
for our test case (100 deployed webapps).  After some digging we determined
that these increases were related to JAR scanning
(tomcat.util.scan.DefaultJarScanner).  When setting jarsToSkip=*, the
memory consumption was decreased to something somewhat more reasonable
(2-3x increase over Tomcat 6) and the startup time went back to what we
were used to with Tomcat 6.  However at that time we opted to stay on
Tomcat 6, because our apps were not functional with jarsToSkip=*, and we
deemed it impractical to manage a very large “blacklist” in the jarsToSkip
Property.

· When Tomcat 8 was released, we noted that it included a
“whitelist” version of the jarsToSkip Property, jarsToScan.  Our hopes were
high that we could skip from Tomcat 6 to Tomcat 8, but our test case (again
100 deployed webapps) quickly deflated those hopes:  similar startup times
but at least a *30-fold increase in memory consumption*, regardless of how
we configure jarsToScan or jarsToSkip.

· A heap dump analysis showed that the Tomcat 8 memory increase was
largely due to the size/count of
org.apache.catalina.webresources.JarResourceSet objects.  We attempted to
reduce this by setting the Context/Resources@cachingAllowed atttribute to
“false” and also tried to tune the “cacheMaxSize” and “cacheObjectMaxSize”
attributes.  The only effect that came from these changes was a change in
the object that took up all the space:  instead of
org.apache.catalina.webresources.JarResourceSet objects, they are
org.apache.catalina.webresources.StandardResourceSet objects, taking up
roughly the same amount of space.

Question:

Is there anything else we can adjust to make Tomcat 8’s memory consumption
closer to that of Tomcat 6’s, for our use case?  If not, we are faced with
either running Tomcat 6 past its EOL, or possibly maintaining a very large
jarsToSkip blacklist in Tomcat 7 (until its EOL).  It is not economically
feasible for us to increase the physical memory of our servers by 30x so we
can run Tomcat 8.

Test procedure:


   - Create a "dummy" webapp (using default Maven archetype) named test.war
   - Add several popular Maven dependencies such that the total size of the
   JARs in WEB-INF/lib is about 30MB
   - Download the latest versions of Tomcat 6, 7, and 8.
   - Set Java environment variables for a JMX listener and a 3GB max heap
   - Make 100 copies of test.war (test1.war, test2.war, …, test100.war) and
   drop them in /webapps.
   - Start the container, note the reported startup time, perform an
   explicit major GC, and note the heap utilization.
   - Run each test several times and average the results

Test results:

Case 1 (baseline):



+-+--+---+
| version | startup time | heap usage after major GC |
+-+--+---+
| tomcat6 | 36,711ms | 21,163,288|
| tomcat7 | 104,517ms| 489,992,264   |
| tomcat8 | 156,094ms| 1,010,512,568 |
+-+--+---+


Case 2 (Tomcat 7 and 8 with “jarsToSkip=*”)

+-+--+---+
| version | startup time | heap usage after major GC |
+-+--+---+
| tomcat6 | 36,711ms | 21,163,288|
| tomcat7 | 38,979ms | 72,359,840|
| tomcat8 | 52,040ms | 633,682,336   |
+-+--+---+

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Leo Donahue]

On Feb 12, 2016 2:35 PM, "Dougherty, Gregory T., M.S." <
dougherty.greg...@mayo.edu> wrote:
>
> You are correct, I¹m trying to authorize the web app, not the user.

I am going to start all the way back here and suggest that you abandon this
approach of authenticating a web "app".

This is unconventional by all standards and you will end up with a security
hole(s).

Users authenticate to web apps, nothing further.

The web app itself can also authenticate to the database separate from any
userBUT... the web app still has a "user name".

Your problem is that someone is giving you impossible to meet security
requirements, which results in you wanting to mess with application URLs
and the other stuff you mentioned.

Secret:  even large organizations have some form of clear text passwords,
but who has access to these is strictly controlled.

>
> Goal: I am trying to come up with a way for a Tomcat app to securely store
> and retrieve the password it needs to access a DB.
>

We have given you some options to deal with this.

It seems as though the sys admins either can't or don't want to help you
establish more of an Enterprise architecture, which would solve a lot of
your issues.

> My definition of ³secure² includes ³there exist no files with an
> unencrypted copy of the password².  IIUC, JNDI fails this test.
>
> My requirements include that all web app components are checked in to a
> source control system that malicious users can have read access to.

Are you in control of the source control system?  You know, you can assign
roles to users for certain repositories and restrict access to your code
repo.

If you have developers working at Mayo who are malicious, get rid of them.

> Solution:
> 1: Trusted user creates public:private key pair (1), distributes public
key
> 2: Web app developer creates pubic:private key pair (2), distributes
> public key
> 3: Web app developer encrypted password with private key 2, then public
> key 1, stores with web app
> 4: Web app calls decryption jar that¹s in tomcat/lib, passing in the
> encrypted password from step 3
> 5: Decryption code determines which app called it, pulls the public key
> (3) saved for that app
> 6: Decryption code decrypts with private key 1, public key 3, and returns
> the unencrypted password.
>
> So long as 1: Trusted user can store private key where it¹s secure, but
> accessible to decryption code, and 2: Can correctly determine the calling
> app, I believe this setup is secure.
>
> We log who uploads the web apps, so if user X uploads a bogus ³User Y
> App², we can deal with that.
> --

Developers should never get to deploy anything to production servers.  In
fact, you should have different passwords for different deployment realms.

>From my perspective, you (or someone else is telling you to do this) are
trying to bypass as much security for the appearance of security simply
because of the Tomcat environment you have to work with.

I think we all want you to succeed here, but the approach you're leaning
towards isn't going to work.

> Gregory Dougherty
> Sr. Analyst/Programmer | Information Technology
> Information Technology
> (507) 284-8493 | dougherty.greg...@mayo.edu
>
>
>
>
>
>
>
> On 2/12/16, 2:00 PM, "Leo Donahue"  wrote:
>
> >On Feb 11, 2016 4:56 PM, "Dougherty, Gregory T., M.S." <
> >dougherty.greg...@mayo.edu> wrote:
> >>
> >> I would like to have a jar file in tomcat/lib that can be called from
> >>any
> >of the running web apps.  I need for the code in the jar to behave
> >differently depending on which web app called it.
> >
> >I would agree with what the others are saying here.  It seems you are
> >trying to authorize an entire web app instead of authoring the user of
the
> >web app.
> >
> >If the jar simply needs to take action based on a role of some kind, then
> >could you not tie in a ldap user with appropriate role?
> >
> >Leo
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>

Re: Is there a way for code running on Tomcat 7+ to determine the URL of the Web App it's running under?

[Christopher Schultz]

Gregory,

On 2/15/16 11:18 AM, Dougherty, Gregory T., M.S. wrote:
> How to trade information with a specified URL calling java code in
> tomcat/lib:
> 
> 1: Servlet calls Decryptor.start (String whoIAm, int validator, int xOr)

What prevents the servlet from specifying whoIAm="someoneElse"?

> 2: Code calls Servlet http get with id=randomly generated int

Which code calls Servlet HTTP?

> 3: Servlet returns validator in response to the http get call.  0 is not a
> valid validator, it¹s what the Servlet returns if it doesn¹t have an
> outstanding call to Decryptor.start

But the whole point is that you don't trust the servlet, right? What is
the servlet always returns the same value? Any servlet could impersonate
any other one.

> 4: Having received the random number from the http get call, Servlet xors
> it with xOr, and calls Decryptor.decrypt (String whoIAm, String password,
> int xOred)

So, the servlet makes the HTTP request and manages the checking? I
really don't get it.

> 5: Code checks xOred against whoIAm.  If gets a match decrypts the
> password and returns it from Decryptor.decrypt.

After all that, it's okay for the servlet to be able to read its own
database password? Why not just give the servlet access to a DataSource
with the password already set inside it?

> What I¹d rather have is a Tomcat provided Class with a static method that
> returns the calling URL.

Show me an implementation that Tomcat could provide that the servlet
could not tamper with.

> But so long as I can make the following two calls from my code:
> 
> URL theURL = new URL (urlString);
>  HttpURLConnectionuc = (HttpURLConnection) theURL.openConnection ();
> 
> I think I¹m good.
> 
> Flaws?

Let's see:

  Constructed dubious use case? CHECK
  Rolled your own security code? CHECK
  Used highly-secure XOR algorithm? CHECK
  Complicated enough to seem plausible? CHECK
  Confused objective leads to trivial attacks? CHECK

I think you've got all the bases covered, here.

I still don't understand what's being protected from whom, here. It
looks like the code is very complicated in order to keep the password
from the code, but then the code is given the password anyway. If you
trust the code but not the admin, then why bother with the complex code?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does Tomcat need a non-ssl connector?

[Christopher Schultz]

Msh,

On 2/14/16 3:25 PM, m...@kimwana.com wrote:
> On Thu, Feb 11, 2016 at 02:17:38PM -0500, m...@kimwana.com wrote:
> 
> redirectPort="443" />
> 
>  clientAuth="false"
> disableUploadTimeout="true" enableLookups="false" maxThreads="25"
> keystoreFile="/opt/tomcat/conf/keystore/dishwater.jks" 
> keystorePass="ImFr3eZ1inG"
> protocol="org.apache.coyote.http11.Http11NioProtocol" scheme="https"
> secure="true" sslProtocol="TLS" />
> 
> It works! Requests to port 80 area sent to 443, and there's no need
> to append a non-standard port!

Correct. jsvc allows you to bind to low-numbered ports because jsvc runs
with elevated privileges.

> The counterintuitive bit for me is, port 443 is seemingly configured
> twice.

How so?

> No need for iptables or httpd! Woot!

jsvc offers better features, too, such as being able to re-start the
service if it goes down unexpectedly, rotating stdout/stderr streams, etc.

A reverse-proxy such as httpd is never necessary unless you want to do
load-balancing. Especially when properly-configured, Tomcat can serve
static content just as fast as Apache httpd.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to set tomcat 8.0.20 container character encoding of request and response to UTF-8 intead of ISO-8859-1

[Christopher Schultz]

Akshat,

On 2/16/16 7:08 AM, Akshat Tandon wrote:
> We need to set tomcat 8.0.20 container character encoding of request and
> response to UTF-8 intead of ISO-8859-1 ,
> 
>  What is the setting for the same ?
> 
>  We tried setting as mentioned below ,
> https://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q1 But that requires
> creating filter etc .
> 
>  Is there any elegant way where we can just change some configuration set
> to make it applicable at container level ?

What, you don't like the filter already bundled with Tomcat that you can
simply configure? It really can't get any easier than that.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Block urls using X-forwaed-for

[Christopher Schultz]

Dave,

On 2/15/16 12:31 PM, Dave Fisher wrote:
> You can take the source code for RemoteIPFilter and have it look at the 
> X-Forwarded-For header.
> 
> Even better offer a patch to RemoteIPFilter to allow the header field name to 
> be configured. 
> 
> You are not the only one with this use case.

Even better, just read the documentation. No patch is required, and
anyone with this use case can use the out-of-the-box component.

http://tomcat.apache.org/tomcat-8.0-doc/config/filter.html#Remote_IP_Filter/Initialisation_parameters

-chris

>> On Feb 15, 2016, at 7:07 AM, André Warnier (tomcat)  wrote:
>>
>>> On 15.02.2016 13:03, Dhanushka Parakrama wrote:
>>> Hi Guys
>>>
>>> I have a Tomcat instance behind the load balancer (LB) , LB will pass the
>>> client ip address to backed tomcat instance using *X-forward-for*  header .
>>>
>>> I basically want to filter user traffic based on the *X-forwaed-for* header
>>> in tomcat instance
>>>
>>> i have configured the following filter in tomcat , but it not working
>>>
>>>
>>> 
>>> Remote Address Filter
>>> org.apache.catalina.filters.RemoteIpFilter
>>> 
>>> allow
>>>
>>> localhost|205\.97\.96\.\d+|::1|0:0:0:0:0:0:0:1
>>> 
>>> 
>>>
>>> 
>>> Remote Address Filter
>>> /mgt/*
>>> 
>>>
>>>
>>> Do you guys have any advice how to achieve it .
>>
>> Maybe have a look at : http://tuckey.org/urlrewrite/ ?
>>
>>
>>
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does Tomcat need a non-ssl connector?

[Christopher Schultz]

Msh,

On 2/13/16 12:28 PM, m...@kimwana.com wrote:
> On Fri, Feb 12, 2016 at 05:36:56PM -0500, Christopher Schultz wrote:
> Chuck,
> 
> On 2/12/16 3:06 PM, Caldarale, Charles R wrote:
> From: James H. H. Lampert [mailto:jam...@touchtonecorp.com] 
> Subject: Re: Does Tomcat need a non-ssl connector?

> On 2/12/16, 11:40 AM, m...@kimwana.com wrote:

>> Perhaps I should have phrased this differently. I want to
>> force clients to ssl. When they hit http://app.myurl.com their
>> browser should load https://app.myurl.com

> Wouldn't mind knowing that myself. All the Tomcat installations
> I'm responsible for are set up to simply reject non-secured
> connections (that's EASY, just comment out the non-secured
> connector); I'm sure some customers would like it to behave as
> you describe.

 Read the servlet spec.
> 
> +1
> 
 Simply set transport-guarantee to CONFIDENTIAL for all URL
 patterns (/*).  You can do this in the global conf/web.xml, if
 desired.
> 
>> Yeah-no. Nothing is that simple ;)
> 
> Not quite: there is still a bit of Tomcat configuration that needs to
> be done. The default configuration will do this correctly (redirect
> port 8080 -> 8443). If you want different port numbers, you'll need to
> read the configuration reference for the HTTP connector.
> 
>> jsvc is working for me but the redirect 80 --> 8080 (ssl) shows up in the 
>> url https://tomcat7.mynatnet.yo/8080
> 
>> I need to figure out how to have that uri load with ssl as 
>> https://tomcat7.mynatnet.yo


If jsvc is working for you, what's the port number you chose for SSL?
8080? (That's odd, most people use 8443). Why not just use the standard
port for HTTPS?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

How to set tomcat 8.0.20 container character encoding of request and response to UTF-8 intead of ISO-8859-1

[Akshat Tandon]

We need to set tomcat 8.0.20 container character encoding of request and
response to UTF-8 intead of ISO-8859-1 ,

 What is the setting for the same ?

 We tried setting as mentioned below ,
https://wiki.apache.org/tomcat/FAQ/CharacterEncoding#Q1 But that requires
creating filter etc .

 Is there any elegant way where we can just change some configuration set
to make it applicable at container level ?


*Thanks and RegardsAkshat Tandon*