Re: Authentication with Browser stopped working / missing exception handling in getRemainingLifetime

2021-11-24 Thread Mark Thomas

On 24/11/2021 08:06, Mark Thomas wrote:

On 23/11/2021 20:42, Michael B Allen wrote:

On Tue, Nov 23, 2021 at 2:59 PM Thomas Hoffmann (Speed4Trade GmbH)
 wrote:


Short Addendum:

The "destroyed" flag gets set, when the dispose-method of the 
GSSCredentialImpl was invoked.
Currently, I have no clue when and how it happens, but I have seen 
this problem every few months.
So it is only occurring sometimes. Maybe if the Kerberos ticket 
expires and the http session is still alive (?)


Nevertheless, the application should be able to recover from this 
situation and handles it like "not authenticated".


So as suspected it may actually be an invalid credential that maybe
Tomcat had a hand in. If Tomcat disposed the credential and then
subsequently tried to use it for any reason, that would be "invalid".
So that might warrant investigation before submitting a bug report.


That might be possible.

Tomcat calls dispose when the user explicitly logs out or the session 
expires. The OP is using http2 so parallel requests are likely.


I'll look into this some more.


It looks like concurrent requests for an expired session will trigger 
this. Avoiding the IllegalStateExcpetion would require adding 
synchronization to Request.getUserPrincipal(). Given that issue occurs 
once every few months my preference is to catch the ISE rather than 
avoid it since avoiding it has a (small) impact on every request.


I've checked and multiple calls to dispose() are safe so the current 
approach appears to be sound. I'll add some comments to the code for 
future maintainers.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Authentication with Browser stopped working / missing exception handling in getRemainingLifetime

2021-11-24 Thread Mark Thomas

On 23/11/2021 20:42, Michael B Allen wrote:

On Tue, Nov 23, 2021 at 2:59 PM Thomas Hoffmann (Speed4Trade GmbH)
 wrote:


Short Addendum:

The "destroyed" flag gets set, when the dispose-method of the GSSCredentialImpl 
was invoked.
Currently, I have no clue when and how it happens, but I have seen this problem 
every few months.
So it is only occurring sometimes. Maybe if the Kerberos ticket expires and the 
http session is still alive (?)

Nevertheless, the application should be able to recover from this situation and handles 
it like "not authenticated".


So as suspected it may actually be an invalid credential that maybe
Tomcat had a hand in. If Tomcat disposed the credential and then
subsequently tried to use it for any reason, that would be "invalid".
So that might warrant investigation before submitting a bug report.


That might be possible.

Tomcat calls dispose when the user explicitly logs out or the session 
expires. The OP is using http2 so parallel requests are likely.


I'll look into this some more.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: help with Valve

2021-11-23 Thread Mark Thomas

On 23/11/2021 17:42, Rob Sargent wrote:
Thank you.  Does this look like a believable deployment (presuming the 
property is in fact set)?


    cat ./localhost/sgs/META-INF/context-valve.xml
       
       


No.

A context.xml file placed in META-INF must be called context.xml

The contents needs to look like this:







Note the quotes on maxDays. The parser might let you get away without 
them but the standard Tomcat config files always use them.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: help with Valve

2021-11-23 Thread Mark Thomas

On 23/11/2021 16:48, Rob Sargent wrote:

Is the Access Log Valve available for use in an embedded environment?


Yes.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Handling database connection pooling outside Java, without DBCP et al?

2021-11-23 Thread Mark Thomas

On 23/11/2021 13:34, Olaf Kock wrote:

I don't have experience with this particular setup, but one sentence (in
fact, one word) caught my attention:

On 23.11.21 14:23, jkla...@iki.fi wrote:

We're in the process of adopting ProxySQL in front of MySQL, to act as the 
connection pooler and for separating read and write traffic to different 
database instances. After this, we have no need for DBCP or any other 
Java-level pooling – in fact, having two levels of connection pooling would 
probably be detrimental to performance, and certainly to our ability to 
diagnose issues.


The keyword here is "probably": I'd say: Finish your migration with all
the current setup, and check if performance is worse than you expect (or
can live with) (and if diagnosis is hindered).

Then, if you /still/ want to eliminate Java pooling: The connection
pooling interfaces aren't that big - I'm not aware of a
non-pooling-connection-pool (or a configuration that disables what the
pool has been written for in the first place), but implementing your own
non-pooling-connection-pool should not be that much work, as you're only
handing out new connections unconditionally, and close them when the app
asks you to close them. And then configure Tomcat to use your
implementation instead of DBCP.


No need for this.

DBCP looks exactly like a database driver to the application. So, rather 
than configuring DBCP as a wrapper for the database driver, you just 
configure the database driver directly.


The short version is:
- remove the pooling configuration from the resource definition
- change / set the factory attribute the the correct value for your
  driver (probably com.mysql.cj.jdbc.MysqlDataSourceFactory)

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT] Handling database connection pooling outside Java, without DBCP et al?

2021-11-23 Thread Mark Thomas

On 23/11/2021 13:51, Richardson, Diane wrote:

I get the emails but how can I send an email for assistance?


Please don't hijack threads.

See http://tomcat.apache.org/lists.html#taglibs-user

Specifically, "Posting questions to the list"

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Authentication with Browser stopped working / missing exception handling in getRemainingLifetime

2021-11-22 Thread Mark Thomas

On 22/11/2021 07:38, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello,
we are using apache-tomcat-9.0.54 with LDAP authentication under Windows 2012R2.
One of the user complained that access with Firefox stopped working.





Would it be better to also catch IllegalStateException and instead of checking 
left == 0 to change it to left <= 0 ?


Both fair points. Thanks for bringing them to the community's attention. 
I've fixed both issues for the next set of releases.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: JASPIC Provider for FORM based Authentication

2021-11-22 Thread Mark Thomas

On 22/11/2021 12:00, Keil, Matthias (ORISA Software GmbH) wrote:

Hello everyone,

I take up a topic of my own again. The point there was that I would like to 
accommodate both the configuration and the actual Server Auth module within the 
application.
That worked well with your advice.

Unfortunately, I have now reached the point where this solution works for one 
application, but a parallel application on the same Tomcat then also uses the Server 
Auth module. There is a  entry in the web.xml of the second 
application.
In my opinion, this is ignored as soon as JASPIC is configured for this Tomcat 
(either statically with jaspic-providers.xml or dynamically by implementing an 
AuthConfigProvider).

Now here are my questions:
1. Is there a possibility to activate the JASPIC provider for only one of the 
two applications?


The intention is that the appContext attribute for the provider in the 
jaspic-providers.xml file limits the JASPIC configuration to a single 
web application.



2. OR there is an AuthConfigProvider that could implement the FORM based 
authentication.


Not that I am aware of.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.0.13 available

2021-11-15 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.0.13.

This release is targeted at Jakarta EE 9.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

The notable changes compared to 10.0.12 include:

- Experimental OpenSSL support through the Panama API incubating in Java
  17, with support for OpenSSL 1.1+

- Add support for custom caching strategies for web application
  resources. This initial implementation allows control over whether or
  not a resource is cached.

- Improve robustness of JNDIRealm for exceptions occurring when getting
  the connection.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.0-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.1.0-M7 (alpha) available

2021-11-15 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.1.0-M7 (alpha).

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10.1.0-M7 is a milestone release of the 10.1.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 10.1.x so that they may provide feedback. The notable 
changes compared to 10.1.0-M6 include:


- Servlet API updates for Servlet 6 including refactoring
  HttpServlet.doHead(), support for generic attributes on Cookies, more
  consistent URI handling including an option to reject 'suspicious'
  URIs

- EL API updates for EL 5.0 including changes to ELResolver.getType()

- Experimental OpenSSL support through the Panama API incubating in Java
  17, with support for OpenSSL 1.1+

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.1-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Mimicking two distinct virtual hosts just like in HTTPd

2021-11-08 Thread Mark Thomas

On 08/11/2021 15:36, Michael Osipov wrote:

Folks,

consider the following in httpd.conf:

Listen {IP}:8443
Listen {IP}:8444

later:

   DocumentRoot /www/webapps1
   ServerName {hostname}
   mod_ssl config...


   DocumentRoot /www/webapps2
   ServerName {hostname}
   mod_ssl config2...


The second virtual host shall deliver only a subset of webapps1, but 
configured client cert auth to avoid issues with PHA and renotiation.


Now, I am looking for the same in Tomcat. Given that I have one Server, 
one Service, two Connectors one Engine and two Hosts there is no way to 
bind a Host in Tomcat to an listen address, but only to a hostname/IP 
address.
 From my understanding of the server.xml I would need set up *two* 
Service elements with one Engine, Connector and Host each.


Is my understanding correct?


Yes. Connector elements are associated with a Service so if you want 
different Hosts on different ports you need different Services.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: ROOT.xml and META-INF/context.xml

2021-11-07 Thread Mark Thomas

On 07/11/2021 10:44, Greg Huber wrote:

Hello,

I am testing CookieProcessor for SameSite stuff.

In my dev environment,  I use ROOT.xml with

workDir="/home/me/project/work" />


I have a context.xml file in META-INF but it seems to ignore it. When I 
deploy via a war it is picked up OK.


A web application can only have one context.xml file. If you have 
deployed it via a $CATALINA_BASE/conf//ROOT.xml then any 
context.xml file in the application will be ignored.


When you deploy via a WAR, the context.xml will be used (and may be 
extracted).



webapp/META-INF/context.xml

webapp/WEB-INF/.


Is this normal logic?


Yes.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: End of life dates

2021-11-04 Thread Mark Thomas

On 04/11/2021 13:50, David J Pearson wrote:

Hi - What are the end of life / end of support dates for v8.5 and v9
please ?


They haven't been set yet.

The Tomcat project will provide at least 12 months notice of EOL for any 
major Tomcat version.


That said, major versions have been reaching EOL about once every three 
years. Tomcat 7 was EOL earlier this year so sometime in 2024 for Tomcat 
8 seems like a reasonable guess at the moment.


Tomcat 9 is a special case. It would normally be ~3 years after EOL for 
Tomcat 8 (so ~2027) but as the last Tomcat release that supports Java EE 
(Tomcat 10 onwards supports Jakarta EE with the associated change in API 
package names) the Tomcat community intends to support it for as long as 
there is community interest in doing so.


The rough plan is that rather than EOL Tomcat 9, we will provide Tomcat 
9.10.x that will track the Tomcat 10 releases (but with the Java EE 
APIs). When Tomcat 10 reaches EOL 9.10.x will become 9.11.x and track 
the Tomcat 11 releases (but with the Java EE APIs) and so on as long as 
there is interest/demand for Java EE support.


Note that this is only a rough plan for Tomcat 9 and that we are 
probably 5 to 6 years away from implementing it so lots of things might 
happen between now and then.


HTH,

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Strange Oracle JDBC Driver error on Application Deployment

2021-11-03 Thread Mark Thomas

On 02/11/2021 22:26, jonmcalexan...@wellsfargo.com.INVALID wrote:

I have an application team that is getting the following stack trace while starting 
Tomcat 8.5.70. I've done some searching but can't find anything. In looking at their 
context.xml it appears that they have jmxEnabled="false" in each of the 
resources.


Jon,

The line numbers listed in the stack trace aren't consistent with the 
version number quoted above. I've tried stepping back through old 8.5.x 
versions and I can't find a match unless I go back 10 years (before 
Tomcat 8.5.x and even 8.0.x existed).


It looks like you are seeing this issue:
https://bz.apache.org/bugzilla/show_bug.cgi?id=54194

Possible explanations:

- the application is packaged with a (very) old version of Tomcat's
  jdbc-pool and is using that rather than the version provided by Tomcat

- The Tomcat instances are running a much older Tomcat version than
  8.5.70

- The Tomcat instances are using some form of custom patch

- Something else.

HTH,

Mark



Any assistance would be grand.

Thanks,

 Stack Trace 

02-Nov-2021 13:01:45.809 SEVERE [localhost-startStop-1] 
org.apache.tomcat.jdbc.pool.DataSource.registerJmx Unable to register JDBC pool 
with JMX
 java.lang.NullPointerException
 at 
org.apache.tomcat.jdbc.pool.DataSource.registerJmx(DataSource.java:129)
 at 
org.apache.tomcat.jdbc.pool.DataSource.preRegister(DataSource.java:98)
 at 
org.apache.tomcat.util.modeler.BaseModelMBean.preRegister(BaseModelMBean.java:927)
 at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1007)
 at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:919)
 at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
 at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
 at 
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
 at 
org.apache.tomcat.util.modeler.Registry.registerComponent(Registry.java:637)
 at 
org.apache.catalina.core.NamingContextListener.addResource(NamingContextListener.java:1014)
 at 
org.apache.catalina.core.NamingContextListener.createNamingContext(NamingContextListener.java:552)
 at 
org.apache.catalina.core.NamingContextListener.lifecycleEvent(NamingContextListener.java:245)
 at 
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
 at 
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5130)
 at 
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
 at 
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753)
 at 
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:727)
 at 
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:695)
 at 
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1016)
 at 
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1903)
 at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
02-Nov-2021 13:01:46.066 SEVERE [localhost-startStop-1] 
org.apache.tomcat.jdbc.pool.DataSource.registerJmx Unable to register JDBC pool 
with JMX
 java.lang.NullPointerException
 at 
org.apache.tomcat.jdbc.pool.DataSource.registerJmx(DataSource.java:129)
 at 
org.apache.tomcat.jdbc.pool.DataSource.preRegister(DataSource.java:98)
 at 
org.apache.tomcat.util.modeler.BaseModelMBean.preRegister(BaseModelMBean.java:927)
 at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.preRegister(DefaultMBeanServerInterceptor.java:1007)
 at 

Re: Host-wide Singleton Instance

2021-11-03 Thread Mark Thomas

On 02/11/2021 20:47, Jerry Malcolm wrote:
I am adding a redis implementation  (jedis) to my application.  I have a 
jedis implementation class that holds the connection pool and interfaces 
with jedis.  That class needs to be instantiated once per host and then 
referenced from that point on by all of the webapps in the host.  Is 
there an 'architected/correct' way to set this up?


I'd suggest configuring it as a global JNDI resource [1] and then using 
resource links to make it available to the web applications that need 
it. If every web application needs it, you can add the resource link to 
$CATALINA_BASE/conf/context.xml and then will add the link to every web 
application deployed on the Tomcat instance.


Mark


[1] 
http://tomcat.apache.org/tomcat-9.0-doc/jndi-resources-howto.html#Global_configuration


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: About the comment of org.apache.tomcat.util.threads.TaskQueue

2021-10-27 Thread Mark Thomas

On 27/10/2021 02:58, Poison wrote:

Ok, I'm just curious, because 
org.apache.catalina.tribes.util.ExecutorFactory.TribesThreadPoolExecutor 
inherits java.util.concurrent.ThreadPoolExecutor but 
org.apache.tomcat.util.threads.ThreadPoolExecutor does not.


They are implementing different behaviours.


Similarly, in the open source project dubbo, EagerThreadPoolExecutor inherits 
java.util.concurrent.ThreadPoolExecutor to achieve similar functions as the 
tomcat thread pool.


https://github.com/apache/dubbo/blob/dubbo-2.7.5/dubbo-common/src/main/java/org/apache/dubbo/common/threadpool/support/eager/EagerThreadPoolExecutor.java#L30


The Dubbo implementation does not appear to achieve the behaviour Tomcat 
requires.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: About the comment of org.apache.tomcat.util.threads.TaskQueue

2021-10-26 Thread Mark Thomas

On 26/10/2021 09:47, Poison wrote:

Thank you for your detailed explanation. Now I understand the background of 
this part of the comment. When corePoolSize is equal to maxThreads, the native 
implementation will create threads first.


There is another question. Why does 
org.apache.tomcat.util.threads.ThreadPoolExecutor almost copy the code of 
java.util.concurrent.ThreadPoolExecutor instead of implementing it by inheriting 
java.util.concurrent.ThreadPoolExecutor? This is what I don't 
understand.Hope you can explain the design concept behind, thank you.


j.u.c.ThreadPoolExecutor isn't designed in a way we can extend it and 
change the things we need to change. Copying it was the only option.


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: About the comment of org.apache.tomcat.util.threads.TaskQueue

2021-10-26 Thread Mark Thomas

On 26/10/2021 02:45, Poison wrote:

Thank you, I know the role of TaskQueue, but the comment about "normal queue" 
on the TaskQueue class is still incomprehensible.


In the java.util.concurrent.ThreadPoolExecutor#execute method, the comment mentions: "3. If we 
cannot queue task, then we try to add a new thread. If it fails, we know we are shut down or 
saturated and so reject the task.". Explain that the JDK prioritizes putting tasks into the 
queue rather than creating threads first, which is not consistent with the statement that 
"normal queue" mentioned in TaskQueue prioritizes thread creation.


You need to look at the custom queue and custom ThreadPoolExecutor together.

From the Javadoc of j.u.c.ThreadPoolExecutor:


 * When a new task is submitted in method {@link #execute(Runnable)},
 * if fewer than corePoolSize threads are running, a new thread is
 * created to handle the request, even if other worker threads are
 * idle.  Else if fewer than maximumPoolSize threads are running, a
 * new thread will be created to handle the request only if the queue
 * is full.  By setting corePoolSize and maximumPoolSize the same, you
 * create a fixed-size thread pool. By setting maximumPoolSize to an
 * essentially unbounded value such as {@code Integer.MAX_VALUE}, you
 * allow the pool to accommodate an arbitrary number of concurrent
 * tasks. Most typically, core and maximum pool sizes are set only
 * upon construction, but they may also be changed dynamically using
 * {@link #setCorePoolSize} and {@link #setMaximumPoolSize}.


The problem is that the default behaviour prioritises in this way:
- start corePoolSize threads
- queue tasks
- start additional threads up to maximumPoolSize if the queue is full

That isn't the behaviour we want in Tomcat. In Tomcat we want:
- start corePoolSize threads
- start additional threads up to maximumPoolSize
- queue tasks using an (effectively) infinite queue

That change has significant implications. These are most obviously seen 
in Tomcat's TaskQueue.offer(Runnable) implementation.


The closest you can get to the required Tomcat behaviour with 
j.u.c.ThreadPoolExecutor is by setting corePoolSize to maxThreads. 
However, this has the side effect that threads will be created for each 
new task until maxThreads have been started - hence the comment that it 
prioritises thread creation rather than re-using idle threads. You 
essentially end up with a fixed size thread pool of maxThreads - rather 
than what we want which (by default) is a thread pool of at least 
minSpareThreads that grows, if necessary, up to maxThreads and then 
shrinks back down to minSpareThreads while idle.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: xsd version used for web.xml etc

2021-10-21 Thread Mark Thomas

On 21/10/2021 10:37, S Abirami wrote:

Hi Thomas,

How I can identify whether the schema validation enabled or not.
I checked startup logs and other configuration.

I am unable to find it.


The quick test is to add an unknown element to web.xml and see what 
happens. If you get an error, validation is enabled. If it is ignored, 
validation is not enabled.


It is typically configured in $CATALINA_BASE/conf/context.xml

See https://tomcat.apache.org/tomcat-9.0-doc/config/context.html
Of particular interest:
xmlNamespaceAware
xmlValidation

Mark




Regards,
Abirami.S

-Original Message-
From: Mark Thomas 
Sent: Thursday, October 21, 2021 2:40 PM
To: users@tomcat.apache.org
Subject: Re: xsd version used for web.xml etc

On 21/10/2021 09:45, S Abirami wrote:

Hi All,

In web.xml, if we didn't define any xsd schema or dtd schema which version of 
xsd will be loaded for Tomcat 9.0.45.


By default none - whether a schema is defined or not. Schemas are only loaded 
if validation is enabled.

With validation disabled, Tomcat will treat the content of a web.xml file as if 
it is using the schema associated with the Servlet 4.0 specification.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: xsd version used for web.xml etc

2021-10-21 Thread Mark Thomas

On 21/10/2021 09:45, S Abirami wrote:

Hi All,

In web.xml, if we didn't define any xsd schema or dtd schema which version of 
xsd will be loaded for Tomcat 9.0.45.


By default none - whether a schema is defined or not. Schemas are only 
loaded if validation is enabled.


With validation disabled, Tomcat will treat the content of a web.xml 
file as if it is using the schema associated with the Servlet 4.0 
specification.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

2021-10-19 Thread Mark Thomas

On 19/10/2021 06:20, Natraj Thekkan wrote:

Hi Mark or Chris,

Based on Chris statement, it has to be addressed in tomcat.


No, you has misunderstood Chris's statement. All the evidence so far 
points to user error.


Again, you need to provide the simplest, *complete* test case (i.e. the 
source code for an executable Java class that starts a Tomcat instance 
that listens for HTTP/2 connections) that responds to TLS 1.0 and 1.1 
connections when configured not to.



Can I raise a Bug in Bugzilla for this observation?.


No.

Mark




Regards,
Natraj
-Original Message-
From: Christopher Schultz 
Sent: Monday, October 18, 2021 10:14 PM
To: users@tomcat.apache.org
Subject: Re: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

Natraj,

On 10/18/21 01:19, Natraj Thekkan wrote:

@Mark
Thanks for your response.

We have tested by removing that line of code, still client able to establish 
the connection with server using TLSv1 and TLSv1.1. Below one is configured in 
java.security file.

jdk.tls.disabledAlgorithms=SSLv3,TLSv1,TLSv1.1,RC4,MD5withRSA,ADH,DH,DHE,
  DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
  include jdk.disabled.namedCurves


Note that OpenSSL will ignore the jdk.tls.disabledAlgorithms setting.

Mark (and others), maybe we should take jdk.tls.disabledAlgorithms into account 
when configuring OpenSSL through JSSE, since a user might expect that all JSSE 
providers will respect that setting.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

2021-10-18 Thread Mark Thomas

On 18/10/2021 06:19, Natraj Thekkan wrote:

Hi,

@Mark
Thanks for your response.

We have tested by removing that line of code, still client able to establish 
the connection with server using TLSv1 and TLSv1.1. Below one is configured in 
java.security file.

jdk.tls.disabledAlgorithms=SSLv3,TLSv1,TLSv1.1,RC4,MD5withRSA,ADH,DH,DHE,
 DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
 include jdk.disabled.namedCurves

Please suggest the way to restrict the TLSv1,TLSv1.1 version when 
OpenSSLImplementation is used.


The code you are using should be sufficient.

Please provide the simplest, *complete* test case (i.e. the source code 
for an executable Java class that starts a Tomcat instance that listens 
for HTTP/2 connections) that responds to TLS 1.0 and 1.1 connections 
when configured not to.


(We can provide our our test certificate.)

Mark




Regards,
Natraj

-Original Message-
From: Mark Thomas 
Sent: Thursday, October 14, 2021 4:11 PM
To: users@tomcat.apache.org
Subject: Re: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

On 14/10/2021 10:28, Natraj Thekkan wrote:

Hi,

We are using tomcat version 9.0.46.
Could you please provide suggestion to restrict the TLS version in HTTP2 over 
HTTPS with OpenSSL implementation?.


The code below is sufficient, assuming that is then the connector that is being 
used by the clients.

You should be able to remove to remove the

sslHostConfig.setSslProtocol("TLS");

line. It is only used with JSSE.

Mark




Regards,
Natraj
From: Natraj Thekkan
Sent: Wednesday, October 13, 2021 10:15 AM
To: 'users@tomcat.apache.org' 
Subject: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

Hi,

We have tried to restrict the TLS version in https connection establishment in 
embedded tomcat for OpenSSL based implementation. With this part of the code, 
TLSv1.0/TLSv1.1 client also able to connect with our https server. Please let 
us know how we can restrict the TLS version in HTTP2 over HTTPS in OpenSSL 
implementation.

Below code is used while creating connector.

private final String[] enabledProtocol = new String[] { "TLSv1.2" };


SSLHostConfig sslHostConfig = new SSLHostConfig();

sslHostConfig.setInsecureRenegotiation( false );

sslHostConfig.setCertificateFile( certLocation );

sslHostConfig.setCertificateKeyFile( certKeyLocation );

sslHostConfig.setCertificateKeyPassword( certKeyPassword );

if( isClientAuthreq && caCertificatePath != null &&
!caCertificatePath.isEmpty() )

{

sslHostConfig.setCertificateVerification(
CertificateVerification.REQUIRED.toString() );

sslHostConfig.setCaCertificateFile( caCertificatePath );

}

sslHostConfig.setSslProtocol("TLS");

sslHostConfig.setEnabledProtocols( enabledProtocol );
this.addSslHostConfig( sslHostConfig );
IntrospectionUtils.setProperty( this, "SSLEnabled", "true" );
IntrospectionUtils.setProperty( this, "sslImplementationName",
"org.apache.tomcat.util.net.openssl.OpenSSLImplementation" );


Regards,
Natraj




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: How do I install and use Apache Tomcat?

2021-10-16 Thread Mark Thomas

On 16/10/2021 10:41, Turritopsis Dohrnii Teo En Ming wrote:

Subject: How do I install and use Apache Tomcat?

Good day from Singapore,

How do I install and use Apache Tomcat? I understand it is a Java web server.


Which operating system do you want to use?

Do you have a specific version of Java that you want to use (e.g. 
because that is the version installed by default for your chosen 
operating system).


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 8.5.37 is automatically redeploying apps on every Saturday

2021-10-16 Thread Mark Thomas

On 15/10/2021 21:15, Shekhar Naidu wrote:

The tomcat is not running in any containers.
We don’t have anything Linux cron.

The containerbackgroundprocessor which I mentioned is within the tomcat.
The tomcat’s Catalina.out file printing that name when doing the app
undeploy and deploy on Saturday.


Is Tomcat and/or the WAR file using a local filesystem?

The only things that will trigger an undeploy are:
- if the WAR file disappears
- if the WAR file changes its timestamp

If any sort of network file system is in use, I'd look at what that is 
doing around the time of the undeploy. I'd also look at the timestamp of 
the WAR file before and after the undeploy/deploy.


Mark





On Fri, Oct 15, 2021 at 11:17 AM Darryl Philip Baker <
darryl.ba...@northwestern.edu> wrote:


On 10/15/21, 10:05 AM, "jonmcalexan...@wellsfargo.com.INVALID"
 wrote:

 > -Original Message-
 > From: Shekhar Naidu 
 > Sent: Friday, October 15, 2021 7:45 AM
 > To: users@tomcat.apache.org
 > Subject: Tomcat 8.5.37 is automatically redeploying apps on every
Saturday
 >
 > Hi all,
 >
 > > We are seeing a weird behavior in our new Linux environments.
Since we
 > >> migrated from RHEL6 to 8, we started seeing issue with tomcat.
Tomcat
 > >> is auto redeploying our apps on every Saturday around 12:20AM.
 > >> We don’t have any schedulers running on our machines. We verified
 > >> localhost_access.log and found no request to the tomcat manager
to do
 > >> the redeploy.
 > >>
 > >> The Catalina.out prints that “ContainerBackgroundProcessor” is
doing
 > >> the undeploy and deploy of our app.
 > >> It does not print any other details like why it is doing that.
 > >>
 > >> We are basically out of Ideas as where to look as we have looked
 > >> everything on the Linux side, crion jobs etc..
 > >>
 > >> Appreciate your help with this. Thank you in advance.
 > >>
 > >> Thank you
 > >> Shekar
 > >>
 > > --
 > Shekhar
 > simplysek...@gmail.com
 > shekharna...@gmail.com

I would check /etc/cron.weekly for a restart and your logrotate configuration
to be sure there isn't a restart

that happens as part of the log rotation.

Darryl Baker, GSEC  (he/him/his)
Sr. System Administrator
Distributed Application Platform Services
Northwestern University
4th Floor
2020 Ridge Avenue
Evanston, IL  60208-0801
darryl.ba...@northwestern.edu
(847) 467-6674 


--

Shekhar
simplysek...@gmail.com
shekharna...@gmail.com




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Form based auth does not provide the option to show error reason in the error page

2021-10-15 Thread Mark Thomas

On 15/10/2021 07:05, Werner Dähn wrote:




So why has this not been done? What am I missing?


Accepted security good practice is not to provide any information to a 
user as to the reason for a failed authentication. The idea is that it 
could help an attacker by, for example, letting them know they have a 
valid user name but an invalid password.


I'm not entirely convinced by the arguments used to support the above 
position. They generally seem to be based on the assumption that a brute 
force attack is possible. I'd argue that any system susceptible to a 
brute force attack has problems irrespective of whether it provides 
feedback on authentication failures.


I do think there is an argument to be made that trading reduced 
usability (no feedback on authentication failures) for allegedly better 
security (brute force attacks are harder) is not a sensible trade-off. 
That said, I appear to be in the minority. Again.



Does an enhancement request exist??


No.

I do think there is an argument for providing information on the reason 
for the authentication failure via a mechanism that allows system 
administrators to decide if they want to pass it on to the users or not. 
Something like a request attribute that could be included in a custom 
error page for example.


However, the current Tomcat code for authentication is structured in 
such a way that exposing the reason for an authentication failure would 
require a reasonable amount of refactoring. I don't think an enhancement 
request along these lines will be rejected, but neither do I think it 
will be implemented quickly. I'd expect a fair amount of discussion 
about how to refactor the Realm interface to expose this information.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[SECURITY] CVE-2021-42340 Apache Tomcat DoS

2021-10-14 Thread Mark Thomas

CVE-2021-42340 Denial of Service

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
Apache Tomcat 10.1.0-M1 to 10.1.0-M5
Apache Tomcat 10.0.0-M10 to 10.0.11
Apache Tomcat 9.0.40 to 9.0.53
Apache Tomcat 8.5.60 to 8.5.71

Description:
The fix for bug 63362 introduced a memory leak. The object introduced to 
collect metrics for HTTP upgrade connections was not released for 
WebSocket connections once the WebSocket connection was closed. This 
created a memory leak that, over time, could lead to a denial of service 
via an OutOfMemoryError.


Mitigation:
Users of the affected versions should apply one of the following
mitigations:
- Upgrade to Apache Tomcat 10.1.0-M6 or later
- Upgrade to Apache Tomcat 10.0.12 or later
- Upgrade to Apache Tomcat 9.0.54 or later
- Upgrade to Apache Tomcat 8.5.72 or later

History:
2021-10-14 Original advisory
2021-10-14 Correct CVE reference in body of advisory

References:
[1] https://tomcat.apache.org/security-10.html
[2] https://tomcat.apache.org/security-9.html
[3] https://tomcat.apache.org/security-8.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[SECURITY] CVE-2021-42340 Apache Tomcat DoS

2021-10-14 Thread Mark Thomas

CVE-2021-41079 Denial of Service

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
Apache Tomcat 10.1.0-M1 to 10.1.0-M5
Apache Tomcat 10.0.0-M10 to 10.0.11
Apache Tomcat 9.0.40 to 9.0.53
Apache Tomcat 8.5.60 to 8.5.71

Description:
The fix for bug 63362 introduced a memory leak. The object introduced to 
collect metrics for HTTP upgrade connections was not released for 
WebSocket connections once the WebSocket connection was closed. This 
created a memory leak that, over time, could lead to a denial of service 
via an OutOfMemoryError.


Mitigation:
Users of the affected versions should apply one of the following
mitigations:
- Upgrade to Apache Tomcat 10.1.0-M6 or later
- Upgrade to Apache Tomcat 10.0.12 or later
- Upgrade to Apache Tomcat 9.0.54 or later
- Upgrade to Apache Tomcat 8.5.72 or later

History:
2021-10-14 Original advisory

References:
[1] https://tomcat.apache.org/security-10.html
[2] https://tomcat.apache.org/security-9.html
[3] https://tomcat.apache.org/security-8.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

2021-10-14 Thread Mark Thomas

On 14/10/2021 10:28, Natraj Thekkan wrote:

Hi,

We are using tomcat version 9.0.46.
Could you please provide suggestion to restrict the TLS version in HTTP2 over 
HTTPS with OpenSSL implementation?.


The code below is sufficient, assuming that is then the connector that 
is being used by the clients.


You should be able to remove to remove the

sslHostConfig.setSslProtocol("TLS");

line. It is only used with JSSE.

Mark




Regards,
Natraj
From: Natraj Thekkan
Sent: Wednesday, October 13, 2021 10:15 AM
To: 'users@tomcat.apache.org' 
Subject: Restriction of TLS version in HTTP2 over HTTPS with OpenSSL

Hi,

We have tried to restrict the TLS version in https connection establishment in 
embedded tomcat for OpenSSL based implementation. With this part of the code, 
TLSv1.0/TLSv1.1 client also able to connect with our https server. Please let 
us know how we can restrict the TLS version in HTTP2 over HTTPS in OpenSSL 
implementation.

Below code is used while creating connector.

private final String[] enabledProtocol = new String[] { "TLSv1.2" };


SSLHostConfig sslHostConfig = new SSLHostConfig();

sslHostConfig.setInsecureRenegotiation( false );

sslHostConfig.setCertificateFile( certLocation );

sslHostConfig.setCertificateKeyFile( certKeyLocation );

sslHostConfig.setCertificateKeyPassword( certKeyPassword );

if( isClientAuthreq && caCertificatePath != null && 
!caCertificatePath.isEmpty() )

{

sslHostConfig.setCertificateVerification( 
CertificateVerification.REQUIRED.toString() );

sslHostConfig.setCaCertificateFile( caCertificatePath );

}

sslHostConfig.setSslProtocol("TLS");

sslHostConfig.setEnabledProtocols( enabledProtocol );
this.addSslHostConfig( sslHostConfig );
IntrospectionUtils.setProperty( this, "SSLEnabled", "true" );
IntrospectionUtils.setProperty( this, "sslImplementationName", 
"org.apache.tomcat.util.net.openssl.OpenSSLImplementation" );


Regards,
Natraj




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Security Vulnerability Question

2021-10-13 Thread Mark Thomas

On 13/10/2021 19:16, Kenaw, Seretseab wrote:

Hello,

Our IT team just notified us with a severe security vulnerability on our web 
application with the Tomcat version that we are using (9.0.12). What 
remediations can we use to quickly fix the issue?


Upgrade Tomcat.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Help needed reg Context

2021-10-13 Thread Mark Thomas

On 13/10/2021 14:19, Mohan T wrote:

Dear All,

We are using Tomcat 8.5 on Suse linix.

We are deploying one of our artifacts as below

hub#app#classic#admin.war

The components are also deployed and the context is also created 
Successfully.


Is there any other alternative way to set the context other than using 
*#*.


Please guide us.


Move the WAR file *outside* of the Host's appBase. i.e. You need to move 
it out of the webapps directory.


You then have two options.

1. Recommended
Add a file named
$CATALINA_BASE/conf/Catalina/localhost/hub#app#classic#admin.xml

with the following content:



2. Not recommended
Nest the following inside the Host element in server.xml


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Missing TLS cipher suite definition

2021-10-10 Thread Mark Thomas

On 10/10/2021 13:00, Christopher Schultz wrote:

On 10/9/21 04:52, Mark Thomas wrote:




If the user is using e.g. BouncyCastle, IBM's JRE, Corretto, etc. those 
ciphers might be available in those environments. (It looks like BC 
supports this cipher suite, but I couldn't find any information on IBM 
or Corretto stating one way or the other).


We have supported cipher lists from at least some of those in the test 
suite checking for missing mappings. There is always the scope to 
additional supported cipher lists from other JVMs and/or JSSE providers.


Will them being missing from the Ciphers enum prevent them from being 
used at all? OR will it only prevent them from being aliases of each other?


Looking at the source, my reading is a cipher needs to be in Ciphers to 
used.


I'll note that in that case it is a DSA based cipher suite so I'd be 
surprised to find it in use in a production scenario.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Missing TLS cipher suite definition

2021-10-09 Thread Mark Thomas

On 08/10/2021 19:34, Farber, Ilja wrote:

Hi all,

I noticed org.apache.tomcat.util.net.openssl.ciphers.Cipher does not define the 
cipher suites defined by rfc 6367 and 6209. The ciphers are listed
https://docs.oracle.com/javase/9/docs/specs/security/standard-names.html

and should be valid for TLS 1.2.



For example TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256


The above cipher is 0xC05C and is present in Ciphers.


or TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_GCM_SHA256


The above cipher is 0xC086. As far as I am aware it is neither supported 
by Java nor OpenSSL hence it is not present in Ciphers.



Is there a reason, why these cipher suites are not in enum Cipher?


The purpose of the Enum is to map between Java cipher definitions and 
OpenSSL cipher definitions. If a cipher is unsupported by both there is 
no point including it.


There are Tomcat unit tests that should check for unknown ciphers so I'd 
expect any new ciphers to trigger test failures. We do see these from 
time to time as OpenSSL adjusts its ciphers so I think they are working 
correctly.


If you are aware of a cipher that is supported by any current version of 
Java or OpenSSL that is missing from Ciphers and isn't triggering a test 
failure then please bring it to our attention.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Test valve with tomcat-embed 9?

2021-10-08 Thread Mark Thomas

On 08/10/2021 11:43, Me Self wrote:

I would like to test a custom tomcat valve with tomcat-embed and junit. Is
that possible?

Found a few tomcat-embed samples on the web but most seem to only deal with
setting up a webapp - something along the lines:

@BeforeAll
public static void setup() throws LifecycleException {
   Tomcat tomcat = new Tomcat();
   tomcat.setPort(...);
   StandardContext ctx = (StandardContext) tomcat.addWebapp("/", new
File("src/main/webapp/").getAbsolutePath());

What would I need to do to add a valve? And btw. it's a maven project so
the valve is compiled to "target/classes".


https://github.com/apache/tomcat/tree/main/test/org/apache/catalina/valves

Mark



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: JASPIC Plugin for OIDC/JWT/OAuth

2021-10-08 Thread Mark Thomas

On 07/10/2021 18:37, Michael Kolenda wrote:

Hey Tomcat Users,

I've run into an interesting behavior with a custom JASPIC provider. When
there is an existing session i.e. JSESSIONID cookie, It appears the
groups/roles are not checked again... even when the new groups are provided
in the client Subject (JASPIC's validate() ). When attempting stateless
authentication via JWT/OAuth how can I ignore a previously set session for
an individual request?

It appears to be based around equals() on my Principal object. I can make
it so Principal's generated via stateless authentication protocols are
never equal, but then I get a new session id in the response. I don't want
a session id at all for this request


I'm only basing this on looking at Tomcat's source code so I may be on 
the wrong track.


You probably want to set cache="false" on your authenticator. That will 
stop Tomcat trying to cache the authenticated principal in the session.


From your description and looking at the source for AuthenticatorBase, 
I think that should address the issue you are seeing.


You might also want to check if alwaysUseSession has been set. If not, 
the default of false is fine but I don't think you want this set to true.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding

2021-10-08 Thread Mark Thomas

On 07/10/2021 22:23, Javateck wrote:

Hi Mark,

Just wondering whether we have a radar to track this, will it be in release 
notes for next release?


The fix is in 9.0.54 and is listed in the changelog.

Mark



Thanks,
Andrew


On Sep 27, 2021, at 8:54 AM, Mark Thomas  wrote:

On 27/09/2021 15:55, Mark Thomas wrote:

On 27/09/2021 09:08, Goldengate liu wrote:
Hi Mark,

I’m uploading some test files

Thanks for the test case. I'm looking at this now.


Bug found and fixed.

One thing to note is that with chunked encoding it is possible for you to see 
isReady() return true only for the subsequent read to return 0 bytes. This 
happens when just (or only part of) the chunked header is available.

The sample code you provided handled this correctly.

The fix will be in the October release round. The release process for that 
should hopefully start later today.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Understanding websocket support in Tomcat

2021-10-06 Thread Mark Thomas

On 06/10/2021 11:02, Deshmukh, Kedar wrote:

Hi,

I would like to understand,

How many concurrent websocket connections are allowed in tomcat ?


As many as your hardware / OS will support.


Is there any limit ?


maxConnections on the Connector. Defaults to 8192. Use -1 for unlimited.


Are connector worker-threads consumed for any websocket connect ?


Yes. Connections are maintained without using a thread. Processing an 
incoming message uses a thread until the message (or partial message) 
processing is complete.



If not, then, is there any special configuration available for websockets ?


No.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: The import javax.servlet cannot be resolved

2021-10-05 Thread Mark Thomas

On 05/10/2021 04:08, Dick Hildreth wrote:

Tomcat 9.0.53
Windows Server 2019 Standard version 1809
OpenJDK  jdk-11.0.8.10-hotspot

I have a JSP/JavaBean webapp.  I deployed all of the class files into the
webapp's classes subdirectory (no WAR file) and the external JAR files are
in the webapp's lib directory.  Of course, the JSPs are in the
webapp's context root.

I try running the webapp from localhost:8080/[appname]/LoginPage.jsp and I
get the subject error with the stacktrace pointing at my JavaBean line:

import javax.servlet.http.HttpServletRequest

I've tried copying *servlet-api.jar* from the tomcat lib directory into my
webapp\lib directory


Never do that.


and I've tried putting *javaee-api-8.0.jar* into my
webapp\lib directory and neither helped.


Nor that.

Create the simplest possible web application that re-creates this issue 
(single JSP and single bean). If creating that doesn't help you figure 
out what is wrong, put the source somewhere where we can access it (e.g. 
Github) and we can take a look.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Specifying a Custom Authenticator Class

2021-10-05 Thread Mark Thomas

On 05/10/2021 03:40, Jerry Malcolm wrote:
An earlier post suggested I just implement a CredentialHandler, which 
would be great.  But it looked like the credential handler is given 
"id/pw" extracted from the base64.  Or will it actually return whatever 
it finds in the base64 token?  "A:B:C:D:E:F" instead of "id:pw"?


CredentialHandler only gets passed the password.

I 
realize that renaming the header prefix to "Malcolm" or whatever would 
be  more architecturally pure.  But how much more code is involved to 
get the same result if my authorization header prefix is now "Malcolm" 
instead of "Basic"?


Probably not that much. Looking at Tomcat's BasicAuthenticator you'd 
have to override most of it to do this anyway so I'd probably copy it as 
the starting point and then edit it.


Mark



Just exploring my options.

Thanks for the discussion.  It's helping me a lot.

Jerry


On 10/4/2021 8:49 AM, Christopher Schultz wrote:

Michael,

On 10/3/21 11:58, Michael Osipov wrote:

Am 2021-10-02 um 02:48 schrieb Jerry Malcolm:
I need to write a custom BasicAuthenticator class to decode a 
specialized encoding of the authToken.  I have been scouring google 
for info.  I found one post where the answer included the statement:


This would clearly violate Basic auth scheme and the according RFC. I 
highly recommend against. Don't abuse Basic. Create your own 
scheme/header and solve your problem with it.


This is a very good point.

Instead of:

Authorization: Basic [base64stuff]

Using "Bearer" might be a better choice, though that is also covered 
by a specific RFC and might be confusing to overload that token 
("Bearer") for another purpose.


You could just do:

Authorization: Malcolms [token]

If you are going to write a custom authenticator, anyway. You'll need 
to have a custom client, of course, but you will already have that 
kind of thing because no standard HTTP client would format your 
authentication tokens in this way.


Another dumb question: why use your own custom stuff instead of the 
standard(s)?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.0.12 available

2021-10-04 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.0.12.

This release is targeted at Jakarta EE 9.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

The notable changes compared to 10.0.11 include:

- Further robustness improvements to HTTP/2 flow control window
  management

- Improvements to the DataSourceUserDatabase

- Fix an issue that caused some Servlet non-blocking API reads of the
  HTTP request body to incorrectly use blocking IO.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.0-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.1.0-M6 (alpha) available

2021-10-04 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.1.0-M6 (alpha).

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10.1.0-M6 is a milestone release of the 10.1.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 10.1.x so that they may provide feedback. The notable 
changes compared to 10.1.0-M5 include:


- Servlet API updates for Servlet 6 including removal of all deprecated
  code, updated schemas and a new API for connection and request IDs.

- EL API updates for EL 5.0 including deprecation of the use of
  FeatureDescriptor, improvements to BeanELResolver and the addition of
  MethodReference

- Further robustness improvements to HTTP/2 flow control window
  management

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.1-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Specifying a Custom Authenticator Class

2021-10-02 Thread Mark Thomas

On 02/10/2021 01:48, Jerry Malcolm wrote:
I need to write a custom BasicAuthenticator class to decode a 
specialized encoding of the authToken.  I have been scouring google for 
info.  I found one post where the answer included the statement:


"Extending from AuthenticatorBase is a great idea, and you can avoid 
Tomcat's standard authenticator by configuring your authenticator as a 
in your application's META-INF/context.xml file."


That is  precisely what I want to do. But I cannot find any 
documentation on how to configure a different authenticator class in a 
context.xml file.  I'm sure I'm just missing it, or I'm using totally 
incorrect words in the googe searches to find it.


Can someone please point me to the documentation for this?


http://tomcat.apache.org/tomcat-10.0-doc/config/valve.html#Authentication

Your Valve needs to implement Authenticator (which it will if you extend 
AuthenticatorBase) so Tomcat knows it is providing authenticator rather 
than just a regular Valve.


You'll end up with something like:


  


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0.52 http2 flow control issues

2021-10-01 Thread Mark Thomas

On 20/09/2021 07:28, Mark Thomas wrote:

On 10/09/2021 11:42, Mark Thomas wrote:

Hi Erik,

Thanks for the report. I'm looking at this now.

I'm testing with a simple index page that references 3 largish images 
(~6MB each).


I've found an issue with HTTP/2, sendfile and StackOverflowExcpetion 
that I have a local fix for.


With that fix in place, I can see a flow control issue. Somehow, a 
stream is getting a larger allocation from the connection control 
window than the stream control window. That leads to some internal 
values having unexpected (negative) values and things quickly escalate 
to the connection closing abruptly from there. I'm currently looking 
into how this happens.


I'm not sure if I am seeing a different issue to you or just a 
different symptom of the same issue. I'll keep the thread updated with 
progress.


I found the root cause - there were further concurrency issues in the 
connection flow control window management. I've refactored the code to 
simplify the approach and (hopefully) make it more robust. I am no 
longer able to recreate the issue I was seeing.


The fixes will be in the October release round. If anyone would like to 
test this sooner than that, you can build from source or I can make a 
test build available on request.


The 9.0.54 release vote is in progress. If you'd like to test this, 
details of where to get the files are on 9.0.54 VOTE thread on the dev 
list. (Note: Only if the VOTE passes is this an official release. Until 
then it is made available for testing purposes only). There will be 
announcement on this list if the VOTE passes and the release becomes 
official. If all goes well, that should be early next week.


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat presentations on ApacheCon 2021

2021-09-27 Thread Mark Thomas

On 27/09/2021 20:27, Усманов Азат Анварович wrote:


Hi everyone! Does anybody know where/when to find the  video/audio/slides (if 
any) from the last weeks's tomcat track on ApacheCon 2021?Because I completely 
missed it last week.
  I'm assuming all of these would be added to tomcat presentations page 
http://tomcat.apache.org/presentations.html or  
https://www.youtube.com/c/ApacheTomcatOfficial/videos at some point in time.I'm in no 
rush , just wanna make sure  I haven't missed anything which could be useful on a  daily 
basis. Especialy considering the fact that  I've had a few aha ("I wish I'd knew 
this earlier") type  moments after watching  tomcat presentations before.


The conference team has a few hundred videos to process. They should 
start to appear over the next few weeks. Mine was pretty much the same 
as the one from ApacheCon Asia which is already available.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: AW: JASPIC AuthConfigProvider packaged with the web application not found

2021-09-27 Thread Mark Thomas

On 23/09/2021 07:03, Keil, Matthias (ORISA Software GmbH) wrote:

Hi Bernd,

Yes, I would like to define my Server Auth module in the jaspic-providers.xml 
and then provide the class with the web application.


Sorry, that isn't going to be supported. You either need to provide 
everything at the container level or everything at the web application 
level.


The main concern with configuration at the container level and 
implementation at the web application level is that web applications can 
be undeployed at which point everything breaks.


I'll update the documentation to make these two options clearer.

Mark





Mit vielen Grüßen

Matthias Keil

-Ursprüngliche Nachricht-
Von: Bernd Schatz 
Gesendet: Dienstag, 21. September 2021 23:25
An: users@tomcat.apache.org
Betreff: Re: JASPIC AuthConfigProvider packaged with the web application not 
found

Hi,


Am 19.09.21 um 19:48 schrieb Keil, Matthias (ORISA Software GmbH):

Hello everyone and thanks for the hints.
They also work as expected and I can package the provider in the web 
application .

Nevertheless, the Configuration Reference 
(https://tomcat.apache.org/tomcat-9.0-doc/config/jaspic.html) suggests that you 
define your own provider in jaspic-providers.xml and Tomcat will then find it.
I am really only interested in a separate server auth module (SAM). Since I saw 
no way in the documentation to pack this into the web application. That's why I 
tried the way through the provider.



You want to define the class in the  jaspic-providers.xml but package the 
provider implementation(s) in the application(s) ?



As I said, your suggestions work, but there are also a number of additional 
classes needed to provide the actual SAM.
Thank you again


If you dont need the whole flexibility of JASPI you can also do something like 
this:


public class MyAuthProvider implements AuthConfigProvider,
ServerAuthConfig, ServerAuthModule, ServerAuthContext





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding

2021-09-27 Thread Mark Thomas

On 27/09/2021 15:55, Mark Thomas wrote:

On 27/09/2021 09:08, Goldengate liu wrote:

Hi Mark,

   I’m uploading some test files


Thanks for the test case. I'm looking at this now.


Bug found and fixed.

One thing to note is that with chunked encoding it is possible for you 
to see isReady() return true only for the subsequent read to return 0 
bytes. This happens when just (or only part of) the chunked header is 
available.


The sample code you provided handled this correctly.

The fix will be in the October release round. The release process for 
that should hopefully start later today.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0 async read becomes blocking with chunked transfer-encoding

2021-09-27 Thread Mark Thomas

On 27/09/2021 09:08, Goldengate liu wrote:

Hi Mark,

   I’m uploading some test files


Thanks for the test case. I'm looking at this now.

Mark



   Below are the steps:
1. compile GetPayloadServlet.java and put it to 
webapps/test/WEB-INF/classes/test/

2. put web.xml to webapps/test/WEB-INF/
3. start tomcat (9.0.19), I tried with 9.0.53, I got the same result
4. use HttpPostTest(I’m using Apache httpclient-4.5.1.jar, 
httpcore-4.4.4.jar), the test is sending data with chunked 
transfer-encoding, one step here
4.1. put a debug point before streamWrite at flushBuffer method in 
org.apache.http.impl.io.SessionOutputBufferImpl, the purpose is to pause 
flushing to the socket

*private* *void* flushBuffer() *throws* IOException {
*final* *int* len = *this*.buffer.length();
*if* (len > 0) {
streamWrite(*this*.buffer.buffer(), 0, len);
*this*.buffer.clear();
*this*.metrics.incrementBytesTransferred(len);
}

}

      5. put a debug point on GetPayloadServlet

@Override
public void onDataAvailable() throws IOException {
byte buffer[] = new byte[BUFFER_SIZE];
while(inputStream.isReady()) {// when isReady returns true, 
inputStream.read(buffer) is actually blocked with chunked encoding

int read = inputStream.read(buffer);
if (read < 0) {
break;
}
bos.write(buffer, 0, read);
}
}

6. we can see once SessionOutputBufferImpl.flushBuffer flushes the 
data,inputStream.read(buffer) inside GetPayloadServlet will be 
un-blocked with read


   Or am I doing something wrong? this is a basic use case.

   Thanks,
   Andrew



On Sep 22, 2021, at 1:14 AM, Mark Thomas <mailto:ma...@apache.org>> wrote:


On 22/09/2021 08:22, Goldengate liu wrote:

Hi Chris,
Servlet 3.1 spec defines that ServletInputStream can be used to 
read as non-blocking way as long as there is data ready locally by 
calling isReady method and check the ready condition before calling 
read, and read should throw IllegalStateException if called by 
caller when data is not ready


The above only applies if the servlet is in async mode. Is it?

correct, when it’s running as async mode


OK. You are going to need to provide the simplest complete example 
that demonstrates this then. Something like the source for a single 
Servlet and a curl command to send a request to it.


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org 
<mailto:users-unsubscr...@tomcat.apache.org>
For additional commands, e-mail: users-h...@tomcat.apache.org 
<mailto:users-h...@tomcat.apache.org>







-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Possible UpgradeInfo memory leak

2021-09-24 Thread Mark Thomas

On 23/09/2021 09:36, Harri Pesonen wrote:

Hello, while looking at Tomcat 8.5.61 heap dump in VisualVM, in Dominators by 
Retained Size, two biggest ones are:

org.apache.tomcat.util.net.NioEndpoint#1  12 382 781 B (13,7%)
org.apache.coyote.http11.upgrade.UpgradeGroupInfo#1  7 066 212 B (7,8%)

I am wondering about UpgradeGroupInfo, because it has very large array of 
UpgradeInfos:

oname = javax.management.ObjectName#1240 : 
Catalina:Upgrade=websocket,name="https-jsse-nio-10.8.35.86-8443",type=GlobalRequestProcessor
 31 B (0%)  363 B (0%)
upgradeInfos = java.util.ArrayList#10079 : 146 098 elements 
20 B (0%)  7 066 144 B (7,8%)
elementData = java.lang.Object[]#22702 : 160 065 items   640 276 B (0,7%)   
 7 066 124 B (7,8%)
[0] = org.apache.coyote.http11.upgrade.UpgradeInfo#144 B (0%)  44 B 
(0%)

Single UpgradeInfo is very small, but there are 146 098 of them.
org.apache.coyote.http11.upgrade.UpgradeInfo   146 098 (12,8%)

I am not sure what this UpgradeInfo is, it looks like some statistics about 
upgraded connection, in this case from http to websocket.
To me it looks like these UpgradeInfos are not removed when connection is 
closed.
Any comments?


Thanks for the report. We are looking into it.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Custom error page

2021-09-24 Thread Mark Thomas

On 24/09/2021 11:56, Jan Pernica wrote:

Hi

how can I easly create error page for the whole server? Curretly if I 
add to conf/web.xml

     
     500
     /error/error.html
     
     
     404
     /error/error.html
     
And put into webapps/ROOT/error/error.html page it works only for ROOT 
context. Other contexts shows empty page. I have to propagate error page 
into all contexts.


How to do it in one step?


http://tomcat.apache.org/tomcat-10.0-doc/config/valve.html#Error_Report_Valve/Attributes

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: mirrors are broken?

2021-09-22 Thread Mark Thomas

On 22/09/2021 10:00, jean-frederic clere wrote:

Hi,

https://tomcat.apache.org/download-90.cgi gives me:
+++
Error!

/var/www/dyn/closer.lua:322: attempt to index local 'cdn_uri_check' (a 
nil value)

+++

Have we break something with the mirror logic? Or is my favorite mirror 
broken?


https://bz.apache.org/bugzilla/show_bug.cgi?id=65587

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0 async read becomes blocking

2021-09-22 Thread Mark Thomas

On 22/09/2021 08:22, Goldengate liu wrote:

Hi Chris,
Servlet 3.1 spec defines that ServletInputStream can be used to read as 
non-blocking way as long as there is data ready locally by calling isReady 
method and check the ready condition before calling read, and read should throw 
IllegalStateException if called by caller when data is not ready


The above only applies if the servlet is in async mode. Is it?


correct, when it’s running as async mode


OK. You are going to need to provide the simplest complete example that 
demonstrates this then. Something like the source for a single Servlet 
and a curl command to send a request to it.


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0 async read becomes blocking

2021-09-22 Thread Mark Thomas

On 21/09/2021 23:01, Javateck wrote:

Hi Chris,

Servlet 3.1 spec defines that ServletInputStream can be used to read as 
non-blocking way as long as there is data ready locally by calling 
isReady method and check the ready condition before calling read, and 
read should throw IllegalStateException if called by caller when data is 
not ready


The above only applies if the servlet is in async mode. Is it?

Mark




Agree that InputStream read api is blocking by nature, but if the data 
is already there in local buffer, then it’s not, it’s just exposing as 
ServletInputStream


https://javaee.github.io/servlet-spec/downloads/servlet-3.1/Final/servlet-3_1-final.pdf 






On Sep 21, 2021, at 2:26 PM, Christopher Schultz 
 wrote:


Andrew,

On 9/21/21 13:54, Javateck wrote:

Hi,
With NIO connector with Servlet 3.1 support, I’m registering with a 
ReadListener, while it got the first read signal from tomcat 
container (I tried 9.0.19 and 9.0.53), the read call is blocked after 
isReady returns true

  if (ServletInputStream.isReady()) {
   ServletInputStream.read(buffer);  // this becomes blocking
  }
I tried with jetty, it’s working fine
When I did the test, I was holding the sending packet from client side
Not sure whether anyone has tried this


InputStream is always blocking.

Are you trying to use async? That's not the way to use async...

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0.52 http2 flow control issues

2021-09-20 Thread Mark Thomas

On 10/09/2021 11:42, Mark Thomas wrote:

Hi Erik,

Thanks for the report. I'm looking at this now.

I'm testing with a simple index page that references 3 largish images 
(~6MB each).


I've found an issue with HTTP/2, sendfile and StackOverflowExcpetion 
that I have a local fix for.


With that fix in place, I can see a flow control issue. Somehow, a 
stream is getting a larger allocation from the connection control window 
than the stream control window. That leads to some internal values 
having unexpected (negative) values and things quickly escalate to the 
connection closing abruptly from there. I'm currently looking into how 
this happens.


I'm not sure if I am seeing a different issue to you or just a different 
symptom of the same issue. I'll keep the thread updated with progress.


I found the root cause - there were further concurrency issues in the 
connection flow control window management. I've refactored the code to 
simplify the approach and (hopefully) make it more robust. I am no 
longer able to recreate the issue I was seeing.


The fixes will be in the October release round. If anyone would like to 
test this sooner than that, you can build from source or I can make a 
test build available on request.


Mark




Mark


On 10/09/2021 11:02, Erik Nilsson wrote:

Hi,

We still have a problem with the http2 flow control and RST_STREAMS 
when running nghttp -vnsay 

I have attached a screenshot and Tomcat debug logging.

/Erik



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[SECURITY] CVE-2021-41079 Apache Tomcat DoS

2021-09-15 Thread Mark Thomas

CVE-2021-41079 Denial of Service

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
Apache Tomcat 10.0.0-M1 to 10.0.2
Apache Tomcat 9.0.0-M1 to 9.0.43
Apache Tomcat 8.5.0 to 8.5.63

Description:
When Tomcat was configured to use NIO+OpenSSL or NIO2+OpenSSL for TLS, a 
specially crafted packet could be used to trigger an infinite loop 
resulting in a denial of service.


Mitigation:
Users of the affected versions should apply one of the following
mitigations:
- Upgrade to Apache Tomcat 10.0.4 or later
- Upgrade to Apache Tomcat 9.0.44 or later
- Upgrade to Apache Tomcat 8.5.64 or later

Note: This issue was fixed in Apache Tomcat 10.0.3 but the release vote 
for the 10.0.3 release candidate did not pass. Therefore, although users 
must download 10.0.4 to obtain a version that includes a fix for this 
issue, version 10.0.3 is not included in the list of affected versions.


Credit:
The Apache Tomcat Security Team would like to thank:
- Thomas Wozenilek for originally reporting this issue
- David Frankson of Infinite Campus for providing a test case that
  reproduced the issue.

History:
2021-09-15 Original advisory

References:
[1] https://tomcat.apache.org/security-10.html
[2] https://tomcat.apache.org/security-9.html
[3] https://tomcat.apache.org/security-8.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat8.0.53 & Java related issues

2021-09-14 Thread Mark Thomas

On 14/09/2021 03:59, zhuyix...@orientalmind.com wrote:

Dear Sir or Madam:
 Howdy.I'm a Java developer.I am learning related knowledge of 
Tomcat.Version for 8.0.53.
 At present,I have some problems and I hope I can get help.
 Currently I'm using the 
org.apache.catalina.util.RequestUtil.parseParameters().
 I found this method deprecated after version 8.5.xx.I can't find in the 
notes or documentation why it was abandoned


It was no longer used.


and what is the recommended alternative.


org.apache.tomcat.util.http.Parameters

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.0.11 available

2021-09-13 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.0.11.

This release is targeted at Jakarta EE 9.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

The notable changes compared to 10.0.10 include:

- Add a UserDatabase implementation as a superset of the DataSourceRealm
  functionality.

- Update the internal fork of Apache Commons DBCP to 2.9.0 and Apache
  Commons Pool to 2.11.1

- Update the packaged version of the Tomcat Native Library to 1.2.31 to
  pick up Windows binaries built with OpenSSL 1.1.1l.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.0-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.1.0-M5 (alpha) available

2021-09-13 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.1.0-M5 (alpha).

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10.1.0-M5 is a milestone release of the 10.1.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 10.1.x so that they may provide feedback. The notable 
changes compared to 10.1.0-M4 include:


- Remove the deprecated APR/Native connector which includes the HTTP APR
  and the AJP APR connector. Also remove the Java interfaces to the
  APR/Native library that are not used by the OpenSSL integration for
  the NIO and NIO2 connectors.

- Add a UserDatabase implementation as a superset of the DataSourceRealm
  functionality.

- Update the internal fork of Apache Commons DBCP to 2.9.0 and Apache
  Commons Pool to 2.11.1

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.1-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Question about serving a 404

2021-09-10 Thread Mark Thomas

On 10/09/2021 16:44, James H. H. Lampert wrote:

Our Tomcat team has been struggling with this issue for a few days:

If a request comes in for https://foo.com/bar.html, which doesn't exist, 
then a 404 is returned, and we see a standard Tomcat 404 page.


But if a request comes in for https://foo.com/bar.jsp, which also 
doesn't exist, then our webapp takes control, and returns a 200 with a 
redirect to a "you are not signed on" page.


In the most recent attempt to correct this, they now return a 404 page 
of their own design for both of the above scenarios. Unfortunately, if 
the webapp context we're trying to reach is installed as something other 
than ROOT (i.e., if we call it "baz," then https://foo.com/baz), then 
even a correct URL still returns a 404 page.


Seems to me that there's something wrong with this picture. Seems to me 
that a request for a nonexistent "bar.jsp" should behave the same as one 
for a nonexistent "bar.html."


Is there something I can pass along to the Tomcat team?


Sounds like whatever is responding *.jsp requests is a little over 
eager. Depending on exactly what that is (Tomcat's JSP Servlet, custom 
Servlet, Filter) there are different things that might be able to help.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat 9.0.52 http2 flow control issues

2021-09-10 Thread Mark Thomas

Hi Erik,

Thanks for the report. I'm looking at this now.

I'm testing with a simple index page that references 3 largish images 
(~6MB each).


I've found an issue with HTTP/2, sendfile and StackOverflowExcpetion 
that I have a local fix for.


With that fix in place, I can see a flow control issue. Somehow, a 
stream is getting a larger allocation from the connection control window 
than the stream control window. That leads to some internal values 
having unexpected (negative) values and things quickly escalate to the 
connection closing abruptly from there. I'm currently looking into how 
this happens.


I'm not sure if I am seeing a different issue to you or just a different 
symptom of the same issue. I'll keep the thread updated with progress.


Mark


On 10/09/2021 11:02, Erik Nilsson wrote:

Hi,

We still have a problem with the http2 flow control and RST_STREAMS when 
running nghttp -vnsay 

I have attached a screenshot and Tomcat debug logging.

/Erik



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat hangs

2021-09-09 Thread Mark Thomas

On 09/09/2021 11:50, Mehrdad Taagholi wrote:

HiI use apache tomcat 8.0.32 and oracle-jdk-8u66 and redhat 6.After working 
with the system for a few hours and the load on the system increases, suddenly 
the tomcat hangs and no logs are printed and it is not possible to connect via 
jvisualvm and I can not get any dump and I have to reload Tomcat.I have 
increased maxthreads and use the HttpProtocol protocol.Please suggest a way to 
fix the my tomact.


Take three thread dumps ~5 seconds apart when it hangs. Post the 
complete thread dumps somewhere we can see them.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Http TRACE method headers in response body

2021-09-09 Thread Mark Thomas

On 08/09/2021 20:50, Christopher Schultz wrote:

Mark,

On 9/8/21 11:28, Mark Thomas wrote:

On 08/09/2021 16:15, Gilles Robert wrote:

My issue is that even though TRACE is disabled, we see the "malicious"
header in the response.


You need to talk to the Spring folks then. Default Tomcat behaviour is 
to return a 405 with an error message in the response. I've just 
doubled checked this with telnet and 9.0.x.



As an aside, the idea that any TRACE response is a security issue with 
the server, whether it contains a "malicious" header or not is 
nonsense. The only thing a user agent anywhere should be doing with a 
TRACE response is displaying it as plain text. If a user agent does 
something else with the response, and especially if it does something 
reckless like treating it is HTML, then than is a security issue with 
the user agent, not the server.




Super duper vuln:

$ curl -X TRACE --header '' myurl | bash

RCE every time, bro.


:)

You would have got bonus points if the first character was '#' rather 
than '$'.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Http TRACE method headers in response body

2021-09-08 Thread Mark Thomas

On 08/09/2021 16:15, Gilles Robert wrote:

My issue is that even though TRACE is disabled, we see the "malicious"
header in the response.


You need to talk to the Spring folks then. Default Tomcat behaviour is 
to return a 405 with an error message in the response. I've just doubled 
checked this with telnet and 9.0.x.



As an aside, the idea that any TRACE response is a security issue with 
the server, whether it contains a "malicious" header or not is nonsense. 
The only thing a user agent anywhere should be doing with a TRACE 
response is displaying it as plain text. If a user agent does something 
else with the response, and especially if it does something reckless 
like treating it is HTML, then than is a security issue with the user 
agent, not the server.



Mark




On Wed, 8 Sept 2021 at 17:01, Mark Thomas  wrote:


On 08/09/2021 14:14, Gilles Robert wrote:

Hi,

Using Spring boot (2.5.4) with Tomcat (9.0.52), the HTTP TRACE method
is disabled by default and returns a 405 method not allowed, which is
what I expect security-wise. My issue is that if one gives a malicious
header:

header: malicious: alert('malicious call');

it's given back in the response:

TRACE /xyz/error HTTP/1.1
malicious: alert('malicious call');
user-agent: PostmanRuntime/7.22.0
accept: */*
host: localhost:8080
accept-encoding: gzip, deflate, br
content-length: 0
connection: keep-alive

This is conform to the RFC 2616 which states:

"If the request is valid, the response SHOULD contain the entire
request message in the entity-body, with a Content-Type of
"message/http"."


Do you mean that you are seeing the TRACE response even when TRACE is
disabled?

Or is the issue that if TRACE is enabled, then you see the "malicious"
header in the response?

Mark




My penetration test team is complaining about it.

How can I remove any HTML entities from the TRACE response, without
having to enable it, cleaning the tags and returning the 405 myself?

Thanks!

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Http TRACE method headers in response body

2021-09-08 Thread Mark Thomas

On 08/09/2021 14:14, Gilles Robert wrote:

Hi,

Using Spring boot (2.5.4) with Tomcat (9.0.52), the HTTP TRACE method
is disabled by default and returns a 405 method not allowed, which is
what I expect security-wise. My issue is that if one gives a malicious
header:

header: malicious: alert('malicious call');

it's given back in the response:

TRACE /xyz/error HTTP/1.1
malicious: alert('malicious call');
user-agent: PostmanRuntime/7.22.0
accept: */*
host: localhost:8080
accept-encoding: gzip, deflate, br
content-length: 0
connection: keep-alive

This is conform to the RFC 2616 which states:

"If the request is valid, the response SHOULD contain the entire
request message in the entity-body, with a Content-Type of
"message/http"."


Do you mean that you are seeing the TRACE response even when TRACE is 
disabled?


Or is the issue that if TRACE is enabled, then you see the "malicious" 
header in the response?


Mark




My penetration test team is complaining about it.

How can I remove any HTML entities from the TRACE response, without
having to enable it, cleaning the tags and returning the 405 myself?

Thanks!

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: JNDIRealm does not retry on read timeouts or closed connections

2021-09-06 Thread Mark Thomas

On 06/09/2021 09:52, Osipov, Michael (LDA IT PLM) wrote:



My question is: Mark, you have direct access to JBS, would you be 
willing to file this issue directly or do you want me to file through 
bugreport.java.com first and when it arrives in JBS you could drop a 
comment that this also affects Tomcat?


Best if you create it and I comment.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: AW: AW: Orphaned thread by JNDIRealm / clearReferencesThreads reports memory leak

2021-09-06 Thread Mark Thomas

On 06/09/2021 08:47, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello Mark,

thank you for taking a look at it and confirming the behaviour.
I will file a bug as suggested.


Thanks, I see it.

I have added my own comment. I'll give others some time to comment and 
aim to fix this for the October release round.


Mark




Thank you and have a good start into the new week!
Thomas

-Ursprüngliche Nachricht-
Von: Mark Thomas 
Gesendet: Montag, 6. September 2021 09:36
An: users@tomcat.apache.org
Betreff: Re: AW: Orphaned thread by JNDIRealm / clearReferencesThreads reports 
memory leak

Hi Thomas.

Got it.

I think it would be best if you opened a Bugzilla entry for this.

One view is that the JRE should not be doing this - or at least doing it in a 
way that doesn't run the risk of memory leaks in a Java EE / Jakarta EE 
environment. We have raised JRE bugs for issues like this in the past and they 
have been fixed.

The counter view is that this thread is not a one-off and is not short-lived 
(the typical cases for JRE bugs that have been fixed) and Tomcat has already 
acknowledged - with the useContextClassLoader flag - that it needs to be taking 
action to ensure that any threads are created with the expected class loader.

If the consensus is that Tomcat needs to handle this then we'd need to ensure 
that every version of authenticate() also set the class loader if required. 
We'd probably want to refactor the existing handling to make sure we don't try 
setting the class loader twice.

Whatever changes we make we need to be able to reference them back to the 
discussion of why we made them and a Bugzilla issue is how we do that.

Thanks,

Mark


On 05/09/2021 22:03, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello Mark,

thanks for your reply.
I already tried that option. But this flag only controls, how the 
InitialDirContext is created.
The worker thread within com.sun.jndi.ldap.Connection is created with that 
corresponding classloader the first time.

The problem is when the worker-thread within the com.sun.jndi.ldap.Connection 
dies and is re-established again.
In that case, the flag useContextClassLoader is not considered any more because 
the InitialDirContext is already instantiated.

The call stack when the InitialDirContext is already instantiated but the 
connection gets re-established looks like:
JNDIRealm.authenticate --> JNDIRealm.getUserBySearch  -->
LdapCtx.dosearch --> LdapCtx.ensureOpen --> LdapCtx.connect  -->
LdapClient.getInstance --> Connection.

In this call chain, the flag useContextClassLoader is not used any more as the 
InitialDirContext already exists.
The get() method just provides the existing JNDIConnection without switching 
any classloader.
Now however, the context classloader of the application is used, independent of the 
setting "useContextClassLoader".
Therefore, the second time when the worker thread gets created, it inherits the 
classloader of the application and is reported as leaking during undeployment.

Greetings, Thomas




Von: Mark Thomas 
Gesendet: Sonntag, 5. September 2021 11:55
An: users@tomcat.apache.org
Betreff: Re: Orphaned thread by JNDIRealm / clearReferencesThreads
reports memory leak
  
Thomas,


Try setting:

useContextClassLoader="false"

for the JNDIRealm.

Mark


On 02/09/2021 08:33, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello,

we are using the org.apache.catalina.realm.JNDIRealm for authentication of 
users against our windows AD.
When undeploying the application, we see the following warning in our logs:

WARNING [Catalina-utility-1] org.apache.catalina.loader.Webapp
ClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to 
have started a thread named [Thread-106] but has failed to stop it. This is 
very likely to create a memory leak. Stack trace of thread:
     java.base@11.0.3/java.net.SocketInputStream.socketRead0(Native
Method)
 
java.base@11.0.3/java.net.SocketInputStream.socketRead(SocketInputStr

eam.java:115)
 
java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.ja

va:168)
 
java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.ja

va:140)
 
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.read(SSLSocket

InputRecord.java:448)
 
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.bytesInComplet

ePacket(SSLSocketInputRecord.java:68)
 
java.base@11.0.3/sun.security.ssl.SSLSocketImpl.readApplicationRecord

(SSLSocketImpl.java:1104)
 
java.base@11.0.3/sun.security.ssl.SSLSocketImpl$AppInputStream.read(S

SLSocketImpl.java:823)
 
java.base@11.0.3/java.io.BufferedInputStream.fill(BufferedInputStream

.java:252)
 
java.base@11.0.3/java.io.BufferedInputStream.read1(BufferedInputStrea

m.java:292)
 
java.base@11.0.3/java.io.BufferedInputStream.read(BufferedInputStream

.java:351)
 
java.naming@11.0.3/com.sun.jndi.ldap.Connection.run(Connection.java:8

32)
     java.base@11.0.

Re: Exception in Log files

2021-09-06 Thread Mark Thomas

On 06/09/2021 08:16, Mohan T wrote:

Hi,

We could see the below exception in log files .

java.io.FileNotFoundException:  apache-tomcat-8.5.35/lib/commons-cli.jar (No 
such file or directory)
The file is not there in that location.

How to get rid of this exception


With the information you have provided so far? No idea.

A good starting point would be providing the the full stack trace that 
goes along with that Exception.


Mark



DISCLAIMER: This communication contains information which is confidential and the 
copyright of Ramco Systems Ltd, its subsidiaries or a third party ("Ramco"). 
This email may also contain legally privileged information. Confidentiality and legal 
privilege attached to this communication are not waived or lost by reason of mistaken 
delivery to you.This email is intended to be read or used by the addressee only. If you 
are not the intended recipient, any use, distribution, disclosure or copying of this 
email is strictly prohibited without the express written approval of Ramco. Please delete 
and destroy all copies and email Ramco at le...@ramco.com immediately. Any views 
expressed in this communication are those of the individual sender, except where the 
sender specifically states them to be the views of Ramco. Except as required by law, 
Ramco does not represent, warrant and/or guarantee that the integrity of this 
communication has been maintained nor that the communication is free of errors, virus, 
interception or interference. If you do not wish to receive such communications, please 
forward this communication to market...@ramco.com and express your wish not to receive 
such communications henceforth.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: AW: Orphaned thread by JNDIRealm / clearReferencesThreads reports memory leak

2021-09-06 Thread Mark Thomas

Hi Thomas.

Got it.

I think it would be best if you opened a Bugzilla entry for this.

One view is that the JRE should not be doing this - or at least doing it 
in a way that doesn't run the risk of memory leaks in a Java EE / 
Jakarta EE environment. We have raised JRE bugs for issues like this in 
the past and they have been fixed.


The counter view is that this thread is not a one-off and is not 
short-lived (the typical cases for JRE bugs that have been fixed) and 
Tomcat has already acknowledged - with the useContextClassLoader flag - 
that it needs to be taking action to ensure that any threads are created 
with the expected class loader.


If the consensus is that Tomcat needs to handle this then we'd need to 
ensure that every version of authenticate() also set the class loader if 
required. We'd probably want to refactor the existing handling to make 
sure we don't try setting the class loader twice.


Whatever changes we make we need to be able to reference them back to 
the discussion of why we made them and a Bugzilla issue is how we do that.


Thanks,

Mark


On 05/09/2021 22:03, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello Mark,

thanks for your reply.
I already tried that option. But this flag only controls, how the 
InitialDirContext is created.
The worker thread within com.sun.jndi.ldap.Connection is created with that 
corresponding classloader the first time.

The problem is when the worker-thread within the com.sun.jndi.ldap.Connection 
dies and is re-established again.
In that case, the flag useContextClassLoader is not considered any more because 
the InitialDirContext is already instantiated.

The call stack when the InitialDirContext is already instantiated but the 
connection gets re-established looks like:
JNDIRealm.authenticate --> JNDIRealm.getUserBySearch  --> LdapCtx.dosearch --> 
LdapCtx.ensureOpen --> LdapCtx.connect  --> LdapClient.getInstance --> Connection.

In this call chain, the flag useContextClassLoader is not used any more as the 
InitialDirContext already exists.
The get() method just provides the existing JNDIConnection without switching 
any classloader.
Now however, the context classloader of the application is used, independent of the 
setting "useContextClassLoader".
Therefore, the second time when the worker thread gets created, it inherits the 
classloader of the application and is reported as leaking during undeployment.

Greetings, Thomas




Von: Mark Thomas 
Gesendet: Sonntag, 5. September 2021 11:55
An: users@tomcat.apache.org
Betreff: Re: Orphaned thread by JNDIRealm / clearReferencesThreads reports 
memory leak
 
Thomas,


Try setting:

useContextClassLoader="false"

for the JNDIRealm.

Mark


On 02/09/2021 08:33, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello,

we are using the org.apache.catalina.realm.JNDIRealm for authentication of 
users against our windows AD.
When undeploying the application, we see the following warning in our logs:

WARNING [Catalina-utility-1] org.apache.catalina.loader.Webapp
ClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to 
have started a thread named [Thread-106] but has failed to stop it. This is 
very likely to create a memory leak. Stack trace of thread:
    java.base@11.0.3/java.net.SocketInputStream.socketRead0(Native Method)
    
java.base@11.0.3/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
    java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.java:168)
    java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.java:140)
    
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
    
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
    
java.base@11.0.3/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1104)
    
java.base@11.0.3/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:823)
    
java.base@11.0.3/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
    
java.base@11.0.3/java.io.BufferedInputStream.read1(BufferedInputStream.java:292)
    
java.base@11.0.3/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
    java.naming@11.0.3/com.sun.jndi.ldap.Connection.run(Connection.java:832)
    java.base@11.0.3/java.lang.Thread.run(Thread.java:834)

The warning is not always shown but quite often.

Summary of the analysis of the problem:
On tomcat startup, the worker-thread is running under the tomcat classloader. 
But when a reconnect happens, the thread is running with the classloader of the 
web application and gets thus reported.

The details:
Digging into the problem via remote debugging showed the reason how this 
happens:
During startup, Tomcat is initializing the JNDIRealm. The open-method of JNDIRealm is 
switching the classloader between bootstrap-CL and tomcat-lib-CL, depending on the 
attribute "useContextClassLoader

Re: Orphaned thread by JNDIRealm / clearReferencesThreads reports memory leak

2021-09-05 Thread Mark Thomas

Thomas,

Try setting:

useContextClassLoader="false"

for the JNDIRealm.

Mark


On 02/09/2021 08:33, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello,

we are using the org.apache.catalina.realm.JNDIRealm for authentication of 
users against our windows AD.
When undeploying the application, we see the following warning in our logs:

WARNING [Catalina-utility-1] org.apache.catalina.loader.Webapp
ClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to 
have started a thread named [Thread-106] but has failed to stop it. This is 
very likely to create a memory leak. Stack trace of thread:
  java.base@11.0.3/java.net.SocketInputStream.socketRead0(Native Method)
  
java.base@11.0.3/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
  java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.java:168)
  java.base@11.0.3/java.net.SocketInputStream.read(SocketInputStream.java:140)
  
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
  
java.base@11.0.3/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
  
java.base@11.0.3/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1104)
  
java.base@11.0.3/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:823)
  
java.base@11.0.3/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
  
java.base@11.0.3/java.io.BufferedInputStream.read1(BufferedInputStream.java:292)
  
java.base@11.0.3/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
  java.naming@11.0.3/com.sun.jndi.ldap.Connection.run(Connection.java:832)
  java.base@11.0.3/java.lang.Thread.run(Thread.java:834)

The warning is not always shown but quite often.

Summary of the analysis of the problem:
On tomcat startup, the worker-thread is running under the tomcat classloader. 
But when a reconnect happens, the thread is running with the classloader of the 
web application and gets thus reported.

The details:
Digging into the problem via remote debugging showed the reason how this 
happens:
During startup, Tomcat is initializing the JNDIRealm. The open-method of JNDIRealm is 
switching the classloader between bootstrap-CL and tomcat-lib-CL, depending on the 
attribute "useContextClassLoader".
Afterwards the context-Object is created (createDirContext). Within this 
LdapCtx, an LdapClient is used to communicate with the AD-Server.
This LdapClient uses a com.sun.jndi.ldap.Connection for TCP communication. This 
connection opens the reported Worker-Thread.
This can be seen at 
https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.naming/share/classes/com/sun/jndi/ldap/Connection.java
 around line 243 --> worker = Obj.helper.createThread(this);

So far, so good.

Somehow, the com.sun.jndi.ldap.Connection is sometimes closed and the thread 
dies. At least, the thread is not visible any more. Maybe because of a timeout 
on the AD-server side or something else happened.
If a new user accesses the site, the JNDIRealm is authenticating the user.
This triggers the following chain (path is shortened): JNDIRealm.getUserBySearch --> 
LdapCtx.dosearch --> LdapCtx.ensureOpen --> LdapCtx.connect --> LdapClient.getInstance 
--> Connection.
This creates a new com.sun.jndi.ldap.Connection and thus a new thread. But this 
time, the thread is connected to the classloader of the web-application.
On undeployment, the thread is thus reported to be orphaned.

It was tested with Tomcat 9.0.52, Windows 10, OpenJDK 11.0.12_7.

As the authentication is conducted within tomcat, before the application is 
triggered, I am not sure if the problem can be tackled on application side.

Thanks in advance,
Thomas

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat Native 1.2.31 released

2021-09-02 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat Native 1.2.31 stable.

The key features of this release are:
- Windows binaries built using OpenSSL 1.1.1l
- Fix an issue when building with OpenSSl 3.0.0

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/native-doc/miscellaneous/changelog.html

Downloads:
http://tomcat.apache.org/download-native.cgi

The Apache Tomcat Native Library provides portable API for features
not found in contemporary JDK's. It uses Apache Portable Runtime as
operating system abstraction layer and OpenSSL for SSL networking and
allows optimal performance in production environments.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: UserDatabaseRealm and DIGEST

2021-08-24 Thread Mark Thomas

On 24/08/2021 17:28, jonmcalexan...@wellsfargo.com.INVALID wrote:

Ok, so I've been reading thru the documentation on DIGEST but not entirely sure 
I have it right. What is the best practice for DIGEST and what algorithms are 
allowed, such as is sha-256 allowed?


First, a question of clarification.

Do you mean HTTP DIGEST authentication or do you mean storing password 
hashes rather than the actual passwords in the UserDatabaseRealm?


Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: clearReferencesThreads issues warning about 2 threads, spawned by JDK in printing components

2021-08-23 Thread Mark Thomas

On 23/08/2021 08:10, Thomas Hoffmann (Speed4Trade GmbH) wrote:




Is there anything, the application can prevent this?


Yes. Call Thread.setContextClassLoader(ClassLoader) before calling the 
code that creates those threads, passing the common class loader. 
Afterwards, reset the TCCL back to the web application class loader.



Should Tomcat maybe skip the warning if the thread is demonized?


No. The threads have the web app class loader as their TCCL so you have 
a potential memory leak. The warning is correct.



Or maybe these threads should be ignored when checking for orphaned threads?


No, for the same reason.


It was tested with Tomcat 9.0.52, Windows 10, Coretto-JDK 11.0.12_7.


You might want to consider raising a bug against the JDK. It could be 
argued that those threads should be created with a specific class loader 
to avoid memory leaks in container environments.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 8.5.70 available

2021-08-17 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 8.5.70.

Apache Tomcat 8 is an open source software implementation of the Java
Servlet, JavaServer Pages, Java Unified Expression Language, Java
WebSocket and Java Authentication Service Provider Interface for
Containers technologies.

Apache Tomcat 8.5.70 is a bugfix and feature release. The notable 
changes compared to 8.5.69 include:


- Correct a regression in the previous release in the HTTP/2 flow
  control window management along with additional improvements to HTTP/2
  flow control

- Make the CorsFilter simpler to extend

- To avoid unnecessary cache revalidation, do not add an HTTP Expires
  header when setting adding an HTTP header of CacheControl: private

Along with lots of other bug fixes and improvements.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-8.5-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-80.cgi

Migration guides from Apache Tomcat 7.x and 8.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat "JNDI Datasource How-To" documentation & driver managers

2021-08-14 Thread Mark Thomas

On 14/08/2021 01:51, Andrew Tanton wrote:

In the Tomcat "JNDI Datasource How-To" documentation page
,
there is an unusually opinionated section, which discusses the Java service
provider (driver manager) mechanism:


I suspect that was me after spending quite a but of time unpicking 
various issues associated with DriverManager where Tomcat was getting 
blamed. Goes to check the history...


Yep. Here is the bug report:
https://bz.apache.org/bugzilla/show_bug.cgi?id=52025
which triggered this doc update:
https://svn.apache.org/viewvc?view=revision=1184919


"*However, the implementation is fundamentally broken in all Java versions
for a servlet container environment. The problem is
that java.sql.DriverManager will scan for the drivers only once.*"

Can someone help me understand what this means in more practical terms?


This will be a lot simpler to explain with the source code to hand:
https://github.com/openjdk/jdk/blob/master/src/java.sql/share/classes/java/sql/DriverManager.java


The page goes on to say:

"*...web applications that have database drivers in
their WEB-INF/lib directory cannot rely on the service provider mechanism
and should register the drivers explicitly.*"

And, indeed, I have found that placing my JDBC driver in Tomcat's
$CATALINA_HOME/lib
or $CATALINA_BASE/lib will be loaded correctly, without explicit
registration.

MY QUESTIONS:

(1) I don't understand why the "scan only once" limitation results in this
behavior - so what am I missing, here, conceptually?


There are several inter-related elements.

Scan once caused problems as if two web apps both have JDBC drivers then 
the DriverManager scan will only load the Driver for the app that 
triggers the scan first.


An added complication is that the memory leak protection code 
essentially ensures that the scan is performed by Tomcat internal code 
so no JDBC drivers in WEB-INF/lib for any wweb application are seen by 
the scan. Hence, the JDBC drivers need to be in CATALINA_BASE/lib.

 > (2) Where is this "scan only once" behavior documented?

It is sort of implied in the DriverManager Javadoc but you need to read 
the source code to get a clear picture.



(3) What is it about a servlet container environment which allows this
problem to exist?


Dynamic loading and unloading of web applications and use of a dedicated 
class loader per web application.


You can also get various memory leaks associated with DriverManager as 
well (which Tomcat automatically protects you against).


Mark




Thank you.

For reference, here is the documentation link I used above:


http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html

(The wording in the documentation has been this way for several Tomcat
versions, going back a few years.)




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: No way to return error from broken streaming connection?

2021-08-10 Thread Mark Thomas
On August 10, 2021 8:00:45 AM UTC, Marcin Wisnicki  wrote:
>However I've noticed that in Tomcat the behaviour can only be achieved
>by throwing an exception before close.
>If I don't throw and simply return or call close before exception
>Tomcat will write terminal '0'.
>
>Is there a way to trigger immediate close without that write('0')
>without using exceptions?

No.

Return indicates the response is complete. Error conditions are indicated with 
exceptions.

Mark


>
>On Tue, 10 Aug 2021 at 03:27, Marcin Wisnicki 
>wrote:
>>
>> Thanks, it seems like it must be one of Spring Boot's filters
>breaking
>> it. I should've tried without it first, mea culpa.
>> In a pure tomcat container it works as you described whereas with
>> Spring something inserts final 0 so the aborted connection looks like
>> a valid chunked response to clients.
>>
>> I'll move my question to a more appropriate forum.
>>
>> On Tue, 10 Aug 2021 at 02:45, Mark Thomas  wrote:
>> >
>> > On 10/08/2021 02:25, Marcin Wisnicki wrote:
>> > > I have a servlet (it's really a SpringBoot controller but it
>shouldn't
>> > > matter?) in Tomcat 9.0.46 that streams responses of unknown size
>to
>> > > the browser.
>> > >
>> > > I've discovered that if the streaming process fails in the middle
>of
>> > > the write, the client will simply get an incomplete file and
>think it
>> > > was successful.
>> >
>> > Then you have a broken client.
>> >
>> > > Indeed there is no way to tell from the browser side there was an
>error.
>> >
>> > That statement is incorrect.
>> >
>> > >void doGet(HttpServletResponse resp) {
>> > >  resp.setStatus(200);
>> > >  resp.setHeader("Content-Disposition", "...");
>> > >  resp.setContentType("application/octet-stream");
>> > >  var out = resp.getOutputStream();
>> > >  out.flush();
>> > >  if (true) throw new RuntimeException("simulated error");
>> > >  // status is already sent here, can't change it!
>> > >}
>> > >
>> > > As far as I can tell a web server should close connection with
>TCP RST
>> > > or write some garbage that's not a valid chunked response.
>> >
>> > You won't see a TCP RST. You will see invalid chunked encoding.
>> >
>> > > This is what some other servers do such as express.js.
>> > >
>> > > Unfortunately Tomcat does neither and also there seems to be no
>way to
>> > > trigger that using either Servlet or Tomcat api.
>> >
>> > Again, that statement is incorrect.
>> >
>> > > I've tested it by setting "Connection: close" then writing
>garbage and
>> > > it seems browser recognized download error.
>> > > But then I lose chunking in the output stream and can't actually
>write
>> > > a response.
>> > >
>> > > How do people solve it?
>> >
>> > Look at the following from a test using telnet to make a request to
>the
>> > sample servlet above:
>> >
>> > $ telnet localhost 8080
>> > Trying ::1...
>> > Connected to localhost.
>> > Escape character is '^]'.
>> > GET /tomcat-bugs/user004 HTTP/1.1
>> > Host: localhost
>> >
>> > HTTP/1.1 200
>> > Content-Type: text/plain
>> > Transfer-Encoding: chunked
>> > Date: Tue, 10 Aug 2021 06:38:06 GMT
>> >
>> > Connection closed by foreign host.
>> > $
>> >
>> > The headers are sent when the Servlet flushes the response. That is
>as
>> > required by the Servlet specification.
>> >
>> > What tells the client that the response is incomplete is that
>chunked
>> > encoding is being used but there is no final chunk.
>> >
>> > Mark
>
>-
>To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: No way to return error from broken streaming connection?

2021-08-10 Thread Mark Thomas

On 10/08/2021 02:25, Marcin Wisnicki wrote:

I have a servlet (it's really a SpringBoot controller but it shouldn't
matter?) in Tomcat 9.0.46 that streams responses of unknown size to
the browser.

I've discovered that if the streaming process fails in the middle of
the write, the client will simply get an incomplete file and think it
was successful.


Then you have a broken client.


Indeed there is no way to tell from the browser side there was an error.


That statement is incorrect.


   void doGet(HttpServletResponse resp) {
 resp.setStatus(200);
 resp.setHeader("Content-Disposition", "...");
 resp.setContentType("application/octet-stream");
 var out = resp.getOutputStream();
 out.flush();
 if (true) throw new RuntimeException("simulated error");
 // status is already sent here, can't change it!
   }

As far as I can tell a web server should close connection with TCP RST
or write some garbage that's not a valid chunked response.


You won't see a TCP RST. You will see invalid chunked encoding.


This is what some other servers do such as express.js.

Unfortunately Tomcat does neither and also there seems to be no way to
trigger that using either Servlet or Tomcat api.


Again, that statement is incorrect.


I've tested it by setting "Connection: close" then writing garbage and
it seems browser recognized download error.
But then I lose chunking in the output stream and can't actually write
a response.

How do people solve it?


Look at the following from a test using telnet to make a request to the 
sample servlet above:


$ telnet localhost 8080
Trying ::1...
Connected to localhost.
Escape character is '^]'.
GET /tomcat-bugs/user004 HTTP/1.1
Host: localhost

HTTP/1.1 200
Content-Type: text/plain
Transfer-Encoding: chunked
Date: Tue, 10 Aug 2021 06:38:06 GMT

Connection closed by foreign host.
$

The headers are sent when the Servlet flushes the response. That is as 
required by the Servlet specification.


What tells the client that the response is incomplete is that chunked 
encoding is being used but there is no final chunk.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: More information, Re: Tomcat 8.5.68 failing on takeoff!

2021-08-09 Thread Mark Thomas
On August 9, 2021 5:31:41 PM UTC, "James H. H. Lampert" 
 wrote:
>On 8/9/21 10:24 AM, Mark Thomas wrote:
>> Future versions of Tomcat won't see this issue but if the customer is
>
>> prepared to update Tomcat to fix this issue then they might as well
>just 
>> update Java (assuming that is indeed sufficient to fix this).
>
>Given that they currently seem to be happy as clams on their currently 
>installed Tomcat 7, and that we've only been updating customers to 
>Tomcat 8 in order to proactively deal with any security complaints that
>
>might come up, they can presumably wait for a Tomcat 8 that has the 
>relevant fix.
>
>And this customer is not in any particular hurry for updating
>*anything* 
>(a trait I share).
>
>Any idea when a Tomcat 8 with the relevant fix might come out?

The fix will be in the September releases.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: More information, Re: Tomcat 8.5.68 failing on takeoff!

2021-08-09 Thread Mark Thomas

On 09/08/2021 19:21, James H. H. Lampert wrote:

On 8/6/21 9:17 AM, Konstantin Kolinko wrote:


Try to find what *.jar file in your system contains the above classes.

E.g. searching for string "crimson" in *.jar files.
That string will be visible in the archive file as it is a name of a 
directory.


I've learned that QShell (a *nix-like shell that was added with Java 
support) on an AS/400 does have both "find" and "grep," so it wasn't 
quite so futile as I thought.


If I go into QShell, navigate to the JVM home directory, and do a
"find . -name '*.jar'"
I get

  ./jre/lib/ppc64/compressedrefs/jclSC180/vm.jar
  ./jre/lib/ppc64/default/jclSC180/vm.jar
  ./jre/lib/ext/db2_classes18.jar
  ./jre/lib/ext/CmpCrmf.jar
  ./jre/lib/ext/IBMSecureRandom.jar
  ./jre/lib/ext/cldrdata.jar
  ./jre/lib/ext/dnsns.jar
  ./jre/lib/ext/dtfj.jar
  ./jre/lib/ext/dtfjview.jar
  ./jre/lib/ext/gskikm.jar
  ./jre/lib/ext/healthcenter.jar
  ./jre/lib/ext/ibmcmsprovider.jar
  ./jre/lib/ext/ibmjcefips.jar
  ./jre/lib/ext/ibmjceplus.jar
  ./jre/lib/ext/ibmjceprovider.jar
  ./jre/lib/ext/ibmkeycert.jar
  ./jre/lib/ext/ibmpkcs11impl.jar
  ./jre/lib/ext/ibmsaslprovider.jar
  ./jre/lib/ext/ibmxmlcrypto.jar
  ./jre/lib/ext/ibmxmldsigprovider.jar
  ./jre/lib/ext/ibmxmlencprovider.jar
  ./jre/lib/ext/jaccess.jar
  ./jre/lib/ext/localedata.jar
  ./jre/lib/ext/nashorn.jar
  ./jre/lib/ext/traceformat.jar
  ./jre/lib/ext/xmlencfw.jar
  ./jre/lib/ext/zipfs.jar
  ./jre/lib/aggressive.jar
  ./jre/lib/charsets.jar
  ./jre/lib/dataaccess.jar
  ./jre/lib/ddr/j9ddr.jar
  ./jre/lib/deploy.jar
  ./jre/lib/ibmcertpathfw.jar
  ./jre/lib/ibmcertpathprovider.jar
  ./jre/lib/ibmcfw.jar
  ./jre/lib/ibmjcefw.jar
  ./jre/lib/ibmjgssfw.jar
  ./jre/lib/ibmjgssprovider.jar
  ./jre/lib/ibmjssefw.jar
  ./jre/lib/ibmjsseprovider2.jar
  ./jre/lib/ibmorb.jar
  ./jre/lib/ibmorbapi.jar
  ./jre/lib/ibmpkcs.jar
  ./jre/lib/ibmsaslfw.jar
  ./jre/lib/javaws.jar
  ./jre/lib/management-agent.jar
  ./jre/lib/math.jar
  ./jre/lib/plugin.jar
  ./jre/lib/resources.jar
  ./jre/lib/rt.jar
  ./jre/lib/se-service.jar
  ./jre/lib/security/US_export_policy_56bit.jar
  ./jre/lib/security/local_policy_56bit.jar
  ./jre/lib/security/policy/limited/US_export_policy.jar
  ./jre/lib/security/policy/limited/local_policy.jar
  ./jre/lib/security/policy/unlimited/US_export_policy.jar
  ./jre/lib/security/policy/unlimited/local_policy.jar
  ./jre/lib/tools/monitoring-api.jar
  ./jre/lib/xml.jar
  ./jre/lib/xmldsigfw.jar
  ./IBMmisc.jar
  ./lib/dt.jar
  ./lib/ibmorbtools.jar
  ./lib/jconsole.jar
  ./lib/tools.jar
  ./lib/IBMi5OSJSSE.jar

If I do a "find . -name '*crimson*'", I get nothing.

If I do a "find . -name '*.jar' -exec grep -l 'crimson' {} \;", I also 
get nothing.


So unless anybody else has any ideas, I'm once again stuck, at least on 
this angle.


Mark Thomas wrote:
Tomcat 7 doesn't have JASPIC support so you'll never see this issue in 
Tomcat 7.


. . . to which I replied (seriously, rather than flippantly) "What's a 
JASPIC?"


I've finally taken a look at what JASPIC is. Interesting. If it's JASPIC 
support in Tomcat 8 that is throwing exceptions and killing manager 
under this particular Java 8 JVM, is there a way to disable it, at least 
until the customer has their PTFs up to date?


Sorry, no. The code in question is always going to get executed.

Future versions of Tomcat won't see this issue but if the customer is 
prepared to update Tomcat to fix this issue then they might as well just 
update Java (assuming that is indeed sufficient to fix this).


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.1.0-M4 (alpha) available

2021-08-07 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.1.0-M4 (alpha).

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10.1.0-M4 is a milestone release of the 10.1.x branch and 
has been made to provide users with early access to the new features in 
Apache Tomcat 10.1.x so that they may provide feedback. The notable 
changes compared to 10.1.0-M2 include:


- The minimum Java version has been increased to Java 11 to align with
  the plans for the Jakarta EE 10 platform

- Correct a regression in the previous release in the HTTP/2 flow
  control window management

- Enable EL lambda expressions to be coerced to functional interfaces.
  This is an implementation of a proposed extension to the Jakarta
  Expression Language specification.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.1-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[ANN] Apache Tomcat 10.0.10 available

2021-08-06 Thread Mark Thomas

The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.0.10.

This release is targeted at Jakarta EE 9.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

The notable changes compared to 10.0.8 include:

- Correct a regression in the previous release in the HTTP/2 flow
  control window management

- Correct a regression the could cause some TLS connections to hang when
  using NIO

- Use of GraalVM native images no longer automatically disables JMX
  support.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.0-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 7.0.x, 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Http11NioProtocol with TLS seems to be very slow for certain requests >= 9.0.48

2021-08-06 Thread Mark Thomas
On August 6, 2021 1:35:24 PM UTC, Benjamin Grenacher 
 wrote:
>Recently updated from 9.0.43 to 9.0.50 and are having similar symptoms
>as already reported ("Possible Http11NioProtocol regression since
>9.0.48?").
>
>Integration test runs have shown this issue seems to occur for browser
>tests only. Since we have larger JS files and requests take about 1 min
>(default connectionTimeout/keepAliveTimeout) this seems to be the same
>issue.
>
>Downgrading to 9.0.48 shows that the issue persists.
>
>Related active configuration is only "compression=off" the rest is
>default. Trying to change configuration (useSendfile, selectorTimeout,
>useBomIfPresent) didn't help.
>
>Seems to be resolved when using HTTP (no TLS).
>
>
>Running on docker swarm through reverse proxy and without explicit
>reverse proxy (swarm internally routes traffic, so kind of "proxy" in
>place).
>
>When running through reverse proxy there occur "I/O Errors" reported by
>the proxy (not restricted to JS file requests - e.g. includes favicon
>and svg requests).
>
>Related error logs (browser):
>
>https://XXX...b85a.js - Failed to load resource:
>net::ERR_CONTENT_LENGTH_MISMATCH
>
>https://XXX..f.min.js 14:1649 Uncaught SyntaxError: Unexpected end of
>input
>
>
>Running with local docker containers (without proxy): Can't reproduce
>issue.
>
>
>Let me know if you need anything else.

Most likely this:

https://bz.apache.org/bugzilla/show_bug.cgi?id=65448

Fix is in 9.0.51 onwards. Should be available in a few days if the vote is 
successful.

Switch to NIO2 if you need an immediate work around.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Question for verification

2021-08-06 Thread Mark Thomas
On August 6, 2021 2:24:13 PM UTC, jonmcalexan...@wellsfargo.com.INVALID wrote:
>Verifying an assumption.
>
>All modern versions of Tomcat (8.5 and above) are compatible with Java
>11.

Yes.

We regularly test Tomcat with the early access versions of each Java release. 
We also have CI systems that test 8, 11 and latest.

Mark

>
>Thanks in advance
>
>Dream * Excel * Explore * Inspire
>Jon McAlexander
>Infrastructure Engineer
>Asst Vice President
>
>Middleware Product Engineering
>Enterprise CIO | Platform Services | Middleware | Infrastructure
>Solutions
>
>8080 Cobblestone Rd | Urbandale, IA 50322
>MAC: F4469-010
>Tel 515-988-2508 | Cell 515-988-2508
>
>jonmcalexan...@wellsfargo.com
>
>Upcoming PTO: 10/30/2020, 11/6/2020, 11/13/2020, 11/20/2020,
>11/27/2020, 12/2/2020, 12/4/2020, 12/11/2020, 12/18/2020, 12/28/2020,
>12/29/2020, 12/30/2020, 12/31/2020
>This message may contain confidential and/or privileged information. If
>you are not the addressee or authorized to receive this for the
>addressee, you must not use, copy, disclose, or take any action based
>on this message or any information herein. If you have received this
>message in error, please advise the sender immediately by reply e-mail
>and delete this message. Thank you for your cooperation.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Wrong logic for NONE as certificateKeystoreFile?

2021-08-06 Thread Mark Thomas

Thanks for the report. Fixed for the September release round.

Mark


On 05/08/2021 14:09, Mikael Sterner wrote:

It seems like the logic implemented for NONE as certificateKeystoreFile
deviates from the documentation. Currently NONE is always interpreted as
a file path, even for PKCS11. Looks like the comparison with NONE should
be inside the parentheses for the negation? A workaround is to use ""
instead of NONE.

https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/SSLUtilBase.java#L196

Yours,
Mikael

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: More information, Re: Tomcat 8.5.68 failing on takeoff!

2021-08-06 Thread Mark Thomas

On 06/08/2021 06:15, Christopher Schultz wrote:

On 8/5/21 18:33, James H. H. Lampert wrote:




java.lang.SecurityException: org.xml.sax.SAXNotRecognizedException: 
Feature: http://apache.org/xml/features/allow-java-encodings
 org.apache.catalina.authenticator.jaspic.PersistentProviderRegistrations.loadProviders(PersistentProviderRegistrations.java:65) 

 org.apache.catalina.authenticator.jaspic.AuthConfigFactoryImpl.loadPersistentRegistrations(AuthConfigFactoryImpl.java:345) 

 org.apache.catalina.authenticator.jaspic.AuthConfigFactoryImpl.(AuthConfigFactoryImpl.java:68) 

 sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
 sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:83) 

 sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57) 


 java.lang.reflect.Constructor.newInstance(Constructor.java:437)
 javax.security.auth.message.config.AuthConfigFactory$1.run(AuthConfigFactory.java:76) 

 javax.security.auth.message.config.AuthConfigFactory$1.run(AuthConfigFactory.java:67) 

 java.security.AccessController.doPrivileged(AccessController.java:696) 

 javax.security.auth.message.config.AuthConfigFactory.getFactory(AuthConfigFactory.java:66) 






Now THAT looks like a bug. Here is the code around that setFeature() call:

     Digester digester = new Digester();

     try {

digester.setFeature("http://apache.org/xml/features/allow-java-encodings;, 
true);

     digester.setValidating(true);
     digester.setNamespaceAware(true);
     } catch (Exception e) {
     throw new SecurityException(e);
     }

That digester.setFeature() call should be in its own try/catch block 
which issues a warning if a SAXException is thrown.


+1

And to reiterate, the very same JVM has no difficulty at all running 
Tomcat 7.0.93.


Something seems odd about that. There must be soe configuration 
difference between your 7.0.x environment and the 8.5.x environment you 
are testing.


Tomcat 7 doesn't have JASPIC support so you'll never see this issue in 
Tomcat 7.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Minor doc bug, DSS should be DSA for certificate type?

2021-08-06 Thread Mark Thomas

Hi Mikael,

Thanks for spotting and reporting this. I've just fixed this typo in all 
the supported branches. The fix will be included in the September 
release round.


Kind regards,

Mark


On 04/08/2021 19:13, Mikael Sterner wrote:

In tomcat/webapps/docs/config/http.xml, it seems like the valid values for
the type attribute of the Certificate element should include DSA instead
of DSS, to match the enum used in the code?

https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/SSLHostConfigCertificate.java#L276

Yours,
Mikael

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: Xms Xmx in JAVA_OPT vs CATALINA_OPTS

2021-08-04 Thread Mark Thomas
On August 3, 2021 7:34:37 PM UTC, Adam Elliott  wrote:

Adam,

As per the reasoning in Olaf's email you really should be using CATALINA_OPTS 
rather than JAVA_OPTS.

Mark

>From my understanding, setenv is not a default file, and any settings
>inside are "custom".  And all of these options are used by the JVM, so
>in your case thy are probably redundant.  I only use JAVA_OPTS under
>setenv.sh and use it in a way to add my options to the startup.
>
>export JAVA_OPTS="$JAVA_OPTS\
> -XX:+UseParallelGC\
> -XX:ParallelGCThreads=20\
> -XX:+CMSClassUnloadingEnabled\
> -Xss256k\
> -Xms974M\
> -Xmn974M\
> -Xmx1948M"
>
>Adam
>
>-Original Message-
>From: Onno van der Straaten  
>Sent: Monday, August 2, 2021 7:57 PM
>To: users@tomcat.apache.org
>Subject: Xms Xmx in JAVA_OPT vs CATALINA_OPTS
>
>Hi,
>I was looking at a Tomcat deployment and noticed settings in setenv.sh
>as shown below. I noticed that Xms and Xmx is in JAVA_OPTS and
>CATALINA_OPTS with the exact same settings. Do these settings make
>sense? What is the purpose of repeating those settings?
>
>I understand that JAVA_OPTS is for the JVM and CATALINA_OPTS is
>specific to Tomcat. How do these settings relate to each other? I am
>assuming they could be different.
>Thanks and Regards,
>Onno
>
>
>JAVA_HOME="/usr/lib/jvm/jdk11_0411_oj9"
>JAVA_OPTS="-Xms3000m -Xmx3000m"
>CATALINA_OPTS="-Dcom.sun.management.jmxremote.port=8375
>-Dcom.sun.management.jmxremote.ssl=false
>-Dcom.sun.management.jmxremote.authenticate=false
>-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
>-Dfile.encoding=UTF-8 -Dsun.net.inetaddr.ttl=300 -server
>-Dorg.apache.tomcat.util.http.Parameters.MAX_COUNT=8192 -Xms3000m
>-Xmx3000m -XX:MaxPermSize=500m -XX:+UseConcMarkSweepGC
>-XX:+CMSIncrementalMode -XX:+PrintGCDateStamps -verbose:gc
>-XX:+PrintGCDetails
>-Xloggc:"/opt/tomcat/tomcat-9.0.40/logs/garbage.log"
>-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
>-XX:GCLogFileSize=100M
>-Djavax.sql.DataSource.Factory=org.apache.commons.dbcp.BasicDataSourceFactory"
>
>
>CAUTION: This email originated from outside of your organization. Do
>not click links or open attachments unless you recognize the sender and
>know the content is safe. Do not reply to emails that contain a From
>address you are unfamiliar with.
>
>-
>To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>For additional commands, e-mail: users-h...@tomcat.apache.org
>>


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: compression?

2021-07-27 Thread Mark Thomas

On 27/07/2021 13:08, Berneburg, Cris J. - US wrote:

Carsten and Mark

Thanks for the info.  :-)

crisb> Weird, when going thru IIS to TC, it's not compressed

c.klein> IIS fetches the requested resource from TC, acting as an HTTP client 
(or are you using AJP with IIS?).

markt> IIS will be using AJP to talk to Tomcat which doesn't support 
compression. You may be able to get IIS to compress the files.

Is it possible to connect IIS to TC using HTTP instead of AJP?  Several "Tomcat IIS 
How-To" articles all mention using AJP (not HTTP) using an ISAPI redirector.


In theory, yes. You'd need to find an HTTP reverse proxy component for IIS.

This looks like the sort of thing you'd need:

https://docs.microsoft.com/en-us/iis/extensions/url-rewrite-module/reverse-proxy-with-url-rewrite-v2-and-application-request-routing

The downside is that you will need to manually configure a lot of the 
stuff AJP does "for free". Correctly configuring a reverse proxy is one 
of those tasks where all sorts of things can catch you out.


I'd probably look at getting IIS to compress the content instead:
https://docs.microsoft.com/en-us/iis/extensions/iis-compression/iis-compression-overview

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat Usage Data Interest

2021-07-26 Thread Mark Thomas

On 26/07/2021 12:13, Coty Sutherland wrote:

Hi all,

I'm curious about whether or not we have/can get some information about the
usage of Tomcat out in the wild. Things like download count across various
versions (including archived version downloads) for the last few years, svn
history and GitHub stats, project website visitors, committer numbers (and
some other info which I can get from the regular board reports), counts of
tomcat-users list unique topics, etc. I'd like to compile data into a
community interest report (or something like that) and try to draw some
insights on which way the Tomcat project is trending. I would also be
looking to include adoption outside of just the vanilla ASF distro, like
the most popular Tomcat Docker container, Ansible collection, tomcat
package downloads from any OS that has the data available, etc.

Does anyone think that such a report has value? Is there already something
like this in existence somewhere (there is an annual jrebel technology
report like https://www.jrebel.com/blog/2020-java-technology-report which
is pretty cool, but it's a survey)? Feel free to tell me that this
undertaking has little value and I can move on to something else :)
Thoughts?


In no particular order.

There is Apache Kibble
https://kibble.apache.org/
The live demo uses ASF data.

The mirror network makes download stats tricky.

We can get Maven central stats via repository.a.o

In terms of whether a report has value, more insight into the community 
is good. The users mailing list is an incredibly small proportion of the 
active Tomcat users. Anything that provides us with a better 
understanding of the wider community can only help. I'd be particularly 
interested in things we could do to broaden our reach. That may well 
create some interesting debate on how to best do that.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: compression?

2021-07-23 Thread Mark Thomas

On 23/07/2021 18:53, Berneburg, Cris J. - US wrote:

Thanks Mark!

cb> 1. compressionMinSize - What are the units, bytes?
Markt> Yes.

cb> 2. compressibleMimeType - If you specify a type explicitly, [...]  Are [the 
defaults]
cb> over-ridden, so they need to be specified explicitly too?  Or is it 
cumulative?
Markt> Default is over-ridden.

OK, that worked when connecting directly to TC:

HTTP/1.1 200
vary: accept-encoding
Content-Encoding: gzip
Content-Type: application/json;charset=UTF-8
Content-Language: en-US
Transfer-Encoding: chunked
Date: Fri, 23 Jul 2021 16:37:48 GMT
Keep-Alive: timeout=20
Connection: keep-alive

Weird, when going thru IIS to TC, it's not compressed:


IIS will be using AJP to talk to Tomcat which doesn't support 
compression. You may be able to get IIS to compress the files.


Mark




HTTP/1.1 200 200
Content-Type: application/json;charset=UTF-8
Content-Language: en-US
Server: Microsoft-IIS/10.0
Date: Fri, 23 Jul 2021 16:34:30 GMT
Content-Length: 3210105

cb> P.S.: If a documentation update is recommended, I would be happy to
cb> make the changes, but I would probably need guidance for that too.  ;-)

Markt> Source file is here:
Markt> https://github.com/apache/tomcat/blob/main/webapps/docs/config/http.xml

Markt> A pull request is fine. If you prefer to provide a patch, use "diff -u"
Markt> format, create a BZ issue and attach the patch.

I'll have a look at it later.  Also, I'm quite a newbie with git.

--
Cris Berneburg
CACI Senior Software Engineer




This electronic message contains information from CACI International Inc or 
subsidiary companies, which may be company sensitive, proprietary, privileged 
or otherwise protected from disclosure. The information is intended to be used 
solely by the recipient(s) named above. If you are not an intended recipient, 
be aware that any review, disclosure, copying, distribution or use of this 
transmission or its contents is prohibited. If you have received this 
transmission in error, please notify the sender immediately.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Strange incomplete response/truncation with Tomcat 9.0.48 AND 9.0.50

2021-07-23 Thread Mark Thomas

On 23/07/2021 15:49, jonmcalexan...@wellsfargo.com.INVALID wrote:

Is there an estimated target date for release of 9.0.51


Normally I'd say early August, some time in the first 2 weeks. But as we 
are entering vacation season it might slip. It largely depends on the 
release manager's availabaility.


Mark




Dream * Excel * Explore * Inspire
Jon McAlexander
Infrastructure Engineer
Asst Vice President

Middleware Product Engineering
Enterprise CIO | Platform Services | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508

jonmcalexan...@wellsfargo.com

Upcoming PTO: 10/30/2020, 11/6/2020, 11/13/2020, 11/20/2020, 11/27/2020, 
12/2/2020, 12/4/2020, 12/11/2020, 12/18/2020, 12/28/2020, 12/29/2020, 
12/30/2020, 12/31/2020
This message may contain confidential and/or privileged information. If you are 
not the addressee or authorized to receive this for the addressee, you must not 
use, copy, disclose, or take any action based on this message or any 
information herein. If you have received this message in error, please advise 
the sender immediately by reply e-mail and delete this message. Thank you for 
your cooperation.


-Original Message-
From: Mark Thomas 
Sent: Friday, July 23, 2021 2:56 AM
To: users@tomcat.apache.org
Subject: Re: Strange incomplete response/truncation with Tomcat 9.0.48
AND 9.0.50

On 22/07/2021 22:06, jonmcalexan...@wellsfargo.com.INVALID wrote:

I have a team that is running into issues since version 9.0.48 where
they are receiving incomplete message responses from Tomcat when the
request was made from WebLogic.


Incomplete responses from 9.0.48 onwards. That sounds like a recently fixed
regression. That issue happened with TLS.




*adrum.js:27 Error: Loading chunk 28 failed.*

(timeout:
https://.../.5af0fea300ccf52ff152.js)


That looks like TLS is being used which is consistent with the suspected root
cause.




*_Network level_* we are seeing *TCP Window Full* intermittently when
this file transfer.


This is also consistent with the likely root cause. The regression was in the
handling of incomplete writes.




After some additional research we assume this issue is related to one
of the known bugs listed in RedHat TC release notes
<https://urldefense.com/v3/__https://tomcat.apache.org/tomcat-9.0-

doc/changelog.html__;!!F9svGWnIaVPGSwU!69FyojmXXQigaRKpGDiwpMgS
sgODh4HrEhdK9d8ZbHZsJjpqNcD2ZmKprbbGjevCxxzKTSc$ >.


Fix:  Expand the unit tests for HttpServlet.doHead()


Not an unreasonable guess but it looks to be an incorrect one.

I always recommend looking at the open bugs and the changelog from the CI
system to see if the issue being observed has already been reported (and
possibly fixed).

https://urldefense.com/v3/__https://ci.apache.org/projects/tomcat/tomcat
-
9.0.x/docs/changelog.html__;!!F9svGWnIaVPGSwU!69FyojmXXQigaRKpGDi
wpMgSsgODh4HrEhdK9d8ZbHZsJjpqNcD2ZmKprbbGjevC8PK9tLQ$

This looks much more like bug 65448 to me:
https://urldefense.com/v3/__https://bz.apache.org/bugzilla/show_bug.cgi?
id=65448__;!!F9svGWnIaVPGSwU!69FyojmXXQigaRKpGDiwpMgSsgODh4HrE
hdK9d8ZbHZsJjpqNcD2ZmKprbbGjevCqMV6SbI$




Any help?


The fix will be in 9.0.51.

Snapshots (NOT formal releases) are available for testing from:
https://urldefense.com/v3/__https://repository.apache.org/content/group
s/snapshots/org/apache/tomcat/tomcat/9.0-
SNAPSHOT/__;!!F9svGWnIaVPGSwU!69FyojmXXQigaRKpGDiwpMgSsgODh4
HrEhdK9d8ZbHZsJjpqNcD2ZmKprbbGjevCZcMI2B0$

Usual caveats apply. These aren't releases. Use them entirely at your own
risk.

In terms of a workaround, switching from NIO to NIO2 should avoid the
issue.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Strange incomplete response/truncation with Tomcat 9.0.48 AND 9.0.50

2021-07-23 Thread Mark Thomas

On 22/07/2021 22:06, jonmcalexan...@wellsfargo.com.INVALID wrote:
I have a team that is running into issues since version 9.0.48 where 
they are receiving incomplete message responses from Tomcat when the 
request was made from WebLogic.


Incomplete responses from 9.0.48 onwards. That sounds like a recently 
fixed regression. That issue happened with TLS.





*adrum.js:27 Error: Loading chunk 28 failed.*

(timeout: https://.../.5af0fea300ccf52ff152.js)


That looks like TLS is being used which is consistent with the suspected 
root cause.




*_Network level_* we are seeing *TCP Window Full* intermittently when 
this file transfer.


This is also consistent with the likely root cause. The regression was 
in the handling of incomplete writes.




After some additional research we assume this issue is related to one of 
the known bugs listed in RedHat TC release notes 
.


Fix:  Expand the unit tests for HttpServlet.doHead()


Not an unreasonable guess but it looks to be an incorrect one.

I always recommend looking at the open bugs and the changelog from the 
CI system to see if the issue being observed has already been reported 
(and possibly fixed).


https://ci.apache.org/projects/tomcat/tomcat-9.0.x/docs/changelog.html

This looks much more like bug 65448 to me:
https://bz.apache.org/bugzilla/show_bug.cgi?id=65448




Any help?


The fix will be in 9.0.51.

Snapshots (NOT formal releases) are available for testing from:
https://repository.apache.org/content/groups/snapshots/org/apache/tomcat/tomcat/9.0-SNAPSHOT/

Usual caveats apply. These aren't releases. Use them entirely at your 
own risk.


In terms of a workaround, switching from NIO to NIO2 should avoid the issue.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat 8.5.57 stops killing sessions after some time

2021-07-21 Thread Mark Thomas

On 21/07/2021 16:00, Ivano Luberti wrote:


Il 21/07/2021 16:44, Mark Thomas ha scritto:
Take 3 thread dumps 5 seconds apart and post them here. 


How to take thread dumps?

  kill -3  ?


That will work. There are lots of ways. This is most of them:

https://www.baeldung.com/java-thread-dump

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: tomcat 8.5.57 stops killing sessions after some time

2021-07-21 Thread Mark Thomas

On 21/07/2021 15:34, Ivano Luberti wrote:
Hello, I'm new to the list but befeore writing this I have searched the 
users mailing list without finding anything useful.


I have an instance of Apache Tomcat/8.5.57 running on a CentOS machine 
with java 1.7.0_261


Several webapps run on this instance.

I have noticed that after a few days tomcat stops killing expired sessions.

Trying to understand where the issue is located , I have enabled debug 
logging of  org.apache.catalina.session package with:


java.util.logging.ConsoleHandler.level = FINEST

org.apache.catalina.session.level = FINEST

So after the latest restart i could observe in catalina.out many log 
rows like


21-Jul-2021 10:02:00.702 FINE 
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] 
org.apache.catalina.session.ManagerBase.processExpires Start expire 
sessions StandardManager at 1626861720702 sessioncount 1
21-Jul-2021 10:02:00.702 FINE 
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] 
org.apache.catalina.session.ManagerBase.processExpires End expire 
sessions StandardManager processingTime 0 expired sessions: 0


Today, after 4 days without issues tomcat stopped again killing expired 
sessions and at the same time no more logs of


org.apache.catalina.session.ManagerBase.processExpires

are reported.

There is no other related log event , simply the last row reported is

21-Jul-2021 12:38:39.370 FINE 
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] 
org.apache.catalina.session.ManagerBase.processExpires Start expire 
sessions StandardManager at 1626871119370 sessioncount 7


Any clue on why org.apache.catalina.session.ManagerBase is not called 
anymore?


Deadlock during background processing maybe?


Suggestion on how to diagnose?


Take 3 thread dumps 5 seconds apart and post them here.


Any chance that some exception occurring in

HttpSessionListener.sessionCreated
or
HttpSessionListener.sessionDestroyed
or
HttpSessionBindingListener.valueBound
or
HttpSessionBindingListener.valueUnbound

can terminate ManagerBase but without generating any logging?


Not that I can see. If the thread fails, it should be automatically 
restarted. An OOME might be able to trigger this but that looks pretty 
far fetched at this point.



Those are the only interaction with session handling I'm aware of.

I have the same webapps running on other tomcat instances, but on older 
tomcat version and there I have no issue of ths kind.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Cache-Control for INTEGRAL transport guarantee?

2021-07-21 Thread Mark Thomas

On 21/07/2021 08:06, Mikael Sterner wrote:

On Tue, Jul 20, 2021, at 10:04, Mark Thomas wrote:

Cache headers have been somewhat of a moving target with different
browsers behaving in different ways at different times over the years.

I wanted to review the current state of things before forming an opinion
on this suggestion. I found the following the most useful:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Pragma


Thanks for the investigation!

This surely seems a difficult area to find a one-size-fits-all solution.
My hope was that a new default for the no-authorization-constraint and
INTEGRAL-transport-guarantee case would make it unnecessary to set these
headers explicitly for static contexts. But if I anyway want to set
something exotic like "immutable" for static resources, the best I could
hope for from such a new default would be a clean slate, i.e. as few
headers set as possible.


I'm wondering if an alternative solution would be to move the code that
sets "Expires: Thu, 01 Jan 1970 00:00:00 GMT" to the
securePagesWithPragma section. If we are sending "Cache-control:
private" then I don't, at the moment, see a need to force revalidation
of that cached content on every request. That would leave the
application free to set its own expires headers (or not) as it saw fit.

Anyone using an HTTP/1.0 proxy would need to be using
securePagesWithPragma anyway so would be unaffected by this.


That would leave the slate cleaner at least. Though I'm not sure why
Expires would still be needed for the securePagesWithPragma branch; if
it's not, I guess it could simply be removed. (Sorry for any confusion
talking about Cache-Control: no-cache in the first post. I hadn't noticed
that securePagesWithPragma was now by default false, so I looked at the
wrong code branch.)


I'm reluctant to remove the Expires header from the the 
securePagesWithPragma branch because that branch is for older HTTP/1.0 
proxies and I'm not confident that it isn't required for some proxies. 
I'm not confident it is required either so my default position is to 
effectively leave it as it is. If someone reports a problem with it with 
an HTTP/1.0 proxy we can look at it then.


For the other branch, I am confident that Cache-Control header will be 
honoured which means that the Expires header is unnecessary. Since the 
Expires header is also triggering unnecessary cache validations then I 
can see a benefit to effectively removing it for that branch.


If I don't see any objections, I'll aim to do that in a couple of days.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: compression?

2021-07-21 Thread Mark Thomas

On 21/07/2021 15:06, Berneburg, Cris J. - US wrote:

Hi Folks :-)

Got some questions about turning on compression.  Looking at the documentation 
(I did not read the whole thing, just the portions in question), I still need 
some clarification.

https://tomcat.apache.org/tomcat-8.5-doc/config/http.html

1. compressionMinSize - What are the units, bytes?


Yes.


2. compressibleMimeType - If you specify a type explicitly, like "application/json", what 
does it do with the defaults, like "text/html"?  Are they over-ridden, so they need to be 
specified explicitly too?  Or is it cumulative?


Default is over-ridden.


Thanks for your time.

P.S.: If a documentation update is recommended, I would be happy to make the 
changes, but I would probably need guidance for that too.  ;-)


Source file is here:
https://github.com/apache/tomcat/blob/main/webapps/docs/config/http.xml

A pull request is fine. If you prefer to provide a patch, use "diff -u" 
format, create a BZ issue and attach the patch.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Log4j2 logging with Tomcat 9 web app

2021-07-21 Thread Mark Thomas

On 20/07/2021 21:05, Ravi Kumar wrote:

Hi,

My web app is based on Tomcat 9.0.45 server. I have migrated from Tomcat 
7 to Tomcat 9 and from log4j 1.x to log4j 2.x.
I have updated the log4j2.properties as per log4j 2.x standard, still my 
tomcat.log file is not getting generated and all the application log are 
coming on console instead of redirecting this to tomcat.log file.


so 1.tomcat.log is not geting  generated
  2. all the contents are logging and showing on the application 
console instead of getting this logged inside the tomcat.log file.


Tomcat 9.0.45 + log4j 2.14.1 is used.

I am also attaching my log4j property file. Please find this attached here.

Kindly suggest me the solution.


https://logging.apache.org/log4j/2.x/log4j-jul/index.html

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: request.getPathInfo() Question

2021-07-20 Thread Mark Thomas

On 19/07/2021 20:55, Jerry Malcolm wrote:
I have a servlet in the ROOT context mapped to "/".  I'm using 
request.getPathInfo() to get everything after the ".com/" in the URL. 
But no matter what is added to the url after .com/, getPathInfo() always 
returns null.  I printed out getRequestURL(), and it shows everything. I 
can parse the path out of getRequestURL().  But I'm curious why 
getPathInfo() is always null.  The docs say it returns null if there's 
no path info.  But that's not the case.  I'm on TC 9.0.16.


Tomcat's behaviour is correct since the servlet is the "default" servlet.

Please see section 12.2 of the Servlet 4.0 specification.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Cache-Control for INTEGRAL transport guarantee?

2021-07-20 Thread Mark Thomas

On 19/07/2021 20:46, Mikael Sterner wrote:

Hi!

I can understand the motivation for adding a Cache-Control header for
CONFIDENTIAL transport guarantees, as discussed in
http://tomcat.10.x6.nabble.com/tomcat-8-0-jre8-user-data-constraint-CONFIDENTIAL-user-data-constraint-adds-Cache-Control-private-tp5077170p5077201.html

But if the transport guarantee is only INTEGRAL (and there are no
authorization constraints), wouldn't it make more sense to let the
resource be fully cached?


Probably.


One use case would be a single-page app where some static resources never
change (e.g. due to content hashes in their names). These resources would
require data integrity, since it's important that the app logic and
content cannot be modified in transit. But if the resources don't require
authorization they aren't secret, so they don't require confidentiality.
And since they never change, it's wasteful for the client to check with
the server to validate its cache on every request.

What I'm proposing, if the above makes sense, would be to add an
additional criteria to the disableProxyCaching logic in AuthenticatorBase,
that goes through the constraints array and checks if there are any
authorization constraints or user data constraint with CONFIDENTIAL
transport guarantee. If not, no cache control headers should be added.

Static resources could then be configured with a transport guarantee of
INTEGRAL and still redirect to a secure connector if needed, but retain
full caching.


Cache headers have been somewhat of a moving target with different 
browsers behaving in different ways at different times over the years.


I wanted to review the current state of things before forming an opinion 
on this suggestion. I found the following the most useful:


https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Pragma

I'm wondering if an alternative solution would be to move the code that 
sets "Expires: Thu, 01 Jan 1970 00:00:00 GMT" to the 
securePagesWithPragma section. If we are sending "Cache-control: 
private" then I don't, at the moment, see a need to force revalidation 
of that cached content on every request. That would leave the 
application free to set its own expires headers (or not) as it saw fit.


Anyone using an HTTP/1.0 proxy would need to be using 
securePagesWithPragma anyway so would be unaffected by this.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: IIS 10.0 as Tomcat reverse proxy does not send auth_type and remote_user AJP heder

2021-07-19 Thread Mark Thomas

On 19/07/2021 10:20, Mark Thomas wrote:

On 13/07/2021 16:35, Paolo Clerici wrote:

I don't see any ISAPI redirector set up there. I was expecting to see
something like the steps described here:
http://tomcat.apache.org/connectors-doc/webserver_howto/iis.html

Yes, if I have not missed something, I think I have done everything
that is written in the document.
The only differences are that there are two sites "prod" and "test" so
the only differences for "test" are:
1) Dll folder: C:\Apache Software Foundation\Jakarta Isapi 
Redirector\test\bin

2) ISAPI filter name: "Jakarta Connector test" (not "tomcat")

isapi_redirect.properties file content:
extension_uri=/jakarta/isapi_redirect.dll
log_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\log\mod_jk.log
log_level=warn
worker_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\conf\workers.properties
worker_mount_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\conf\uriworkermap.properties

workers.properties file content:
worker.list=dgroupnex02,dgroupnex01
worker.dgroupnex02.type=ajp13
worker.dgroupnex02.host=10.1.2.93
worker.dgroupnex02.port=8009
worker.dgroupnex01.type=ajp13
worker.dgroupnex01.host=10.1.2.39
worker.dgroupnex01.port=8009

uriworkermap.properties file content:
/S2W/*=dgroupnex02
/s2wweb/*=dgroupnex01
/websat/*=dgroupnex02

I would like to tell you that ISAPI redirection of all virtual folders
works perfectly. The only thing that doesn't work is sending the
authorization type and user from IIS to Tomcat.
The only application that needs this functionality is "s2wweb".


How did you create the s2wweb virtual directory? Please provide exact 
steps. Was is created under the test site or under the jakarta virtual 
directory?


To be honest, I am far from convinced that I have recreated your 
configuration. Receiving the configuration bit by bit and ambiguities in 
the information received (is the test site configured for anon 
authentication, windows authentication or both?) makes me thing at least 
one key bit of information is missing.


Can you provide the complete set of steps required to configure a clean 
IIS 10 install to recreate this issue?


I have also been trying to recreate your IIS 6.1 setup without success. 
Which versions are you using for:

- operating system
- ISAPI connector
- Tomcat
?

And, similarly to above, what are the steps to recreate your test setup 
from a clean IIS install?


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: IIS 10.0 as Tomcat reverse proxy does not send auth_type and remote_user AJP heder

2021-07-19 Thread Mark Thomas

On 13/07/2021 16:35, Paolo Clerici wrote:

I don't see any ISAPI redirector set up there. I was expecting to see
something like the steps described here:
http://tomcat.apache.org/connectors-doc/webserver_howto/iis.html

Yes, if I have not missed something, I think I have done everything
that is written in the document.
The only differences are that there are two sites "prod" and "test" so
the only differences for "test" are:
1) Dll folder: C:\Apache Software Foundation\Jakarta Isapi Redirector\test\bin
2) ISAPI filter name: "Jakarta Connector test" (not "tomcat")

isapi_redirect.properties file content:
extension_uri=/jakarta/isapi_redirect.dll
log_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\log\mod_jk.log
log_level=warn
worker_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\conf\workers.properties
worker_mount_file=C:\Apache Software Foundation\Jakarta Isapi
Redirector\test\conf\uriworkermap.properties

workers.properties file content:
worker.list=dgroupnex02,dgroupnex01
worker.dgroupnex02.type=ajp13
worker.dgroupnex02.host=10.1.2.93
worker.dgroupnex02.port=8009
worker.dgroupnex01.type=ajp13
worker.dgroupnex01.host=10.1.2.39
worker.dgroupnex01.port=8009

uriworkermap.properties file content:
/S2W/*=dgroupnex02
/s2wweb/*=dgroupnex01
/websat/*=dgroupnex02

I would like to tell you that ISAPI redirection of all virtual folders
works perfectly. The only thing that doesn't work is sending the
authorization type and user from IIS to Tomcat.
The only application that needs this functionality is "s2wweb".


How did you create the s2wweb virtual directory? Please provide exact 
steps. Was is created under the test site or under the jakarta virtual 
directory?


To be honest, I am far from convinced that I have recreated your 
configuration. Receiving the configuration bit by bit and ambiguities in 
the information received (is the test site configured for anon 
authentication, windows authentication or both?) makes me thing at least 
one key bit of information is missing.


Can you provide the complete set of steps required to configure a clean 
IIS 10 install to recreate this issue?


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Tomcat HTTP/2 vs HTTP/1.1

2021-07-19 Thread Mark Thomas

On 18/07/2021 17:44, Deshmukh, Kedar wrote:

Hi,

We are in process of assessing HTTP/2 protocol for our Web applications as 
there are lot of benefits it provides over HTTP/1.1 at least theoretically. 
With given settings in tomcat, we are able to switch to HTTP/2 without trouble. 
For us next step is to see what benefits do we get over HTTP/1.1. Certain tools 
we generally used for web application load testing for e.g. jmeter. But, looks 
like most of the tools are not yet fully compliant with HTTP/2.

As I am subscribed to tomcat-dev group, I can see Tomcat team is putting lot of 
efforts on HTTP/2 support.
Now we would like to know,
Does tomcat dev team perform any assessment associated with these protocols ?


Protocols? You only mentioned HTTP/2.

We ran h2spec a lot during the development phase and run it occasionally 
now.



Are there any existing utility within tomcat to see any benefits ?


No.


Any stats available in tomcat ?


9.0.40, 8.5.60 and 10.0.0-M10 added request statistics (accessible via 
JMX) for HTTP/2, WebSocket and HTTP/1.1 upgraded connections.



Any third party tool to assess performance of HTTP/2 over HTTP/1.1 ?


There is an HTTP/2 extension available for JMeter. I've used it several 
times to investigate HTTP/2 related bugs.



Generally, for what kind of application you would suggest HTTP/2 over HTTP/1.1 ?


One that has lots of small resources per page.

Mark




This information will really help us to take decision to move to HTTP/2 with 
more confidence.

Thanks,
Kedar





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: IIS 10.0 as Tomcat reverse proxy does not send auth_type and remote_user AJP heder

2021-07-13 Thread Mark Thomas

On 13/07/2021 12:29, Paolo Clerici wrote:

Hi Mark,


How did you set up the s2wweb virtual directory?

Physical Path: C:\Apache Software Foundation\virtual\test\s2wweb
Physical Path Credential: blank
Physical Path Credential Logon Type: Clear Text
Virtual Path: /s2wweb
Pass-through authentication: / Connect As: / Path credentials:
Application user (pass-through authentication)


I don't see any ISAPI redirector set up there. I was expecting to see 
something like the steps described here:


http://tomcat.apache.org/connectors-doc/webserver_howto/iis.html

Mark




Thanks,
Paolo
Il giorno mar 13 lug 2021 alle ore 10:27 Mark Thomas
 ha scritto:


On 13/07/2021 08:49, Paolo Clerici wrote:

Hi Mark,


Are you connecting from a machine that isn't part of the Windows AD?

I have tried both from PCs connected to AD and from PCs not connected to AD.


Normally, I'd expect authentication to work without any password prompt.

If I connect from PC AD I am not asked for credentials (correct). If I
connect from a non-AD PC I am prompted for credentials (correctly).
The credential check is done correctly by IIS.


Are any other authentication mechanisms enabled?

For virtual directory "s2wweb" only "Windows Authentication" is
enabled ("Anonymous Authentication" is disabled). For site "test" is
enabled "Anonymous Authentication".


Are your two test machines (working and not working) connecting to the
same Tomcat instance (and on the same port)?

Yes.
Current IIS server needs to be migrated to a new IIS server. The
current server (Windows Server 2008 R2 with IIS 6.1) is connected to
the same Tomcat server (another Windows Server 2008 R2 with Tomcat
7.0) on the same port (8009).


Again, testing a similar setup locally works as expected. The
authenticated Windows user name is passed to Tomcat.

How did you set up the s2wweb virtual directory?

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



  1   2   3   4   5   6   7   8   9   10   >