from:"Christopher Schultz"


Jon,

On 9/4/23 10:41, Jonathan S. Fisher wrote:

Mark thank you again for your leadership and setting expectations. I'm
going to commit to working on this with anyone else that wants to help with
the goal of a patch by year end. I want to nail the patch with minimal
rework that meets Tomcat project quality standards. To that end, I'll
attempt to summarize what you expect here and if you could comment and
correct my understanding that would be appreciated.

It sounds like you're satisfied with the ubiquity of the Proxy protocol and
that it has an RFC


+1


We'll target just implementing the latest version of the Proxy protocol
We'll implement a "TrustedProxies" feature similar to what the Remote IP
Valve does


+1 This is very important. If it makes sense to do some refactoring to 
allow these features to share code, please do that as a completely 
separate commit in your PR. It will make things much easier to read and 
follow.



We'll implement a, or modify the RemoteIp, valve to be able to set the
remote IP from Proxy protocol headers
We'll follow the RFC spec and reject any request that does a proper Proxy
protocol header
I'm particularly interested in the Proxy protocol over Unix Domain Sockets,
so expect to see a lot of the work focused on this, but accepting Proxy
Protocol over TCP looks to be quite important from the comments on this
email chain


TCP is a much more important use-case IMHO.


If I may ask two things:
Can you summarize your desired implementation? What point in the stack
should we target to implement this?
One thing I'm not familiar with on Tomcat is the testing expectations. If
you can point to a set of unit tests and a set of integration tests and say
"Do it like this"


There are tons of unit tests in the Tomcat source tree. IMHO it's much 
more important to have /existing/ unit-tests than waiting for a long 
time to have "the perfect tests case(s)". If you commit decent 
unit-tests and there are opportunities to make them more "Tomcat-like" 
or share more code with existing unit-tests, that work can come later.



Anything else on the original patch you liked/didn't like? (
https://bz.apache.org/bugzilla/show_bug.cgi?id=57830)


Others know much more about the "right" way to get started on this, so 
I'll leave it to them to reply. I wouldn't want to lead you down the 
wrong path.


-chris


On Tue, Aug 29, 2023 at 3:13 PM Mark Thomas  wrote:


On 28/08/2023 18:44, Amit Pande wrote:

Oh, sure. So, what would be the best way to get some conclusion on this

thread?

Provide a patch for review based on the feedback provided here and in
the BZ issue.


https://bz.apache.org/bugzilla/show_bug.cgi?id=57830 The state of the

ticket isn't updated for long. Perhaps add comments/ask the folks on user
list to vote?

That is more likely to irritate folks rather than encourage them to help
you progress your patch.

Mark




Thanks,
Amit

-Original Message-
From: Mark Thomas 
Sent: Monday, August 28, 2023 11:20 AM
To: Tomcat Users List 
Subject: Re: [External] Re: Supporting Proxy Protocol in Tomcat


CAUTION: This email originated from outside the organization. Do not

click links or open attachments unless you recognize the sender and know
the content is safe. If you believe this is a phishing email, use the
Report to Cybersecurity icon in Outlook.




28 Aug 2023 17:11:20 Amit Pande :


Mark,

Just checking - Did this issue get discussed in any of the core
members' meeting?


There are no such meetings. Discussion happens on the mailing lists.

Mark




Thanks,
Amit

-Original Message-
From: Amit Pande
Sent: Monday, July 31, 2023 9:29 AM
To: Tomcat Users List 
Subject: RE: [External] Re: Supporting Proxy Protocol in Tomcat

Yes, understood.

Thank you for clarifying. Even I was referring to initial consensus
without any timeline or approach conclusion.

Thanks,
Amit

-Original Message-
From: Mark Thomas 
Sent: Friday, July 28, 2023 2:48 PM
To: users@tomcat.apache.org
Subject: Re: [External] Re: Supporting Proxy Protocol in Tomcat

On 28/07/2023 19:21, Amit Pande wrote:

Thank you all for the valuable discussion on this topic.

Is it okay to say that we're agreeing to adding proxy protocol
support in Tomcat?


I think that is a little too strong. At this point there is a proposed
approach and no one is objecting but until there is an actual patch to
discuss...

Keep in mind that any committer can veto a change.

My sense is that it should be possible to implement this feature while
addressing any concerns that may be raised but it is not guaranteed.

Mark




Thanks,
Amit

-Original Message-
From: Christopher Schultz 
Sent: Thursday, July 27, 2023 4:13 PM
To: users@tomcat.apache.org
Subject: Re: [External] Re: Supporting Proxy Protocol in Tomcat

All,

On 7/27/23 12:39, Mark Thomas wrote:

On 27/07/2023 16:27, Jonathan S. Fisher

Virtual Threads


All,

I have some questions about Virtual Threads and their use within Tomcat. 
Note that only Tomcat 11 currently has support for Virtual Threads when 
running on a version 19 or later JVM.


My (admittedly limited) understanding is that the use of Virtual 
Threads, specifically within Tomcat, will allow Tomcat to use fewer 
platform threads (e.g. native OS threads) to support a larger number of 
JVM/application threads for request-processing. The justification for 
this improvement is that request-processing threads spend a lot of time 
in an I/O-blocked state which, when using Virtual Threads, would no 
longer consume a platform thread.


There appears to only be one setting to enable Virtual Threads in Tomcat:



or



In both cases, there is only one setting to affect the number of 
"threads" (by any description) which the Executor will ultimately use 
(Connectors without an explicit Executor will create a non-shared 
Executor to be used internally). That setting is "maxThreads" which 
limits the total number of "threads" that the Executor will create.


The implementation of the VirtualThreadExecutor does not seem to have 
any upper-bound on the number of Virtual Threads that will be created at 
all. I believe this means that a large number of incoming requests (i.e. 
a burst) will result in the creation of a large number of Virtual 
Threads. Without an upper-bound, we are expecting that the JVM's 
(virtual) thread scheduler will schedule each application thread to be 
mounted to a platform thread in the order it was added to the queue 
(essentially in request-order).


Before Virtual Threads were introduced, the Executor would use a queue 
of requests to dispatch to available (non-Virtual) threads which would 
use that thread until the request was complete, then return the thread 
to the pool. With Virtual Threads, the same thing is essentially still 
happening except that (a) Tomcat no longer manages the thread pool (a 
JVM-defined one is being used instead) and (b) requests are immediately 
handed a (Virtual) thread to carry their execution.


I believe there are some subtle differences in how Tomcat will behave, 
now. As an example, if I have two applications running, say, the Tomcat 
Manager application and the Tomcat Examples application, without using 
Virtual Threads, each application's thread pools should be "fair" within 
the context of each application: requests are processed in the order 
they are received in Manager and Examples, separately. If all requests 
are equally "expensive" and everything is "fair", then requests to the 
Examples application are scheduled alongside those to the Manager 
application, and they can both execute simultaneously as well as 
separately-manage the order in which the requests are processed.


Once Virtual Threads are introduced, the requests are filed into a 
single JVM-wide thread-scheduling system where activity in one 
application can affect the other.


Let's replace Examples with RealWorldApp, an application that is 
expected to be used by users. Maybe a LOT of users. Without Virtual 
Threads, a high number of requests to RealWorldApp will not cause 
starvation of requests to (maybe the more important, at least for 
admins) the Manager. Once Virtual Threads are introduced, a limitless 
number of requests can be queued in front of a request to Manager, which 
can experience starvation.


While Tomcat did not previously implement any specific priority-queuing 
of requests, the use of separate Executors for each application 
effectively provided that kind of thing. Yes, each Executor can be 
configured separately to either use Virtual Threads or not, and so 
Manager can be configured to NOT use Virtual Threads while RealWorldApp 
can be configured to use Virtual Threads and the balance is "restored". 
But it is no longer possible to have RealWorldApp and RealWorldApp2 and 
Manager all with equally probable request-scheduling when using Virtual 
Threads. You can pick some subset of applications to get (essentially) 
priority by NOT using Virtual Threads, but the whole set of applications 
running in a single JVM will share a single Executor with FIFO behavior. 
If an application creates Virtual Threads (hey, why not?! they are 
cheaper!) then they will be scheduled alongside the request-processing 
threads as well.


Do I understand things correctly? Is my scenario of request-starvation 
for a little-used but potentially critical application (e.g. Manager) a 
real concern, or am I missing something fundamental about the way 
Virtual Threads are scheduled relative to other Virtual Threads, and 
relative to non-virtual threads?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: CIS Tomcat 8 Benchmark (v1.1.0) -- Questions


Mark, Robert, et al,

On 9/5/23 11:26, Mark Thomas wrote:
I spoke to some CIS representatives at a recent conference given that I 
have concerns about the quality of some of the recommendations.


It appears that these benchmarks are effectively crowdsourced. My 
primary concern is that there is no validation of the resulting 
recommendations so they run the risk of reinforcing current widespread 
bad practice as well as good practice.


This is interesting information. Lore really isn't a good source of 
security information. :( It makes me wonder why they are charging for 
that kind of information if it's basically the output of an internet 
search for "how do i secure a Tomcat installation".


Regarding the suggestion to edit the contents of a JAR, that is a bad 
idea on so many levels.


+1

Assuming that you are using signed JAR files, what you end up with is a 
JAR whose signature can no longer be validated. This would require that 
you disable JAR signing, which is probably another no-no when it comes 
to security -- and I'd agree -- if you have signed JARs you ought to be 
validating them. Another option would be to re-sign the modified JAR 
file with an internal certificate but that can present its own 
challenges. Most dev teams would just implement some kind of lazy 
auto-signing process with the cert and key sitting right on the server 
where it's being used, and at that point you are giving an attacker 
everything they need to work-around your security controls.


I'd argue that rather than spending time hiding the current Tomcat 
version - which is nothing more than security by obscurity - system 
admins should be investing time in an update process that allows easy 
updates (and roll-backs) to the Tomcat version. You only need to hide 
the version number if you aren't on top of your security updates. And if 
you aren't on top of your security updates you have bigger problems than 
needing to hide the version number.


+1

https://tomcat.apache.org/presentations.html#latest-split-installation

If you find you need to hide this version number to appease auditors - 
and using smarter auditors isn't an option - then there are a range of 
options as set out in the Tomcat 8.5 security guide. That guide also 
provides the correct way to override the version number (if you really 
need to) without editing the JAR contents. In short, you can simply 
override the individual file by placing at the right place in the file 
system:


$CATALINA_BASE/lib/org/apache/catalina/util/ServerInfo.properties


+1

The only argument /against/ doing that in the past that I've ever seen 
was that they "didn't want to pollute [their] Tomcat installation" which 
is crazy when the alternative is to alter a signed JAR provided by a 
trusted vendor.


This is also probably worth a read:
https://cwiki.apache.org/confluence/display/TOMCAT/Community+Review+of+DISA+STIG

I would bet that many of the items listed here have overlap with some of 
the more dubious items coming from CIS.


Hope that helps,
-chris



On 05/09/2023 14:54, Robert Turner wrote:
Thanks Peter. Just to be clear that I have no concern about the goal 
of the

test -- only the method for obtaining the information, and the "implied"
correction.

After going through the rest of the document I was provided, it seems to
"get more modern" as the questions go on -- suggesting the method of
improvement is additive, and possibly not corrective.


Improvements are definitely corrective as well as additive. Early 
versions of the guide had very odd advice regarding MIME type mapping 
that has since been removed.





On Tue, Sep 5, 2023 at 9:36 AM Peter Kreuser  wrote:


Robert,

While Mark Thomas will have a more detailled answer to this...

The finding behind this test is valid (information disclosure with 
server

version in responses), though the remediation listed here is from looong
time ago, when the was no ErrorReportValve to purge the version info.

So the CIS Tomcat 8(!) Guide is pretty outdated! Probably in more than
this spot...

Peter


Am 05.09.2023 um 14:03 schrieb Robert Turner :

While I think I know the answer to my question, I wanted to 
double-check

with the group to confirm.

I have been asked to perform the CIS Apache Tomcat 8 Benchmark (v1.1.0)

on

our production Tomcat installation, and I am looking through the

questions

/ information extraction requests, and I suspect they are not really
evaluating what they think they are, and furthermore encouraging bad
practices.

For instance, the first entry I have in the spreadsheet I was 
provided is

listed as follows:

CIS Control:
2.1 Alter the Advertised server.info String (Scored)

Description:
The server.info attribute contains the name of the application service.
This value is presented to Tomcat clients when clients connect to the
tomcat server.

Audit Procedures:
Perform the following to determine if the server.info value has been
changed:
Extract the ServerInfo.properties file and exami

Re: Virtual Threads


Mark,

On 9/5/23 15:55, Mark Thomas wrote:

On 05/09/2023 20:38, Christopher Schultz wrote:

All,

I have some questions about Virtual Threads and their use within 
Tomcat. Note that only Tomcat 11 currently has support for Virtual 
Threads when running on a version 19 or later JVM.


Not quite. All current versions support virtual threads when running on 
Java 21 or later.


Thanks for the correction. I just did a quick docs[1] search for 
"virtual" in Tomcat 10.x for example and I didn't see useVirtualThreads, 
so I assumed it wasn't in there. Maybe we need some documentation 
back-ports?


My (admittedly limited) understanding is that the use of Virtual 
Threads, specifically within Tomcat, will allow Tomcat to use fewer 
platform threads (e.g. native OS threads) to support a larger number 
of JVM/application threads for request-processing. The justification 
for this improvement is that request-processing threads spend a lot of 
time in an I/O-blocked state which, when using Virtual Threads, would 
no longer consume a platform thread.


There appears to only be one setting to enable Virtual Threads in Tomcat:



or



In both cases, there is only one setting to affect the number of 
"threads" (by any description) which the Executor will ultimately use 
(Connectors without an explicit Executor will create a non-shared 
Executor to be used internally). That setting is "maxThreads" which 
limits the total number of "threads" that the Executor will create.


The implementation of the VirtualThreadExecutor does not seem to have 
any upper-bound on the number of Virtual Threads that will be created 
at all. I believe this means that a large number of incoming requests 
(i.e. a burst) will result in the creation of a large number of 
Virtual Threads. Without an upper-bound, we are expecting that the 
JVM's (virtual) thread scheduler will schedule each application thread 
to be mounted to a platform thread in the order it was added to the 
queue (essentially in request-order).


The virtual thread scheduler uses a work stealing queue so it isn't 
quite FIFO.


That sounds like nit-picking to me. I suppose if each new Virtual Thread 
is assigned to a platform thread when it's initially queued, things can 
be executed in an not-strictly-FIFO-manner, but one has to assume work 
loads are equal (which they likely are not) and work is spread evenly 
across all the platform threads (which is likely) to have a general 
conversation about all this. I think the work-stealing ForkJoinPool is 
about as close to FIFO as you can get without introducing more expensive 
contention in order to enforce strict FIFO while not gaining much in the 
way of meaningful "fairness".


But it's probably worth mentioning that the queue might not be "strictly 
fair". What *is* fair, though -- and maybe more fair than in the past? 
-- is that pipelined requests have to get back in line instead of 
jumping the queue. I think the old blocking JIO connector would allow 
pipelined requests to skip the queue, but all NIO-based connectors are 
"more fair" than that.


Before Virtual Threads were introduced, the Executor would use a queue 
of requests to dispatch to available (non-Virtual) threads which would 
use that thread until the request was complete, then return the thread 
to the pool. With Virtual Threads, the same thing is essentially still 
happening except that (a) Tomcat no longer manages the thread pool (a 
JVM-defined one is being used instead) and (b) requests are 
immediately handed a (Virtual) thread to carry their execution.


I believe there are some subtle differences in how Tomcat will behave, 
now. As an example, if I have two applications running, say, the 
Tomcat Manager application and the Tomcat Examples application, 
without using Virtual Threads, each application's thread pools should 
be "fair" within the context of each application: requests are 
processed in the order they are received in Manager and Examples, 
separately. If all requests are equally "expensive" and everything is 
"fair", then requests to the Examples application are scheduled 
alongside those to the Manager application, and they can both execute 
simultaneously as well as separately-manage the order in which the 
requests are processed.


The above assumes each application has a separate thread pool (which 
implies a separate Connector).


Yes, I'm sorry, I completely skipped over the "fact" that a separate 
 would be used for such a "priority" application such as the 
Manager. My example was assuming that each application had the 
possibility to use a separate .


Once Virtual Threads are introduced, the requests are filed into a 
single JVM-wide thread-scheduling system where activity in one 
application can affect the other.


Correct.

Let's replace Examples with RealWor

Re: Virtual Thread Configuration In Tomcat 11


William,

On 8/24/23 09:50, William Crowell wrote:

I did some performance testing with virtual threads on Apache Tomcat
11.0.0-M10 and JDK 21 (21+35-2513).  I have a simple REST service
using Spring 6.0.11 that does an insert into MySQL 8.0.32.

I have 3 separate boxes all running Rocky Linux 9.2 on AWS
(t3a.xlarge which is 4 vCPUs and 16GiB RAM):

1) An application server running Tomcat 11.0.0-M10 with JDK 21

2) MySQL 8.0.32

3) JMeter 5.6.2

I know that JDK 21 is nowhere near GA, but I got some interesting
results.  I did 10 test runs with useVirtualThreads=“true”, and 10
test runs without virtual threads (the default).  Each run used 1000
threads in JMeter for about 3 minutes and 20 seconds for a ramp up
time of 3 seconds.

What I found was using platform threads performed at least twice as
fast as virtual threads.  Maybe I am interpreting the results wrong,
but this is what I found consistently across each test.

Again, I realize that performance is not the only goal of virtual
threads.  Just my observations.


This is "interesting", for some values of interesting.

I'd be interested in what the application was doing. Virtual Threads 
seem to be aimed at things like Web Application Containers where there 
is a lot of wasted time in a thread's execution waiting on I/O 
(blocking). If you ignore the application's workload (hah!), the 
application server is almost entirely occupied with (a) reading the 
request and (b) writing the response. There is almost no "computation 
time" and so Virtual Threads make a whole lot of sense.


Once inside the application, that may not be true.

From what little reading I have done on Virtual Threads, there are some 
poison pills in there depending upon application design. For example, if 
your thread blocks while inside a synchronized{} block, then your 
Virtual Thread is "pinned" and will NOT be un-mounted from the platform 
thread during the blocking operation. So all of that high-speed 
context-switching and magic that VT is supposed to provide goes 
completely out the window when you used to have 200 threads with maybe 8 
of them actually running at once but swapping-out preemptively, but now 
you have 200 virtual threads with 8 of them running at once, but of the 
8 running at once, several of them are stopped dead unable to make any 
progress.


At $work, we have plenty of read-through cache routines like this:

private Object somethingBigLock = new Object();
private SomethingBig big = null;
public Object getSomethingBigFromDatabase() {
synchronized(somethingBigLock) {
  if(null == big) {
  big = reallyGetSomethingBigFromDb(); // Non-trivial
  }

  return big;
}
}

Even if reallyGetSomethingBigFromDb() is mostly blocking on I/O -- e.g. 
waiting for the db to respond to a query -- it will consume one of your 
platform threads and lock it up, leaving fewer platform threads 
available to do the other work.


Without a critical analysis of your application and the libraries it 
uses, it's hard to tell if what you are observing is that "virtual 
threads are not as fast as regular threads" or "your application is an 
anti-pattern for Virtual Threads". My guess is that your application 
and/or the libraries you are using need to be re-written so they no 
longer use "classic" synchronization and instead use things like 
ReentrantLock for a similar purpose to play-nice with VT. Or you can 
wait for the (likely) inevitable improvement of the JVM which allows for 
threads holding monitors to be unmounted from platform threads. My guess 
is that is in the future for the JVM.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Virtual Thread Configuration In Tomcat 11

2023-09-06 Thread Christopher Schultz


William,

On 9/5/23 17:41, William Crowell wrote:

Great post earlier today!  This is a super interesting topic to me.

You can find the performance testing results located here: 
http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/

I did 10 runs with 1000 threads with a ramp up time of 3 seconds for a duration 
of 200 seconds.  Ignore the 11th run.  This is a really simple REST application 
with Spring.  It just makes an insert into a MySQL table.  When I get a moment 
I will put my code into GitHub, but again, there is not much to it.  You can 
find a very similar Spring Boot example with embedded Tomcat here:

https://medium.com/@anil.java.story/embracing-virtual-threads-in-spring-boot-4140d3b8a5a

But he uses JDK 20 with --enable-preview turned on.

The mistake I think I made was that the inserts are not really blocking because 
they happen so quickly.  Maybe because I did not put enough load in the test?  
I have no synchronized blocks in my code.  So, it appeared to me that the 
inserts were taking twice the amount of time with virtual threads.  I think 
that extra time is just overhead of pinning virtual threads to a platform 
thread and managing the ForkJoinPool.

What is going to happen is that people are going to “flip the 
switch”…useVirtualThreads=”true”.  They will find that performance will not be 
as good as using an operating system thread and abandon virtual threads because 
their code does not block.

So, I need to come up with an example where my code blocks and redo the perf 
test and prove this out.  I don’t know.


Try setting this system property before running your tests and see if it 
produces any output on the server while the load test is running:


-Djdk.tracePinnedThreads=full

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Virtual Threads

2023-09-06 Thread Christopher Schultz


Mark,

On 9/6/23 03:29, Mark Thomas wrote:

On 05/09/2023 22:02, Christopher Schultz wrote:

Mark,

On 9/5/23 15:55, Mark Thomas wrote:

On 05/09/2023 20:38, Christopher Schultz wrote:

All,

I have some questions about Virtual Threads and their use within 
Tomcat. Note that only Tomcat 11 currently has support for Virtual 
Threads when running on a version 19 or later JVM.


Not quite. All current versions support virtual threads when running 
on Java 21 or later.


Thanks for the correction. I just did a quick docs[1] search for 
"virtual" in Tomcat 10.x for example and I didn't see 
useVirtualThreads, so I assumed it wasn't in there. Maybe we need some 
documentation back-ports?


Odd. It is there as an attribute on the Connector. And in 9.0.x and 
8.5.x too.


https://tomcat.apache.org/tomcat-10.0-doc/config/executor.html

I don't find the word "virtual" anywhere on that page.

The virtual thread scheduler uses a work stealing queue so it isn't 
quite FIFO.


That sounds like nit-picking to me.


It is, but it is nit-picking for a reason. Violetta did some testing 
which showed switching to virtual threads showed a small (about 1%) 
throughput improvement it also showed a much broader spread of 
latencies. That increase in spread is due to the work-stealing queue. 
For those users watch latency carefully, the change was viewed as a 
significant drawback of switching to virtual threads.


That's a significant point, thanks for reiterating it.




I think you have summed things up pretty well. I don't see a way with 
the current API to specify multiple virtual thread schedulers (which 
is what I think you would need to address this).


Yeah, in all of the coverage I've seen online, there are no examples 
where Virtual Threads are used with an "executor" other than one that 
(a) accepts Runnable tasks and (b) just generates a Virtual Thread 
from that task which appears to be "bound" to the build-in default 
JVM-wide Virtual Thread executor.


It would be potentially interesting to see an API which would allow 
different Executors to be used with Virtual Threads, even though the 
whole point of the entire thing is to avoid the (application-level) 
complexity of a thread pool/executor/etc. I think it does make sense, 
however, to be able to prioritize your threads a little bit. I haven't 
read anything about Virtual Threads and their priorities, so I assume 
it's just not part of anything user-facing at this point.


I haven't seen anything that suggests that there will be any such API 
but to be fair, I haven't been following the Loom project that closely.


For those of you that are finding this topic interesting, I'll have a 
talk on this at the ASF conference, Community Over Code in Halifax.


https://communityovercode.org/schedule-list/#TH001

More broadly, there are usually a reasonable number of Tomcat committers 
present at ASF conferences so this is a great opportunity to speak face 
to face with the committers.


+1

-chris


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Virtual Threads

2023-09-07 Thread Christopher Schultz


Mark,

On 9/6/23 16:29, Mark Thomas wrote:

On 06/09/2023 21:24, Christopher Schultz wrote:

On 9/6/23 03:29, Mark Thomas wrote:

On 05/09/2023 22:02, Christopher Schultz wrote:




Thanks for the correction. I just did a quick docs[1] search for 
"virtual" in Tomcat 10.x for example and I didn't see 
useVirtualThreads, so I assumed it wasn't in there. Maybe we need 
some documentation back-ports?


Odd. It is there as an attribute on the Connector. And in 9.0.x and 
8.5.x too.


https://tomcat.apache.org/tomcat-10.0-doc/config/executor.html

I don't find the word "virtual" anywhere on that page.


It is a Connector attribute, not an Executor attribute. There isn't much 
point using an executor with virtual threads.


Okay then perche 
https://tomcat.apache.org/tomcat-11.0-doc/config/executor.html#Virtual_Thread_Implementation 
?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Virtual Thread Configuration In Tomcat 11

2023-09-07 Thread Christopher Schultz


William,

On 9/7/23 08:04, William Crowell wrote:

When I set -Djdk.tracePinnedThreads=short, then I see this:

…
Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
 com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== 
monitors:1
Thread[#39,ForkJoinPool-1-worker-2,5,CarrierThreads]
 com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== 
monitors:1
Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
 com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) 
<== monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
 
com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:893)
 <== monitors:1
 
com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061)
 <== monitors:1
 
com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009)
 <== monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
 com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791) <== 
monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
 com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) 
<== monitors:1
…

I started digging into the MySQL client code: 
https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/ConnectionImpl.java

This class is littered with synchronized blocks, so now this makes sense to me 
why virtual threads would take longer than a regular OS thread.


This is precisely the kind of thing I was expecting to see.

If your "application code" (which includes everything your application 
does, including calling into libraries) contains lots of "synchronized" 
blocks, then I think this is the situation you will often find yourself in.


I would expect that if you were to increase the size of the virtual 
thread pool things would start to get better, but in order to truly use 
virtual threads effectively, you have to track-down and eliminate many 
uses of synchronized blocks and methods in both your application and the 
libraries you use.


I wouldn't expect virtual threads to become a very widely-deployed 
performance-optimization technique in the immediate future. I expect 
that, eventually, lots of code will be re-written to be more 
virtual-thread-friendly.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: question about tomcat manager Server Status page

2023-09-08 Thread Christopher Schultz


Ivano,

On 9/8/23 11:17, Ivano Luberti wrote:

Hi, looking at Server Status and Complete Server Status Page

I can see the following line:

Max threads: 200 Current thread count: 11 Current threads busy: 1 Keep 
alive sockets count: 1


But looking at the thread list under the line I can count 24 lines.

So what is the number of thread currently instantiated by tomcat? 11 or 24?


This is a good question. When I check my localhost Manager running 
8.5.x, I see this:


Max threads: -1 Current thread count: 4 Current threads busy: 1 Keep 
alive sockets count: 1


The number of threads shown in the http-nio-host-port section shows 5 
threads, 4 in the R state and one in the S state.


When running jstack against my JVM, I can see that there are only 4 exec 
threads running.


So I think the claim that there are only 11 threads in your JVM is 
correct. I believe the 24 lines you are seeing are something buggy in 
the Manager's view. I'll see if I can play around with it a little bit 
to see what's happening.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Enhancement request?

2023-09-08 Thread Christopher Schultz


Jon,

On 9/8/23 14:21, Mcalexander, Jon J. wrote:

In seeing the latest messages about the manager application, something that I and my team 
would LOVE to have is just a Status app that provides all the items status wise that the 
Manager app does, without any of the "Application Management" like restarting 
an app, etc. I know all the pieces are out there in Catalina.jar, but I don't have the 
developer knowledge to put it together in a separate servlet that just calls the items 
needed from Catalina.

Is this something that any of the guru's have thought about putting together? 
I'm thinking it would be best to just be a web service, no gui, that you can 
call and get the json or xml output with the data so it can be incorporated 
into a dashboard.


Are you looking for...

1. Something with more limited capabilities (for less-trusted admins, or 
to reduce the possibilities for hacking)


2. Something with fewer distractions

3. Something with a JSON/XML interface instead of HTML

4. Something which has fewer lines of code

?

Or some combination of the above, or something else?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Enhancement request?

2023-09-11 Thread Christopher Schultz


Jon,

On 9/8/23 17:09, Mcalexander, Jon J. wrote:

It would be something that combines all 4 of your items below. I
would be looking for something that can just give health status of
Tomcat and the apps being hosted by that instance. This
wouldn't/shouldn't have any "Admin" rights to do anything other than
provide info. HTML is fine, but I think from an automation and
dashboard reporting standpoint the json/xml stream/soap
response/whatever would be best.
Does the existing non-GUI Manager interface not already provide what you 
are looking for? Or maybe JMX (possibly via the JMXProxyServlet)? Direct 
access to JMX /does/ mean that the connecting client has very high 
privileges, though.



Now, in my possibly short sited view of the world, if wishes were
fishes type of thing, to ME it would be awesome if this was
automagically available via startup without having to do anything
special in the server.xml to make that app available only to
localhost on such and such port. Know what I mean? An instant
statement of health localhost URL. 😊
I think the term "health[y]" might mean different things to different 
people. I think if you need something to be available that reports 
healthiness of your own service and application, you should probably 
build-to-suit.


-chris


-----Original Message-
From: Christopher Schultz 
Sent: Friday, September 8, 2023 3:46 PM
To: users@tomcat.apache.org
Subject: Re: Enhancement request?

Jon,

On 9/8/23 14:21, Mcalexander, Jon J. wrote:

In seeing the latest messages about the manager application, something

that I and my team would LOVE to have is just a Status app that provides all
the items status wise that the Manager app does, without any of the
"Application Management" like restarting an app, etc. I know all the pieces
are out there in Catalina.jar, but I don't have the developer knowledge to put
it together in a separate servlet that just calls the items needed from
Catalina.


Is this something that any of the guru's have thought about putting

together? I'm thinking it would be best to just be a web service, no gui, that
you can call and get the json or xml output with the data so it can be
incorporated into a dashboard.

Are you looking for...

1. Something with more limited capabilities (for less-trusted admins, or to
reduce the possibilities for hacking)

2. Something with fewer distractions

3. Something with a JSON/XML interface instead of HTML

4. Something which has fewer lines of code

?

Or some combination of the above, or something else?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: page extends not working???

2023-09-11 Thread Christopher Schultz


Aryeh,

On 9/9/23 19:36, Aryeh Friedman wrote:

On Sat, Sep 9, 2023 at 1:23 PM Mark Thomas  wrote:


On 09/09/2023 11:52, Aryeh Friedman wrote:

Every other jsp in my webapp (and other webapps on the same tomcat
instance [9.0.75]) works and I am using a the default container but as
curl/catalina.out show BasePage is *NEVER* being called (either the
_jspService() or the getX()):


How have you configured your JSP(s) to use this alternative base class?


sudo cat /usr/local/apache-tomcat-9.0/webapps/tlaitc-dashboard-1a1/index.jsp

<%@page extends="dashboard.web.pages.BasePage"%>
hi x is ${x}

Output shown in log (sorry for not including the JSP the first time)
but to make it easier to find the output is "hi x is " when it should
be "hi x is 123234"... as you notice there are zero errors/warning in
catalina but there is none of the println's also... so the only thing
I can surmise is BasePage is never being called <%@page
extends="dashboard.web.pages.BasePage"%> somehow failed but I have
verified that correct spelling several times and also verified any
syntextual errors [including the contents of the string literal] will
show up in catalina.out (i.e. wrong class name is logged as an error)


Your _jspService method in your base class will never be called, because 
it is overridden by the auto-generated class for your JSP, which does 
not call super._jspService.


I do not believe that this:

Hi X is ${x}

...will result in this.getX() being called from the JSP. References in 
EL ${...} expressions will be resolved against the PageContext (and 
other wider contexts), not against the JSP class currently executing.


If you want to call this.getX() then you will need to do this:

Hi X is <% getX() %>

I wouldn't bother messing-around with class hierarchies in JSP. It 
usually doesn't get you much but almost always requires you to bind 
yourself very closely with the specific implementation of the JSP engine.


It would be far better to use typical MVC-style development where a 
servlet (or similar) handles the real work of the request, possibly 
including writing a value of "x" to the request attributes. Then forward 
your request to your JSP to produce the response content. This will be 
much more straightforward and you will have fewer problems like this, 
where expected behavior is confused by all the magic that JSP provides.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: page extends not working???

2023-09-12 Thread Christopher Schultz


Aryeh,

On 9/11/23 10:05, Aryeh Friedman wrote:

On Mon, Sep 11, 2023 at 9:47 AM Christopher Schultz
 wrote:


Aryeh,

On 9/9/23 19:36, Aryeh Friedman wrote:

On Sat, Sep 9, 2023 at 1:23 PM Mark Thomas  wrote:


On 09/09/2023 11:52, Aryeh Friedman wrote:

Every other jsp in my webapp (and other webapps on the same tomcat
instance [9.0.75]) works and I am using a the default container but as
curl/catalina.out show BasePage is *NEVER* being called (either the
_jspService() or the getX()):


How have you configured your JSP(s) to use this alternative base class?


sudo cat /usr/local/apache-tomcat-9.0/webapps/tlaitc-dashboard-1a1/index.jsp

<%@page extends="dashboard.web.pages.BasePage"%>
hi x is ${x}

Output shown in log (sorry for not including the JSP the first time)
but to make it easier to find the output is "hi x is " when it should
be "hi x is 123234"... as you notice there are zero errors/warning in
catalina but there is none of the println's also... so the only thing
I can surmise is BasePage is never being called <%@page
extends="dashboard.web.pages.BasePage"%> somehow failed but I have
verified that correct spelling several times and also verified any
syntextual errors [including the contents of the string literal] will
show up in catalina.out (i.e. wrong class name is logged as an error)


Your _jspService method in your base class will never be called, because
it is overridden by the auto-generated class for your JSP, which does
not call super._jspService.

I do not believe that this:

Hi X is ${x}

...will result in this.getX() being called from the JSP. References in
EL ${...} expressions will be resolved against the PageContext (and
other wider contexts), not against the JSP class currently executing.

If you want to call this.getX() then you will need to do this:

Hi X is <% getX() %>

I wouldn't bother messing-around with class hierarchies in JSP. It
usually doesn't get you much but almost always requires you to bind
yourself very closely with the specific implementation of the JSP engine.

It would be far better to use typical MVC-style development where a
servlet (or similar) handles the real work of the request, possibly
including writing a value of "x" to the request attributes. Then forward
your request to your JSP to produce the response content. This will be
much more straightforward and you will have fewer problems like this,
where expected behavior is confused by all the magic that JSP provides.


Thanks but I have a very specific use case which the following working
example below should make more clear:


<%@page import="dashboard.web.page.Page"%>
<%
 // THIS WOULD NOT BE NEEDED if <%@page extends="..."%> worked
 //
 // for now we don't need to keep the page object just
 // the setAttributes in our ctx
 new Page(pageContext);
%>



 <%@include file="/widgets/scripts/scripts.jsp"%>


  



and the Page class:

package dashboard.web.page;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.jsp.PageContext;

import org.petitecloud.util.PetiteCloudNullException;

// making this extend the right subclass plus working page
extends="" would mean zero in-line Java
// Copyright (C) 2023 TLAITC and contributors
public class Page
{
 public Page(PageContext ctx)
 {
 _dbc_construction(ctx);

 this.ctx=ctx;

 HttpServletRequest req=(HttpServletRequest) ctx.getRequest();
 String[] parts=req.getRequestURI().split("/");
 int split=2;

 if(parts[0].equals("http:"))
 split+=2;

 name="";
 for(int i=split;i
<%@page extends="dashboard.web.page.Page"%>



 <%@include file="/widgets/scripts/scripts.jsp"%>


  



Side note I am currently adding user detection to the above page class
to that it can auto-enforce ACL's


I'm not sure I understand the point of the whole exercise. Nothing you 
have in the PageContext class constructor cannot be done from within the 
JSP itself, or -- better yet -- in an <%include> used by as many pages 
as you want. *OR* ... you could do it in a Filter and apply it to all 
requests, storing the information in the request attributes, which is 
much more standard than directly modifying the page context.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: page extends not working???

2023-09-12 Thread Christopher Schultz


Aryeh,

On 9/12/23 12:42, Aryeh Friedman wrote:

On Tue, Sep 12, 2023 at 11:42 AM Christopher Schultz
 wrote:


Aryeh,

On 9/11/23 10:05, Aryeh Friedman wrote:

On Mon, Sep 11, 2023 at 9:47 AM Christopher Schultz
 wrote:


Aryeh,

On 9/9/23 19:36, Aryeh Friedman wrote:

On Sat, Sep 9, 2023 at 1:23 PM Mark Thomas  wrote:


On 09/09/2023 11:52, Aryeh Friedman wrote:

Every other jsp in my webapp (and other webapps on the same tomcat
instance [9.0.75]) works and I am using a the default container but as
curl/catalina.out show BasePage is *NEVER* being called (either the
_jspService() or the getX()):


How have you configured your JSP(s) to use this alternative base class?


sudo cat /usr/local/apache-tomcat-9.0/webapps/tlaitc-dashboard-1a1/index.jsp

<%@page extends="dashboard.web.pages.BasePage"%>
hi x is ${x}

Output shown in log (sorry for not including the JSP the first time)
but to make it easier to find the output is "hi x is " when it should
be "hi x is 123234"... as you notice there are zero errors/warning in
catalina but there is none of the println's also... so the only thing
I can surmise is BasePage is never being called <%@page
extends="dashboard.web.pages.BasePage"%> somehow failed but I have
verified that correct spelling several times and also verified any
syntextual errors [including the contents of the string literal] will
show up in catalina.out (i.e. wrong class name is logged as an error)


Your _jspService method in your base class will never be called, because
it is overridden by the auto-generated class for your JSP, which does
not call super._jspService.

I do not believe that this:

Hi X is ${x}

...will result in this.getX() being called from the JSP. References in
EL ${...} expressions will be resolved against the PageContext (and
other wider contexts), not against the JSP class currently executing.

If you want to call this.getX() then you will need to do this:

Hi X is <% getX() %>

I wouldn't bother messing-around with class hierarchies in JSP. It
usually doesn't get you much but almost always requires you to bind
yourself very closely with the specific implementation of the JSP engine.

It would be far better to use typical MVC-style development where a
servlet (or similar) handles the real work of the request, possibly
including writing a value of "x" to the request attributes. Then forward
your request to your JSP to produce the response content. This will be
much more straightforward and you will have fewer problems like this,
where expected behavior is confused by all the magic that JSP provides.


Thanks but I have a very specific use case which the following working
example below should make more clear:


<%@page import="dashboard.web.page.Page"%>
<%
  // THIS WOULD NOT BE NEEDED if <%@page extends="..."%> worked
  //
  // for now we don't need to keep the page object just
  // the setAttributes in our ctx
  new Page(pageContext);
%>



  <%@include file="/widgets/scripts/scripts.jsp"%>


   



and the Page class:

package dashboard.web.page;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.jsp.PageContext;

import org.petitecloud.util.PetiteCloudNullException;

// making this extend the right subclass plus working page
extends="" would mean zero in-line Java
// Copyright (C) 2023 TLAITC and contributors
public class Page
{
  public Page(PageContext ctx)
  {
  _dbc_construction(ctx);

  this.ctx=ctx;

  HttpServletRequest req=(HttpServletRequest) ctx.getRequest();
  String[] parts=req.getRequestURI().split("/");
  int split=2;

  if(parts[0].equals("http:"))
  split+=2;

  name="";
  for(int i=split;i
<%@page extends="dashboard.web.page.Page"%>



  <%@include file="/widgets/scripts/scripts.jsp"%>


   



Side note I am currently adding user detection to the above page class
to that it can auto-enforce ACL's


I'm not sure I understand the point of the whole exercise. Nothing you
have in the PageContext class constructor cannot be done from within the
JSP itself, or -- better yet -- in an <%include> used by as many pages
as you want. *OR* ... you could do it in a Filter and apply it to all
requests, storing the information in the request attributes, which is
much more standard than directly modifying the page context.


The idea is to avoid any custom entries in web.xml (which filters
require)


This is not entirely true, and seems to be an arbitrary requirement. The 
application is configured via web.xml. Why disallow any configuration in 
web.xml?



and the entire point is this is a top level template and
future versions will provide a number of standard setAttri

Re: page extends not working???

2023-09-13 Thread Christopher Schultz


Aryeh,

On 9/12/23 17:50, Aryeh Friedman wrote:

On Tue, Sep 12, 2023 at 1:51 PM Christopher Schultz
 wrote:


Aryeh,

On 9/12/23 12:42, Aryeh Friedman wrote:

On Tue, Sep 12, 2023 at 11:42 AM Christopher Schultz
 wrote:


Aryeh,

On 9/11/23 10:05, Aryeh Friedman wrote:

On Mon, Sep 11, 2023 at 9:47 AM Christopher Schultz
 wrote:


Aryeh,

On 9/9/23 19:36, Aryeh Friedman wrote:

On Sat, Sep 9, 2023 at 1:23 PM Mark Thomas  wrote:


On 09/09/2023 11:52, Aryeh Friedman wrote:

Every other jsp in my webapp (and other webapps on the same tomcat
instance [9.0.75]) works and I am using a the default container but as
curl/catalina.out show BasePage is *NEVER* being called (either the
_jspService() or the getX()):


How have you configured your JSP(s) to use this alternative base class?


sudo cat /usr/local/apache-tomcat-9.0/webapps/tlaitc-dashboard-1a1/index.jsp

<%@page extends="dashboard.web.pages.BasePage"%>
hi x is ${x}

Output shown in log (sorry for not including the JSP the first time)
but to make it easier to find the output is "hi x is " when it should
be "hi x is 123234"... as you notice there are zero errors/warning in
catalina but there is none of the println's also... so the only thing
I can surmise is BasePage is never being called <%@page
extends="dashboard.web.pages.BasePage"%> somehow failed but I have
verified that correct spelling several times and also verified any
syntextual errors [including the contents of the string literal] will
show up in catalina.out (i.e. wrong class name is logged as an error)


Your _jspService method in your base class will never be called, because
it is overridden by the auto-generated class for your JSP, which does
not call super._jspService.

I do not believe that this:

Hi X is ${x}

...will result in this.getX() being called from the JSP. References in
EL ${...} expressions will be resolved against the PageContext (and
other wider contexts), not against the JSP class currently executing.

If you want to call this.getX() then you will need to do this:

Hi X is <% getX() %>

I wouldn't bother messing-around with class hierarchies in JSP. It
usually doesn't get you much but almost always requires you to bind
yourself very closely with the specific implementation of the JSP engine.

It would be far better to use typical MVC-style development where a
servlet (or similar) handles the real work of the request, possibly
including writing a value of "x" to the request attributes. Then forward
your request to your JSP to produce the response content. This will be
much more straightforward and you will have fewer problems like this,
where expected behavior is confused by all the magic that JSP provides.


Thanks but I have a very specific use case which the following working
example below should make more clear:


<%@page import="dashboard.web.page.Page"%>
<%
   // THIS WOULD NOT BE NEEDED if <%@page extends="..."%> worked
   //
   // for now we don't need to keep the page object just
   // the setAttributes in our ctx
   new Page(pageContext);
%>



   <%@include file="/widgets/scripts/scripts.jsp"%>






and the Page class:

package dashboard.web.page;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.jsp.PageContext;

import org.petitecloud.util.PetiteCloudNullException;

// making this extend the right subclass plus working page
extends="" would mean zero in-line Java
// Copyright (C) 2023 TLAITC and contributors
public class Page
{
   public Page(PageContext ctx)
   {
   _dbc_construction(ctx);

   this.ctx=ctx;

   HttpServletRequest req=(HttpServletRequest) ctx.getRequest();
   String[] parts=req.getRequestURI().split("/");
   int split=2;

   if(parts[0].equals("http:"))
   split+=2;

   name="";
   for(int i=split;i
<%@page extends="dashboard.web.page.Page"%>



   <%@include file="/widgets/scripts/scripts.jsp"%>






Side note I am currently adding user detection to the above page class
to that it can auto-enforce ACL's


I'm not sure I understand the point of the whole exercise. Nothing you
have in the PageContext class constructor cannot be done from within the
JSP itself, or -- better yet -- in an <%include> used by as many pages
as you want. *OR* ... you could do it in a Filter and apply it to all
requests, storing the information in the request attributes, which is
much more standard than directly modifying the page context.


The idea is to avoid any custom entries in web.xml (which filters
require)


This is not entirely true, and seems to be an arbitrary requirement. The
application is configured via web.xml. Why disallow any configuration

Re: AW: Solution to "Invalid keystore format" (cross-posted to Tomcat Users List at Apache, and Java 400 List at Midrange)

2023-09-13 Thread Christopher Schultz


Shawn and Mark,

On 9/13/23 09:30, Mark Thomas wrote:

On 13/09/2023 14:00, Shawn Heisey wrote:

On 9/12/23 01:06, Thomas Hoffmann (Speed4Trade GmbH) wrote:

I moved away from using the proprietary java keystore format.
I switched to using Base64 PEM format. This is usually also the 
format you get from the certificate issuer.
No need to convert it into Java format any more and you can also open 
it with any text editor.


I have never been able to get a Java program to accept a 
certificate/key in PEM format.  The closest I've been able to come is 
creating a PKCS12 file with openssl.  Annoying because all the other 
software I use accepts PEM with no problem, and as you have said, PEM 
is the format generally produced by a CA.


How did you get it to take a PEM cert?


Tomcat has supported this for a while. The bulk of th ecode can be found 
in:


https://github.com/apache/tomcat/blob/main/java/org/apache/tomcat/util/net/jsse/PEMFile.java


I also have code on GitHub that is very similar.

https://github.com/ChristopherSchultz/pem-utils

The hard part is the wide variety of "private key" formats that are out 
there in the wild. Reading a certificate in PEM format from Java is 
pretty much a one-liner. But reading a private key in one of the many 
possible formats, encodings, encryption strategies, etc. requires miles 
and miles of code.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: AW: Solution to "Invalid keystore format" (cross-posted to Tomcat Users List at Apache, and Java 400 List at Midrange)

2023-09-14 Thread Christopher Schultz


Brian,

On 9/13/23 23:25, Brian Wolfe wrote:

The PKCS12 is the industry standard keystore format. Your mac should be
creating it in that version. You should get familiar using the pkcs12. Its
not difficult to set it up. keytool and openssl support pkcs12 and have for
some time now. Its possible your older keystores are of the storetype JKS
or JCEKS, JKS used to be the default I think back in Java 6. Anything newer
should throw a warning telling you the industry standard is pkcs12. But you
can still open older formats by specifying the "--storetype" option. Your
getting that error because you probably didn't tell it what kind it is and
its default assumption is wrong.

Using a keystore is much better for managing your keys than using PEM
files.


Why?


It's best practice to have seperate stores for keys and for trust.


+1, and using files in PEM format do not preclude this.


by default java has the "cacerts" file for establishing trust.


True, but not terribly relevant.

-chris


On Wed, Sep 13, 2023 at 8:16 PM James H. H. Lampert
 wrote:


Java Keystores work. And I don't find them especially difficult to work
with (other than new formats not being backward-compatible with older
JVMs, and as one who has made a comfortable living banging out code for
IBM Midrange boxes for over a quarter century, I am quite familiar with
a much worse variation on that theme, namely, unless you explicitly set
the TGTRLS parameter (and have the appropriate previous version compiler
installed, and don't need to go back more than it will let you), your
programs will not even *restore* onto a prior release system.

And the one time I attempted to get anything other than a Java Keystore
to work in Tomcat, on an IBM Midrange box, I failed miserably.

Putting shell-script wrappers around two different versions of keytool
on my work Mac, so that "keytool" launches the Java 8 version, and
"keytool-default" launches the default version (in the unlikely event
that I'd ever need it) was a relatively simple exercise.

--
JHHL

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org






-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: HSTS on 401 / error pages

2023-09-14 Thread Christopher Schultz


Thomas,

Please start a new thread next time.

On 9/14/23 02:20, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello everyone,

I would like to get your opinion about the HttpHeaderSecurityFilter in Tomcat.
I configured HSTS in Tomcat and it works well.
When I do a pen-test with burpsuite it complains that HSTS header is missing on 
401 responses.
I couldn’t find much information about whether HSTS makes sense for error pages.

It seems that Tomcat doesn’t send HSTS on 401 pages but burpsuite expects the 
header.
Are there any pros and cons about sending HSTS on 401 response?


You should always return an HSTS header.

How have you configured your HttpHeaderSecurityFilter? What is causing 
the 401 response? Which application is responding with that status?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: AW: HSTS on 401 / error pages

2023-09-15 Thread Christopher Schultz


Thomas,

On 9/14/23 10:03, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello Chris,


-Ursprüngliche Nachricht-
Von: Christopher Schultz 
Gesendet: Donnerstag, 14. September 2023 15:26
An: users@tomcat.apache.org
Betreff: Re: HSTS on 401 / error pages

Thomas,

Please start a new thread next time.


Sorry, I thought removing all content and subject is sufficient. Maybe the 
message-id header is used internally(?)


Absolutely. That's what "reply" does on a mailing list...




On 9/14/23 02:20, Thomas Hoffmann (Speed4Trade GmbH) wrote:

Hello everyone,

I would like to get your opinion about the HttpHeaderSecurityFilter in

Tomcat.

I configured HSTS in Tomcat and it works well.
When I do a pen-test with burpsuite it complains that HSTS header is

missing on 401 responses.

I couldn’t find much information about whether HSTS makes sense for

error pages.


It seems that Tomcat doesn’t send HSTS on 401 pages but burpsuite

expects the header.

Are there any pros and cons about sending HSTS on 401 response?


You should always return an HSTS header.

How have you configured your HttpHeaderSecurityFilter? What is causing the
401 response? Which application is responding with that status?

-chris



Here are the requested details:

SecurityFilter is set in the web.xml of the application:

httpHeaderSecurity

org.apache.catalina.filters.HttpHeaderSecurityFilter
true

 hstsEnabled
 true

...

Further down in the web.xml is a constraint:

  
  xxx
  /*
  

  
  yyy
  

  
  CONFIDENTIAL
  
  


There is no frontend-server, tomcat is directly accessed from the browser.
It seems that burpsuite didn’t send authentication in the first place and this 
resulted in 401.

If I use curl https:///  I get similar result:
< HTTP/1.1 401
< WWW-Authenticate: Negotiate
< Content-Type: text/html;charset=utf-8
< Content-Language: de
< Content-Length: 439
< Date: Thu, 14 Sep 2023 13:58:10 GMT

When providing credentials to curl, the following headers are also included:
< Strict-Transport-Security: max-age=31536000;includeSubDomains
< X-Frame-Options: DENY
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block

I hope this information helps.


Authentication is checked before any filters run, because authentication 
is performed by a Valve, all of which run before any Filters run.


I'm not sure there is a way around this without

a. Using a fronting server of some kind
b. Getting a change of some kind made to Tomcat
c. Hacking this yourself

(b) is probably the best option, though I'm not sure what the best form 
of server-support for this would be.


Making HttpHeaderSecurity available in a Valve-packaging would do the 
trick, but maybe this makes sense to add at a more fundamental level to 
Tomcat. The problem is that HSTS is only one of many security-related 
headers and maybe it's potential lifetime isn't that long. My guess is 
that sometime in the near future, TLS will simply be required for all 
web traffic. If we bake that kind of thing into core-Tomcat, it becomes 
something we will need to un-bake in the future, and chefs can tell you 
that un-baking things rarely works out well.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to setup Apache web server for a Tomcat deployed Spring application

2023-09-15 Thread Christopher Schultz


Martin,

On 9/15/23 14:48, Martin Moore wrote:

I have a situation where I want to call an Tomcat deployed Spring
application remotely without adding the port number (8080), I had tried to
use 80 in Connector but wasn't able to connect to it when outside the LAN.


What's the motivation, here? It may allow for more flexible solutions. 
Are you trying to use non-8080, or are you trying to use non-secure, or 
both?



So I resorted to creating a proxy server using Apache2 Web Server (along
side the Tomcat application server).

Unfortunately when I call the app using name1.domain.com/app/login it times
out and fails.
Following are the configuration for Apache2 and Tomcat:
In server.xml (Tomcat V8)
 http://qadat.qfls.idealab.unical.it/>com"
proxyPort="80"
   />

httpd.conf (under conf/ in Apache2)
...
LoadModule proxy_module modules/mod_proxy.so
...
ServerName  name1.domain. com:80


ProxyPass /app/login http://localhost:8080/app/login

ProxyPassReverse /app/login http://localhost:8080/app/login




Looks good so far, except I would put trailing / symbols on here:

 ProxyPass /app/login/ http://localhost:8080/app/login/
 ProxyPassReverse /app/login/ http://localhost:8080/app/login/

Your email program seems to have put links into hostnames that makes 
this a little difficult to interpret.



To note that on the local machine tomcat returns the app through
http://localhost:8080/app/login 

How to make the app requests proxied so that name1.domain.
com/app/login works and calls the
localhost:8080/app/login to return the Tomcat Spring app

To note that DocumentRoot was added and then removed and that the app
resides in webapps/ROOT


If you are using the ROOT web app, then you should be proxying / and not 
/app/login/ (unless you really expect to proxy exactly one URL). Is the 
app called 'app' or 'qadat'?


If you want to refer to your application as /app/ then you should deploy 
it at webapps/app


What URL are you actually typing into the browser URL bar?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[ANN] Community Over Code Conference NA 2023 in Halifax, Canada, 7-10 Oct 2023

2023-09-20 Thread Christopher Schultz


All,

Please join us in Halifax in 2½ weeks for Community Over Code, the ASF 
Conference.


The Tomcat and httpd tracks are combined for this conference, being held 
on the second of the four-day conference featuring a wide variety of 
presentations and panel-led discussions about wide-ranging topics 
related to the ASF and the projects you care about.


And of source, the Hallway Track is always a great opportunity to meet 
other developers, users, and committers to chat about whatever is on 
your mind.


The full schedule can be found here: https://communityovercode.org/schedule/

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: SSL Cert install help.

2023-09-22 Thread Christopher Schultz


Bill,

On 9/22/23 13:25, Bill wrote:

Hello All,
  I may have started my SSL Cert install & config at step 2 instead of
step 1... :-(


Most mistakes are recoverable :)


Basically I have created my key store, my p12 file and have my cert all in
a sub directory of the conf directory.


All of those things are usually in the same file. What files do you 
actually have, and what is in each of them, specifically? If you have a 
keystore of any kind (including p12 files), post the output of:


$  keytool -list -keystore [filename]


I have updated the server xml with my connectors per online directions.
Yet my SSL (https) cert/site doesn't work.


Can you please post your  configuration, replacing any secrets?

Also, what do you mean "doesn't work"? Tomcat does not start? 
Connections are refused? Browser doesn't like server's cert? Can't 
complete handshake?



The catalina logs do not provide a whole lot of help for me as a TC novice.
I did see this in the log:

(org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache
Tomcat Native library which allows using OpenSSL was not found on the
java.library.path: [/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib])


This is a warning. If you don't intend to use tcnative, you can disable 
the AprLifecycleListener and it will no longer emit that message.



but I'm pretty sure I didn't install native, but the regular version of TC.


The "native" connector is just a Connector, not all of Tomcat. The 
"regular" version of Tomcat supports several types of connectors, the 
"native" one included.



So my question is, was I supposed to install or turn something on before
beginning the process of key store and p12 file and connector configuration?


No, if you have your keystore in order and refer to it properly in the 
config then that's all you should need.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: I forget: does Tomcat have any problems with not having a ROOT context?


James,

On 9/25/23 12:17, James H. H. Lampert wrote:
I probably asked the question before, but does Tomcat have any problems 
with not having a ROOT context?


I always run with a ROOT context just to be able to do things like 
provide custom responses with clients request 
/no-such-app/application.yml and stuff like that.


This may not be true any more, but I have this comment in my ROOT 
web.xml that get auto-built/deployed by my build process:


  Dummy ROOT context to prevent 400 Bad Request 
responses


So it's possible that requesting /no-such-app/whatever will return 404 
these days, but at one point it returned 400 and I didn't like that.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: SSLHostConfig question


Jon,

On 9/26/23 11:32, Mcalexander, Jon J. wrote:
I have a question around the SSLHostConfig SSL Connector in Tomcat. 
In the   section, if the SSL Certificate is in a

Windows PFS Keystore, is it appropriate to add

certificateKeystoreType="PFX"

or

certificateKeystore="path to pfx file" type="PFX"

I'm finding reference to certificateKeystoreType, but not in regards to 
PKCS12/PFX types.


I don't think Tomcat supports "PFX" files per-se, but the intertubes say 
that PFX is PKCS12, which IS supported. So try using "PKCS12" which I 
think is the default.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: SSLHostConfig question


Mark,

On 9/26/23 12:54, Mark Thomas wrote:

On 26/09/2023 16:50, Christopher Schultz wrote:

Jon,

On 9/26/23 11:32, Mcalexander, Jon J. wrote:
I have a question around the SSLHostConfig SSL Connector in Tomcat. 
In the   section, if the SSL Certificate is in a

Windows PFS Keystore, is it appropriate to add

certificateKeystoreType="PFX"

or

certificateKeystore="path to pfx file" type="PFX"

I'm finding reference to certificateKeystoreType, but not in regards 
to PKCS12/PFX types.


I don't think Tomcat supports "PFX" files per-se, but the intertubes 
say that PFX is PKCS12, which IS supported. So try using "PKCS12" 
which I think is the default.


Default for all keystore types is JKS.

As Chris says, "pkcs12" should work.


Most recent JVMs will open a JKS file or PKCS12 file regardless of which 
type you actually specify. That was their solution to switching the 
default from JKS to PKCS12.


At this point, Tomcat should probably issue a warning if the type is 
"JKS" and just tell users "stop using JKS, use PKCS12 instead".


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: SSLHostConfig question


Jonm

On 9/26/23 15:07, Mcalexander, Jon J. wrote:

Thank you, but which format of the line is correct?

certificateKeystoreType="pkcs12"

or

certificateKeystore="path to pfx file" type="pkcs12"


https://tomcat.apache.org/tomcat-9.0-doc/config/http.html#SSL_Support_-_Certificate

Here are the relevant attributes:

certificateKeystoreFile : path to your file
certificateKeystoreType : type of keystore file
type : type of /key/ (choose RSA, EC, or DSA - but don't choose DSA)

-chris


-Original Message-
From: Mark Thomas 
Sent: Tuesday, September 26, 2023 11:54 AM
To: users@tomcat.apache.org
Subject: Re: SSLHostConfig question

On 26/09/2023 16:50, Christopher Schultz wrote:

Jon,

On 9/26/23 11:32, Mcalexander, Jon J. wrote:

I have a question around the SSLHostConfig SSL Connector in Tomcat.
In the   section, if the SSL Certificate is in a
Windows PFS Keystore, is it appropriate to add

certificateKeystoreType="PFX"

or

certificateKeystore="path to pfx file" type="PFX"

I'm finding reference to certificateKeystoreType, but not in regards
to PKCS12/PFX types.


I don't think Tomcat supports "PFX" files per-se, but the intertubes
say that PFX is PKCS12, which IS supported. So try using "PKCS12"
which I think is the default.


Default for all keystore types is JKS.

As Chris says, "pkcs12" should work.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 10 on RHEL 8 with Java 17

2023-09-27 Thread Christopher Schultz


Chris,

On 9/27/23 10:30, Christopher Bland wrote:

Hi All,

I just deployed Tomcat v10.1.13  on a new machine.  When I start Tomcat it says 
it has started but I don’t see the daemon running and I don’t have any logs.  I 
tried running Catalina.sh directly.

# ./catalina.sh start
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME:/usr/lib/jvm/java-17-openjdk-17.0.8.0.7-2.el8.x86_64
Using CLASSPATH:   
/usr/local/tomcat/bin/*:/usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Using CATALINA_OPTS:   -XX:+UseG1GC -Xmx3000m
Tomcat started.

Not running – No Daemon + No Logs

# ./catalina.sh debug
Using CATALINA_BASE:   /usr/local/tomcat
Using CATALINA_HOME:   /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JAVA_HOME:   /usr/lib/jvm/java-17-openjdk-17.0.8.0.7-2.el8.x86_64
Using CLASSPATH:   
/usr/local/tomcat/bin/*:/usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
Using CATALINA_OPTS:   -XX:+UseG1GC -Xmx3000m
invalid option: --add-opens=java.base/java.lang=ALL-UNNAMED

Usage: jdb   

where options include:

There are several of the --add-opens statements that cause startup to fail in 
catalina.sh

# Add the module start-up parameters required by Tomcat
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.lang=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.io=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.base/java.util.concurrent=ALL-UNNAMED"
JAVA_OPTS="$JAVA_OPTS --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED"


Not sure what to do.


Try:

$ ./catalina.sh run

and post the output if it's not obvious what's happening.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[SECURITY] [CORRECTION] CVE-2023-41081 Apache Tomcat Connectors (mod_jk) Authentication Bypass

2023-09-28 Thread Christopher Schultz


CVE-2023-41081 Apache Tomcat Connectors (mod_jk) Authentication Bypass

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
- Apache Tomcat Connectors mod_jk Connector 1.2.0 to 1.2.48

Description:
In some circumstances, such as when a configuration included
"JkOptions +ForwardDirectories" but the configuration did not provide 
explicit mounts for all possible proxied requests, mod_jk would use an 
implicit mapping and map the request to the first defined worker. Such 
an implicit mapping could result in the unintended exposure of the 
status worker and/or bypass security constraints configured in httpd. As 
of JK 1.2.49, the implicit mapping functionality has been removed and 
all mappings must now be via explicit configuration.

Only mod_jk is affected by this issue. The ISAPI redirector is not affected.

Mitigation:
Users of affected versions should apply one of the following mitigations:
- Upgrade to Apache Tomcat Connector (mod_jk) 1.2.49 or later.
- Ensure explicit mounts are configured for all possible proxied
  requests

Credit:
This vulnerability was reported responsibly to the Tomcat security team 
by Karl von Randow.


History
2023-09-13 Original advisory
2023-09-28 Updated summary

References:
[1] http://tomcat.apache.org/security-jk.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[SECURITY] [CORRECTION] CVE-2023-41081 Apache Tomcat Connectors (mod_jk) Authentication Bypass

2023-09-28 Thread Christopher Schultz


CVE-2023-41081 Apache Tomcat Connectors (mod_jk) Authentication Bypass

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
- Apache Tomcat Connectors mod_jk Connector 1.2.0 to 1.2.48

Description:
In some circumstances, such as when a configuration included
"JkOptions +ForwardDirectories" but the configuration did not provide 
explicit mounts for all possible proxied requests, mod_jk would use an 
implicit mapping and map the request to the first defined worker. Such 
an implicit mapping could result in the unintended exposure of the 
status worker and/or bypass security constraints configured in httpd. As 
of JK 1.2.49, the implicit mapping functionality has been removed and 
all mappings must now be via explicit configuration.

Only mod_jk is affected by this issue. The ISAPI redirector is not affected.

Mitigation:
Users of affected versions should apply one of the following mitigations:
- Upgrade to Apache Tomcat Connector (mod_jk) 1.2.49 or later.
- Ensure explicit mounts are configured for all possible proxied
  requests

Credit:
This vulnerability was reported responsibly to the Tomcat security team 
by Karl von Randow.


History
2023-09-13 Original advisory
2023-09-28 Updated summary

References:
[1] http://tomcat.apache.org/security-jk.html

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Migrating Tomcat 8/9 and a single webapp to Java 17 disconfigures Tomcat logs

2023-09-29 Thread Christopher Schultz


Alcides,

On 9/28/23 14:55, Alcides Moraes wrote:

Hello everyone,

I’m new to the list even though I’ve been a Java web developer for many years, 
I’ve never had the need to post here, but this time I think I may have stumbled 
upon a bug, and nothing turns up online on this issue.

We’re migrating our containerized legacy webapps from Java 8/11 to Java 17. 
They all ran on Tomcat 8.5, now we’re upgrading to 9.
We customize a base image from tomcat:9.0.80-jdk17-temurin-focal. Amongst other 
things, we add a logging.properties to customize tomcat’s log format.
This always worked well, but after our upgrade to Java 17, there’s a weird 
behavior that has stumped us.
During Tomcat’s startup, the logs are formatted correctly as we specify in 
logging.properties. However, after a certain point in the logs, the logs revert 
to the “default” JUL/JULI format.
Apart from this, everything works as expected. But we need the formatted logs 
because we parse them with LogStash and OpenSearch.

Here’s an excerpt of the logs when this happens:

local-corporativo-comum-1  | 2023-09-27T18:57:34.188 INFO 
[org.apache.coyote.http11.Http11NioProtocol] (thread-1) Initializing ProtocolHandler 
["http-nio-8080"]
local-corporativo-comum-1  | 2023-09-27T18:57:34.266 INFO 
[org.apache.catalina.startup.Catalina] (thread-1) Server initialization in 
[3419] milliseconds
local-corporativo-comum-1  | 2023-09-27T18:57:34.606 INFO 
[com.hazelcast.config.UrlXmlConfig] (thread-1) Configuring Hazelcast from 
'file:/usr/local/tomcat/conf/hazelcast-local.xml'.
local-corporativo-comum-1  | 2023-09-27T18:57:36.775 INFO 
[org.apache.catalina.core.StandardService] (thread-1) Starting service 
[Catalina]
local-corporativo-comum-1  | 2023-09-27T18:57:36.789 INFO 
[org.apache.catalina.core.StandardEngine] (thread-1) Starting Servlet engine: 
[Secure Web Server]
local-corporativo-comum-1  | 2023-09-27T18:57:36.863 INFO 
[org.apache.catalina.startup.HostConfig] (thread-1) Diretório de instalação da 
aplicação web [/usr/local/tomcat/webapps/corporativo-comum]
local-corporativo-comum-1  | 2023-09-27T18:57:52.647 INFO 
[br.leg.senado.util.tomcat.DataSourceFactory] (thread-1) Criando instância do 
datasource corporativo-comumDS
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
br.leg.senado.util.tomcat.DataSourceFactory getObjectInstance
local-corporativo-comum-1  | INFORMAÇÕES: Criando instância do datasource 
monitoraaplDS
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
org.apache.catalina.core.ApplicationContext log
local-corporativo-comum-1  | INFORMAÇÕES: 1 Spring WebApplicationInitializers 
detected on classpath
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
org.apache.catalina.core.ApplicationContext log
local-corporativo-comum-1  | INFORMAÇÕES: Initializing Spring root 
WebApplicationContext
local-corporativo-comum-1  | 18:57:55.751 [main] INFO 
org.springframework.web.context.ContextLoader - Root WebApplicationContext: 
initialization started

The red text is tomcat logging using the defaults (which localize log levels to 
our locale which is pt-br).
I suspect that the point where this happens is when the webapp is being 
initialized. A webapp shouldn’t be able to interfere with Tomcat’s log 
behavior, right?
The webapp does not use JUL, it uses logback, and it logs correctly during and 
after its startup.
The logs you see @ 18:57:52 that says “Criando instância...” are of custom 
datasource resources specified in a context.xml file.
They have a custom factory class, passed from a custom jar in tomcat's class 
path.
This jar has worked and logged correctly since ever, we didn’t even recompile 
them to Java 17, we kept them as they were (Java 8).

I’ve tried changing the way this component logs, by calling org.apache.juli 
instead of java.util.logging, removed all logging whatsoever, but nothing 
changes this behavior.
Any suggestions on debugging this? If you need more info don’t hesitate to ask.

Thanks in advance for any help on this.


I feel like I've heard of things like this happening before. It had 
something to do with an application re-initializing and having a private 
logging.properties file which ended up updating the global logging 
configuration.


Can you search the entire (container's) disk for logging.properties 
files /and also all the JAR files you are using/ to see if there is a 
stray file somewhere that could be picked-up at some point after initial 
boot?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: stack traces from Tomcat 10.1.12

2023-09-29 Thread Christopher Schultz


Thomás,

On 9/29/23 03:03, Tomás García wrote:

I've noticed these stack traces happening in the same row with Tomcat
10.1.12, Java 17 and Spring Boot 3.1.3. I don't have a way to
reproduce them unfortunately. I thought that it could be related to
https://bz.apache.org/bugzilla/show_bug.cgi?id=67235 but not sure.
Sharing them here in case it could be useful to pinpoint any issue in
case these are not already fixed in current development branch of
Tomcat or its latest version.


Can you provide any more context? I don't see, for example, any 
application code in these stack traces which means something has caused 
Tomcat's own internal async state tracking to get confused about something.


Are you using application-managed threads to handle your async requests? 
Can you provide some sample code for how you use them?


Do these errors happen at arbitrary times, or could they be happening 
around application re-deployment, Tomcat shut-down, etc.?


Anything else you could provide which would help?

-chris


Stacktraces:
logger   org.apache.catalina.connector.CoyoteAdapter
messageException while processing an asynchronous request

java.lang.IllegalStateException: Calling [asyncComplete()] is not
valid for a request with Async state [COMPLETING]
   at
org.apache.coyote.AsyncStateMachine.asyncComplete(AsyncStateMachine.java:345)
   at
org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:508)
   at org.apache.coyote.Request.action(Request.java:514)
   at
org.apache.catalina.core.AsyncContextImpl.complete(AsyncContextImpl.java:91)
   at
org.apache.catalina.core.AsyncContextImpl.setErrorState(AsyncContextImpl.java:427)
   at
org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:155)
   at
org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:243)
   at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:57)
   at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:894)
   at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1740)
   at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
   at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
   at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
   at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.base/java.lang.Thread.run(Thread.java:833)

logger   org.apache.coyote.http11.Http11NioProtocol
messageError reading request, ignored

java.lang.NullPointerException: Cannot invoke
"org.apache.catalina.Context.bind(boolean, java.lang.ClassLoader)"
because "this.context" is null
   at
org.apache.catalina.core.AsyncContextImpl.fireOnComplete(AsyncContextImpl.java:101)
   at
org.apache.coyote.AsyncStateMachine.asyncPostProcess(AsyncStateMachine.java:286)
   at
org.apache.coyote.AbstractProcessor.asyncPostProcess(AbstractProcessor.java:197)
   at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:78)
   at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:894)
   at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1740)
   at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
   at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
   at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
   at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
   at java.base/java.lang.Thread.run(Thread.java:833)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Migrating Tomcat 8/9 and a single webapp to Java 17 disconfigures Tomcat logs

2023-10-02 Thread Christopher Schultz


Alcides,

On 9/29/23 15:34, Alcides Moraes wrote:

Forgot to expand the webapps/WEB-INF/lib jars as well…

root@8ad4f1dcd125:/usr/local/tomcat# find ./ -type f -name logging.properties
./conf/logging.properties
./webapps/corporativo-comum/WEB-INF/lib/org/springframework/boot/logging/java/logging.properties

So there’s springboot's logging.properties. Should it really affect tomcat’s 
logging?


I wouldn't expect it to. But something is definitely triggering a 
re-load of the global logging configuration.


If you have decided to move-on to log4j2, then that's great. But if you 
are only doing that because you can't explain what's happening, I think 
it would be better for you to track-down the root problem. If you don't 
track that down, you might be ignoring something important that really 
needs to get fixed.


-chris


Em 29 de set. de 2023, à(s) 16:18, Alcides Moraes  
escreveu:

Hi Christopher,

Thanks for the suggestion, we do add some jars to Tomcat lib (mainly 
Prometheus, Hazelcast)
I expanded every jar inside tomcat/lib and ran a find command.

root@05ae85e03d7d:/# find ./ -type f -name logging.properties
./usr/local/tomcat/conf/logging.properties
./opt/java/openjdk/conf/logging.properties

The only other logging.properties is the one from JDK. I tried changing its 
content just to see if that was what was being used, but it had no effect.

So the issue still remains, but we have worked around it by configuring tomcat 
to use log4j2 as per this documentation:
https://logging.apache.org/log4j/2.x/log4j-appserver.html
  
With this and the log4j-jul bridge, all logs are now formatted correctly.



Em 29 de set. de 2023, à(s) 08:56, Christopher Schultz 
 escreveu:

Alcides,

On 9/28/23 14:55, Alcides Moraes wrote:

Hello everyone,
I’m new to the list even though I’ve been a Java web developer for many years, 
I’ve never had the need to post here, but this time I think I may have stumbled 
upon a bug, and nothing turns up online on this issue.
We’re migrating our containerized legacy webapps from Java 8/11 to Java 17. 
They all ran on Tomcat 8.5, now we’re upgrading to 9.
We customize a base image from tomcat:9.0.80-jdk17-temurin-focal. Amongst other 
things, we add a logging.properties to customize tomcat’s log format.
This always worked well, but after our upgrade to Java 17, there’s a weird 
behavior that has stumped us.
During Tomcat’s startup, the logs are formatted correctly as we specify in 
logging.properties. However, after a certain point in the logs, the logs revert 
to the “default” JUL/JULI format.
Apart from this, everything works as expected. But we need the formatted logs 
because we parse them with LogStash and OpenSearch.
Here’s an excerpt of the logs when this happens:
local-corporativo-comum-1  | 2023-09-27T18:57:34.188 INFO 
[org.apache.coyote.http11.Http11NioProtocol] (thread-1) Initializing ProtocolHandler 
["http-nio-8080"]
local-corporativo-comum-1  | 2023-09-27T18:57:34.266 INFO 
[org.apache.catalina.startup.Catalina] (thread-1) Server initialization in 
[3419] milliseconds
local-corporativo-comum-1  | 2023-09-27T18:57:34.606 INFO 
[com.hazelcast.config.UrlXmlConfig] (thread-1) Configuring Hazelcast from 
'file:/usr/local/tomcat/conf/hazelcast-local.xml'.
local-corporativo-comum-1  | 2023-09-27T18:57:36.775 INFO 
[org.apache.catalina.core.StandardService] (thread-1) Starting service 
[Catalina]
local-corporativo-comum-1  | 2023-09-27T18:57:36.789 INFO 
[org.apache.catalina.core.StandardEngine] (thread-1) Starting Servlet engine: 
[Secure Web Server]
local-corporativo-comum-1  | 2023-09-27T18:57:36.863 INFO 
[org.apache.catalina.startup.HostConfig] (thread-1) Diretório de instalação da 
aplicação web [/usr/local/tomcat/webapps/corporativo-comum]
local-corporativo-comum-1  | 2023-09-27T18:57:52.647 INFO 
[br.leg.senado.util.tomcat.DataSourceFactory] (thread-1) Criando instância do 
datasource corporativo-comumDS
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
br.leg.senado.util.tomcat.DataSourceFactory getObjectInstance
local-corporativo-comum-1  | INFORMAÇÕES: Criando instância do datasource 
monitoraaplDS
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
org.apache.catalina.core.ApplicationContext log
local-corporativo-comum-1  | INFORMAÇÕES: 1 Spring WebApplicationInitializers 
detected on classpath
local-corporativo-comum-1  | set. 27, 2023 6:57:55 PM 
org.apache.catalina.core.ApplicationContext log
local-corporativo-comum-1  | INFORMAÇÕES: Initializing Spring root 
WebApplicationContext
local-corporativo-comum-1  | 18:57:55.751 [main] INFO 
org.springframework.web.context.ContextLoader - Root WebApplicationContext: 
initialization started
The red text is tomcat logging using the defaults (which localize log levels to 
our locale which is pt-br).
I suspect that the point where this happens is when the webapp is being 
initialized. A webapp shouldn’t be able to interfere with Tomcat’s log 
behavior, right?
Th

Re: Strange catalina.out not created issue

2023-10-02 Thread Christopher Schultz


Jon,

On 10/2/23 13:16, Mcalexander, Jon J. wrote:

A little more info on this. The server rebooted last night and the
auto-startup started the instance correctly, but the Catalina.out
file was not created. It only appears to work properly if you log
onto the server and su to the owner user, then run the stop and start
scripts.
What does the suexec thing ultimately end up calling to launch Tomcat? 
Does it call catalina.sh? What is the effective user-id at the time the 
script is invoked?


Have you tried adding lots of debug logging to your script?

-chris


-Original Message-
From: Mcalexander, Jon J. 
Sent: Monday, October 2, 2023 12:06 PM
To: users@tomcat.apache.org
Subject: Strange catalina.out not created issue

Hi all,
It's me again with another issue that is probably obvious to y'all, but I'm not
seeing it.
We have Tomcat Instances on various services that if Tomcat is started from
the Command Line as the user that owns the Instance, everything starts up
properly and the catalina.out log file gets created. However, if the restart is
performed by a workflow that calls the same scripts but via suexec,
everything starts up properly EXCEPT the catalina.out log file does NOT get
created. Has anyone ran into this before? Currently the instance is running
on Tomcat 9.0.80.

I don't have any log output to give you because ONLY the gc.log is getting
created when this happens.

Thanks,

Dream * Excel * Explore * Inspire
Jon McAlexander
Senior Infrastructure Engineer
Asst. Vice President
He/His

Middleware Product Engineering
Enterprise CIO | EAS | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508

jonmcalexan...@wellsfargo.com
This message may contain confidential and/or privileged information. If you
are not the addressee or authorized to receive this for the addressee, you
must not use, copy, disclose, or take any action based on this message or any
information herein. If you have received this message in error, please advise
the sender immediately by reply e-mail and delete this message. Thank you
for your cooperation.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[OT] Migrating Subversion to a new hostname

2023-10-03 Thread Christopher Schultz


All,

Does anyone have any experience re-locating a Subversion repo? We are 
looking at breaking-up some currently co-hosted services, and we'd like 
to *move* our existing Subversion repo to a new hostname. No other 
changes, but that means anywhere we have code checked-out will have to 
be dealt with.


Is it possible to "update/switch" those working-copies to the new 
server, or do we basically have to delete all working copies and 
re-check-out from the new host?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: SSL Configuration Help :

2023-10-06 Thread Christopher Schultz


Elavarasan,

On 10/6/23 06:32, Elavarasan Pugazhendi wrote:

Hi,

I have a pfx certificate and am trying to import it into a keystore before
configuring it within the tomcat but not able to add the pfx certificate. I
followed the below steps but wasn't able to add the certificate

Tomcat: 9.0.62
OS: RHEL 8

1. keytool -genkey -alias tomcat.net -keyalg RSA -keystore tomcat.jks
Entered the Q&A .
2. Using the pfx file. I create .crt and .key file using the below command
openssl pkcs12 -in crt.pfx -nocerts -out mykey.key
openssl pkcs12 -in crt.pfx -clcerts -nokeys -out mycert.crt
3. export certificate
openssl pkcs12 -export -in mykey.key -chain -CAfile crt.pfx -name
otomcat.net -out tomcat.jks


This last one won't work, at least not the way you expect. openssl can't 
create a JKS file, only PKCS12 / p12 / pfx. By using the openssl pkcs12 
-export command above, you will likely destroy your tomcat.jks file or 
just get a failure.


It's not entirely clear what you are trying to do. Are you trying to 
create a self-signed certificate, or are you trying to get your server 
certificate signed by a Certificate Authority?


If you just need a self-signed cert, then your first command should be 
sufficient -- just remember to set certificateAlias="tomcat.net" if you 
use that alias when creating your initial file. I would recommend using. 
a PKCS12 file when originally creating the keystore, just because JKS 
isn't supported by anything other than Java. All you need to do is make 
sure you have "-keystoretype PKCS12". With your version of Java, that 
may be the default already, but it doesn't hurt to be specific.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: deploying war with tomcat manage fails with no significative errors in logs

2023-10-09 Thread Christopher Schultz


Ivano,

On 10/9/23 16:05, Ivano Luberti wrote:

I solved my own issue:

In my web.xml

I had two times the same mapping for a servlet

   
     reportservlet
/repinvenduti/reportservlet
   

But there was no error message in tomcat logs with this regard.

Maybe tomcat logging is not tuned correctly?


Did you check logs other than catalina.out? I usually find these kind of 
logs in a log file like logs/localhost_*.log


-chris

Because doing the same mistake in Eclipse leads to the following logs 
which clearly expose the poblem cause


java.lang.reflect.InvocationTargetException

at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)

at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)


at java.lang.reflect.Method.invoke(Method.java:497)

at 
org.apache.tomcat.util.IntrospectionUtils.callMethodN(IntrospectionUtils.java:447)


at 
org.apache.tomcat.util.descriptor.web.CallMethodMultiRule.end(WebRuleSet.java:1046)


at org.apache.tomcat.util.digester.Digester.endElement(Digester.java:1001)

at 
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)


at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)


at 
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)


at 
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)


at 
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)


at 
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649)


at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1496)

at 
org.apache.tomcat.util.descriptor.web.WebXmlParser.parseWebXml(WebXmlParser.java:119)


at 
org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1067)


at 
org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:779)


at 
org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:299)


at 
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)


at 
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5130)


at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)

at 
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1429)


at 
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)


at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)


at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)


at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.IllegalArgumentException: The servlets named 
[reportservlet] and [reportservlet] are both mapped to the url-pattern 
[/repinvenduti/reportservlet] which is not permitted


at 
org.apache.tomcat.util.descriptor.web.WebXml.addServletMappingDecoded(WebXml.java:340)


at 
org.apache.tomcat.util.descriptor.web.WebXml.addServletMapping(WebXml.java:333)


... 30 more

ott 09, 2023 10:03:10 PM 
org.apache.tomcat.util.descriptor.web.WebXmlParser parseWebXml


GRAVE: Parse error in application web.xml file at 
[file:/D:/ivano/Met/EclipseWorkspace202109/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/METLocale/WEB-INF/web.xml]


org.xml.sax.SAXParseException; systemId: 
file:/D:/ivano/Met/EclipseWorkspace202109/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps/METLocale/WEB-INF/web.xml; lineNumber: 730; columnNumber: 21; Error at (730, 21) : The servlets named [reportservlet] and [reportservlet] are both mapped to the url-pattern [/repinvenduti/reportservlet] which is not permitted


at 
org.apache.tomcat.util.digester.Digester.createSAXException(Digester.java:1932)


at 
org.apache.tomcat.util.digester.Digester.createSAXException(Digester.java:1964)


at org.apache.tomcat.util.digester.Digester.endElement(Digester.java:1004)

at 
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)


at 
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)


at 
com.sun.org.apache.xerces.intern

Re: Sharing catalina home among tomcat machines in a load balanced environment gives problems with log files

2023-10-10 Thread Christopher Schultz


Mark,

On 10/10/23 06:38, Mark Thomas wrote:
Running multiple instances of Tomcat from the same CATALINA_BASE is 
totally unsupported. This isn't one of those "We don't technically 
support that but you should be OK situations". This is one of the rare 
"You do that and it *will* break and you will be on your own when it 
does." situations.


+1

Both the logs/ and the work/ directories will overwrite each other, 
probably to the point of not only corruption (e.g. log files) but also 
instability -- because of the work/ directory. Files will appear and 
disappear surprisingly to one or the other running Tomcat and you WILL 
have errors, etc.



On 10/10/2023 06:51, Giuseppe Sacco wrote:

>>

I did not use different CATALINA_BASE and CATALINA_HOME since this would
have required to deploy all applications on two different tomcats


In this case, *you have two different Tomcats*, so it is appropriate to 
use different CATALINA_BASE paths.



and my
pipeline only allows to deploy to one tomcat putting an XML file in
$CATALINA_BASE/conf/Catalina/localhost directory.


Maybe you can use a symlink from both directories?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[ANN] Apache Tomcat 10.1.14 available

2023-10-10 Thread Christopher Schultz


The Apache Tomcat team announces the immediate availability of Apache
Tomcat 10.1.14.

Apache Tomcat 10 is an open source software implementation of the
Jakarta Servlet, Jakarta Server Pages, Jakarta Expression Language,
Jakarta WebSocket, Jakarta Authentication and Jakarta Annotations
specifications.

Applications that run on Tomcat 9 and earlier will not run on Tomcat 10 
without changes. Java EE applications designed for Tomcat 9 and earlier 
may be placed in the $CATALINA_BASE/webapps-javaee directory and Tomcat 
will automatically convert them to Jakarta EE and copy them to the 
webapps directory. This conversion is performed using the Apache Tomcat 
migration tool for Jakarta EE tool which is also available as a separate 
download for off-line use.


The notable changes compared to 10.1.13 include:

- Update Tomcat Native to 1.2.39 to pick up Windows binaries built with
  OpenSSL 3.0.11.

- Provide a lifecycle listener that will automatically reload TLS
  configurations a set time before the certificate is due to expire.
  This is intended to be used with third-party tools that regularly
  renew TLS certificates.

- Improve performance of EL expressions in JSPs that use implicit
  objects.

- Several improvements to thread safety and recycling cleanup.

Please refer to the change log for the complete list of changes:
http://tomcat.apache.org/tomcat-10.1-doc/changelog.html

Downloads:
http://tomcat.apache.org/download-10.cgi

Migration guides from Apache Tomcat 8.5.x and 9.0.x:
http://tomcat.apache.org/migration.html

Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[ANN] Apache Tomcat 8.5.94 available

2023-10-10 Thread Christopher Schultz


The Apache Tomcat team announces the immediate availability of Apache
Tomcat 8.5.94.

Apache Tomcat 8 is an open source software implementation of the Java
Servlet, JavaServer Pages, Java Unified Expression Language, Java
WebSocket and JASPIC technologies.

Apache Tomcat 8.5.94 is a bugfix and feature release. The notable
changes compared to 8.5.93 include:

- Update Tomcat Native to 1.2.39 to pick up Windows binaries built with
  OpenSSL 3.0.11.

- Provide a lifecycle listener that will automatically reload TLS
  configurations a set time before the certificate is due to expire.
  This is intended to be used with third-party tools that regularly
  renew TLS certificates.

- Improve performance of EL expressions in JSPs that use implicit
  objects.

- Several improvements to thread safety and recycling cleanup.
Along with lots of other bug fixes and improvements.

Please refer to the change log for the complete list of changes:
https://tomcat.apache.org/tomcat-8.5-doc/changelog.html

Downloads:
https://tomcat.apache.org/download-80.cgi

Migration guides from Apache Tomcat 7.x and 8.0:
https://tomcat.apache.org/migration.html

Please note that Tomcat 8.5.x will reach End-of-life (EOL) on 31 March 
2024. For more information please visit 
https://tomcat.apache.org/tomcat-85-eol.html


Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sharing catalina home among tomcat machines in a load balanced environment gives problems with log files

2023-10-12 Thread Christopher Schultz


Jon,

On 10/10/23 14:26, Mcalexander, Jon J. wrote:

Could you have separate work folders but have the appbase be the in
the shared folder space?
You could do this with some symlinking, but I wouldn't recommend it. If 
you start to do things like dynamic deployment, etc. you will find that 
both servers will be affected and that may not be what you want.


Automation is very flexible and disk space is cheap. Do yourself a favor 
and just separate things so you don't have unforeseen problems.


-chris


-Original Message-----
From: Christopher Schultz 
Sent: Tuesday, October 10, 2023 5:59 AM
To: users@tomcat.apache.org
Subject: Re: Sharing catalina home among tomcat machines in a load
balanced environment gives problems with log files

Mark,

On 10/10/23 06:38, Mark Thomas wrote:

Running multiple instances of Tomcat from the same CATALINA_BASE is
totally unsupported. This isn't one of those "We don't technically
support that but you should be OK situations". This is one of the rare
"You do that and it *will* break and you will be on your own when it
does." situations.


+1

Both the logs/ and the work/ directories will overwrite each other, probably
to the point of not only corruption (e.g. log files) but also instability --
because of the work/ directory. Files will appear and disappear surprisingly to
one or the other running Tomcat and you WILL have errors, etc.


On 10/10/2023 06:51, Giuseppe Sacco wrote:

  >>

I did not use different CATALINA_BASE and CATALINA_HOME since this
would have required to deploy all applications on two different
tomcats


In this case, *you have two different Tomcats*, so it is appropriate to use
different CATALINA_BASE paths.


and my
pipeline only allows to deploy to one tomcat putting an XML file in
$CATALINA_BASE/conf/Catalina/localhost directory.


Maybe you can use a symlink from both directories?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat upgrade from 9.0.80 to 9.0.81

2023-10-12 Thread Christopher Schultz


All,

On 10/11/23 08:06, i...@flyingfischer.ch wrote:


Am 11.10.23 um 14:02 schrieb Alexander Veit:
Caused by: org.apache.http.ConnectionClosedException: Premature end 
of Content-Length delimited message body (expected: 4,999; received: 
3,040)
    at 
org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
    at 
io.restassured.internal.util.IOUtils.toByteArray(IOUtils.java:30)
    at 
io.restassured.internal.http.GZIPEncoding$GZIPDecompressingEntity.getContent(GZIPEncoding.java:69)
    at 
org.apache.http.conn.BasicManagedEntity.getContent(BasicManagedEntity.java:85)
    at 
io.restassured.internal.http.HTTPBuilder.parseResponse(HTTPBuilder.java:546)
    at 
io.restassured.internal.RequestSpecificationImpl$RestAssuredHttpBuilder.super$2$parseResponse(RequestSpecificationImpl.groovy)

    at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:498)
    at 
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)

    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
    at 
groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1268)
    at 
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:144)


Has anyone seen this? I will keep everyone posted after debugging more.


We have experienced the same problem with Tomcat 8.5.94.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Seems to be reported multiple times as this is blocking bug for 
upgrading to the last Tomcat version:



https://bz.apache.org/bugzilla/show_bug.cgi?id=67670


We understand that it is blocking, but if you re using h2, especially 
exposed directly to the internet, you should upgrade to the broken 
release and use Konstantin's recommended workarounds.


Both the h2 Rapid Reset and HTTP Trailer / possible request smuggling 
CVEs are both very important.


We apologize for the regressions. Release votes appear to be going well; 
we will have a new set of releases for everyone very shortly.


Although they are not "official" releases, you are welcome to deploy the 
release-candidates themselves. Assuming they are voted stable, they will 
be identical to the upcoming "official" releases.


See the dev@ list [VOTE] emails for where to get those release-candidate 
artifacts.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: JSF errors when upgrading Tomcat and Eclipse: com.sun.faces.config.JavaClassScanningAnnotationScanner$ConstantPoolInfo.containsAnnotation Unknow type constant pool XX at position XX

2023-10-12 Thread Christopher Schultz


Brian,

On 10/12/23 16:55, Brian Braun wrote:

Hello,

First of all, I apologize if maybe my issue is not exclusively related to
Tomcat, but I think it is.

I started my website many years ago, using Struts 1.2.4 and since then I
have been using it. Some years after that I had the intention to migrate to
JSF (version 2.2.X) and combine both frameworks for a while until I
migrated everything to JSF and Struts was gone. I started learning it,
created a few pages on my site but then I realized that it was not the
ideal framework for me, so I stopped creating more pages with JSF and
continued developing my site with Struts, letting those few JSF pages that
work coexist with the rest of the site running using Struts. Since then, I
haven't touched JSF. I haven't created even one page with it, never
upgraded its version but the JSF JAR and pages are still there. Years
passed and nothing made me think about JSF.

Now I'm using:
- Mac with the M2 chip
- Eclipse (ARM version)
- Tomcat, when developing with Eclipse
- Tomcat 9.0.58 at my production server (which runs Ubuntu 22.04, x86/64)
- Azul's Java 11.0.19+7-LTS aarch64, when developing with Eclipse
- Java 11.0.20.1+1-post-Ubuntu-0ubuntu122.04, at the production server

Well, a few days ago I upgraded Eclipse to Version: 2023-09 (4.29.0) and at
the same time upgraded to Azul's Java 11.0.19+7-LTS aarch64 and to Tomcat
9.0.71 when developing. And since then, I get these "SEVERE" JSF error
messages (many of them) when I start Tomcat:

SEVERE [main]
com.sun.faces.config.JavaClassScanningAnnotationScanner$ConstantPoolInfo.containsAnnotation
Unknow type constant pool NN at position XX

Why is that? Could someone please give me a clue? I just upgraded Eclipse
and Tomcat, didn't do anything else relevant and now I get these messages.
And when I create the WAR file and deploy it on my production server, I get
the same error messages as well over there. It seems like the class files
that gets generated are now different, since I didn't upgrade anything on
my production server, so it is the WAR file that gets generated with
something problematic now.

Thanks in advance!

Here is the complete stuff that Tomcat shows, as a reference:

12-Oct-2023 14:47:18.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version name:
Apache Tomcat/9.0.71
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server built: Jan 9
2023 22:33:01 UTC
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version
number: 9.0.71.0
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Name: Mac OS X
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Version: 14.0
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Architecture: aarch64
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Java Home:
/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Version:
11.0.19+7-LTS
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Azul
Systems, Inc.
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:
/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71
12-Oct-2023 14:47:18.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:
/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71
12-Oct-2023 14:47:18.719 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Dcatalina.base=/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71
12-Oct-2023 14:47:18.719 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Dcatalina.home=/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71
12-Oct-2023 14:47:18.719 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Dwtp.deploy=/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71/webapps
12-Oct-2023 14:47:18.719 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Djava.util.logging.config.file=/Users/brianbraun/BB/ACME/Programacion-Mac-Silicon/EclipseTomcat9.0.71/conf/logging.properties
12-Oct-2023 14:47:18.720 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
12-Oct-2023 14:47:18.720 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: --add-opens=java.base/java.lang=ALL-UNNAMED
12-Oct-2023 14:47:18.720 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: --add-opens=java.base/j

[ANN] Apache Tomcat 8.5.95 available

2023-10-16 Thread Christopher Schultz


The Apache Tomcat team announces the immediate availability of Apache
Tomcat 8.5.95.

Apache Tomcat 8 is an open source software implementation of the Java
Servlet, JavaServer Pages, Java Unified Expression Language, Java
WebSocket and JASPIC technologies.

Apache Tomcat 8.5.95 is a bugfix and feature release. The notable
changes compared to 8.5.94 include:

- Correct a regression in 8.5.94 that broke the Tomcat JBDC
  connection pool

- Correct a regression in 8.5.94 that broke HTTP compression

Please refer to the change log for the complete list of changes:
https://tomcat.apache.org/tomcat-8.5-doc/changelog.html

Downloads:
https://tomcat.apache.org/download-80.cgi

Migration guides from Apache Tomcat 7.x and 8.0:
https://tomcat.apache.org/migration.html

Please note that Tomcat 8.5.x will reach End-of-life (EOL) on 31 March 
2024. For more information please visit 
https://tomcat.apache.org/tomcat-85-eol.html


Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 9 -> Intermittent 404 (3-4 fails in 20-30 million requests daily sometimes )

2023-10-16 Thread Christopher Schultz


Anurag,

On 10/15/23 04:48, Anurag Kumar wrote:


Hi, we are experiencing intermittent 404 errors with both GET and POST 
calls. These errors are quite rare and have proven difficult to 
reproduce in our testing environment. However, on our production system, 
we encounter 3-4 cases daily out of 20-30 million requests where a 404 
error appears in the Tomcat access logs, and the corresponding call 
fails to reach the mapped servlet. Interestingly, the same calls work 
perfectly just a few milliseconds before and after on the same node. 
This inconsistency is causing significant issues, especially when 
critical API calls fail and are not automatically retried.


Is there any open issue related to this problem that we should be aware of?


None that I know of personally.

Can you post your exact Tomcat version, your  configuration 
with any secrets removed and a little more background on the type of 
traffic you are seeing (e.g. HTTP/1.1 v h2, TLS or not, etc.). Are you 
able to tell if these failed requests are part of any kind of pipelined 
requests (HTTP Keep-Alive) or h2 single channels?


Understanding the network topology may be relevant, though its unlikely 
that any lb/rp is doing this, as you can see the logs on the Tomcat 
node. But it may change the way the requests are being handled based 
upon the type of connection between the lb/rp and Tomcat.


Have you double-checked that the URIs are clean and don't contain 
anything unexpected such as lookalike characters, etc.? I suspect this 
is not an issue since you said "critical API calls fail" which leads me 
to understand that you have legitimate customers reporting these 
failures, instead of just investigating unexpected entries in your log 
files.


Is your testing environment reasonably similar to production? What would 
happen if you were to reply a whole day's worth of production-requests 
through your testing environment?


Is there any pattern whatsoever in the failed requests? If you look at 
every failed request for all time, are they randomly distributed 
throughout your URI space, or do you find that some URIs are 
over-represented in your failure data? You may have so few failures that 
you can't draw any conclusions.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Stale tomcat.pid file prevented Tomcat from starting


Darryl,

On 10/17/23 10:30, Darryl Baker wrote:

We are running 9.0.78 on RHEL 7. During our monthly patch and reboot cycle one 
the Tomcat running on one system failed to restart. The error said that there 
was a running version of Tomcat with a low PID number. Just rerunning the start 
“systemctl start tomcat” solved the issue. We use the default PID file. I 
looked at the Catalina.sh script and there is a bunch of logic trying to 
prevent this issue but somehow that failed. Is there any know issues with this 
logic?


I'm sure it's gone, now, but did you check the timestamp(s) on the PID 
file? They may have given you a clue as to whether it was leftover from 
a previous boot cycle, or if Tomcat actually failed to startup during 
*this* reboot cycle.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat minor update


Mark and Aditya,

On 10/18/23 04:21, Mark Thomas wrote:

On 17/10/2023 22:47, Aditya Shastri wrote:

Hello,

We have several tomcat instances that use a single CATALINA_HOME which
is a symlink for a specific version. The Tomcat instance we use is
very barebones and doesn't have any of the apps that come with it.

For example,
The CATALINA_HOME points to a symlink
/opt/tomcat/tomcat-9/tomcat-9-latest ->
/opt/tomcat/tomcat-9/apache-tomcat-9.0.80.

Now, if I want to upgrade to apache-tomcat-9.0.82, I normally do the
following steps:
1. Stop all the running instances of tomcat in the various 
'CATALINA_BASE'.

2. Update the symlink /opt/tomcat/tomcat-9/tomcat-9-latest from
/opt/tomcat/tomcat-9/apache-tomcat-9.0.80 to
/opt/tomcat/tomcat-9/apache-tomcat-9.0.82.
3. Start all the instances.

This method appears to work, and I read it as the most appropriate 
method.


My question is, can I change the symlink,
'/opt/tomcat/tomcat-9/tomcat-9-latest', while all the instances are
running and restart the instances when I have downtime?


Probably not. You might get away with it sometimes but sometimes you are 
going to see errors.



Does Tomcat load all the CATALINA_HOME jar(s) (not including the
webapps folder) and config to memory thereby not caring if the
libraries have changed or does it realize that something has changed?


No. The JVM loads classes when they are first referenced.

The issue will be if you update the symlink, then Tomcat tries to load 
another class and the class from the new version is not compatible with 
the classes from the old version. A failure is unlikely but not 
impossible. I wouldn't risk it.


I wonder if we could solve this, at least on *NIX, by resolving 
CATALINA_HOME by using `readlink -f`. This would allow you to use a 
symlink to point to Tomcat but after catalina.sh is invoked, 
CATALINA_HOME could be replaced with a canonicalized one which does not 
contain a symlink anymore. Maybe there is a similar 
utility/command/path-mangling-magic available on Windows?


That would allow you to change the symlink and not disturb any 
currently-running Tomcat instances. You would obviously not want to 
remove the old version from the disk before shutting-down those 
instances, of course.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 9 -> Intermittent 404 (3-4 fails in 20-30 million requests daily sometimes )

Anurag,

On 10/17/23 10:01, Anurag Kumar wrote:

Thanks, Christopher, for looking into this issue.

Wait until I actually help before thanking me. I'm mostly trying to get 
more information so people smarter than I am can maybe help you. ;)

Tomcat version:
Server version: Apache Tomcat/9.0.74
Server built: Apr 13, 2023 08:10:39 UTC
Server number: 9.0.74.0

Would it be at all possible to upgrade to 9.0.latest on one or more of 
your cluster members? You'd have to read the changelog to see if there 
are any incompatibilities but I suspect you should be okay. There are 
constant improvements, and its possible that 6 months (since 9.0.74) has 
improved something that makes a difference for you.

We became aware of this issue a few days ago when it was reported by a 
customer due to a critical internal API failure, where the possibility 
of unexpected characters was none. Upon investigating the Splunk logs, 
we discovered that this issue had been occurring for at least the past 3 
months, based on the available three-month log data.

We have a single servlet mapped for all URL patterns, and we log the 
requests from this servlet. Internally, we always return a 200 response 
code with the appropriate error page and never throw a 404 response.

Do you ever re-deploy your application while Tomcat is running? Which 
file/log contains the 404 responses?

Here is our Connector configuration:

scheme="https" secure="true" executor="httpsThreadPool" 
acceptCount="250" SSLEnabled="false" connectionTimeout="2" 
URIEncoding="utf-8" enableLookups="false" 
relaxedQueryChars="{}|<>"" compression="on"/>

These 404 issues have been observed on requests created from Chrome, 
HttpURLConnection in Java, and AsyncHttpClient in Java. Our servers are 
behind an Amazon Load Balancer (ALB), and while ALB operates on HTTP2, 
our Tomcat servers are configured for HTTP1.

This issue has been reported on all nine different clusters running the 
same Tomcat version. Our test environment closely mirrors the production 
environment, but we have been unable to reproduce the issue so far, even 
after increasing the number of requests.

It's challenging to identify any specific patterns as the occurrences 
appear to be distributed randomly and happen with very simple GET 
requests. There was one instance where I was able to reproduce the issue 
in production with a straightforward GET request after making 45,000 
calls, but it was never reproduced afterwards through my automation.

:/

Request capture on ALB:-
image.png

This list strips images out. Please send plain-text only.

-chris

On Mon, Oct 16, 2023 at 6:16 PM Christopher Schultz 
mailto:ch...@christopherschultz.net>> wrote:

Anurag,

On 10/15/23 04:48, Anurag Kumar wrote:
 >
 > Hi, we are experiencing intermittent 404 errors with both GET and
POST
 > calls. These errors are quite rare and have proven difficult to
 > reproduce in our testing environment. However, on our production
system,
 > we encounter 3-4 cases daily out of 20-30 million requests where
a 404
 > error appears in the Tomcat access logs, and the corresponding call
 > fails to reach the mapped servlet. Interestingly, the same calls
work
 > perfectly just a few milliseconds before and after on the same node.
 > This inconsistency is causing significant issues, especially when
 > critical API calls fail and are not automatically retried.
 >
 > Is there any open issue related to this problem that we should be
aware of?

None that I know of personally.

Can you post your exact Tomcat version, your  configuration
with any secrets removed and a little more background on the type of
traffic you are seeing (e.g. HTTP/1.1 v h2, TLS or not, etc.). Are you
able to tell if these failed requests are part of any kind of pipelined
requests (HTTP Keep-Alive) or h2 single channels?

Understanding the network topology may be relevant, though its unlikely
that any lb/rp is doing this, as you can see the logs on the Tomcat
node. But it may change the way the requests are being handled based
upon the type of connection between the lb/rp and Tomcat.

Have you double-checked that the URIs are clean and don't contain
anything unexpected such as lookalike characters, etc.? I suspect this
is not an issue since you said "critical API calls fail" which leads me
to understand that you have legitimate customers reporting these
failures, instead of just investigating unexpected entries in your log
files.

Is your testing environment reasonably similar to production? What
would
happen if you were to reply a whole day's worth of production-requests
through your testing environment?

Re: Question about releases available for download


Jon,

On 10/18/23 15:39, Mcalexander, Jon J. wrote:

Thanks Mark. I'm sorry if I stated it incorrectly. I meant the issue
with JDBC being broken, etc. the stuff that prompted the immediate
new releases.

I think the word you were looking for was "regression", not "recursion" ;)

-chris


-Original Message-
From: Mark Thomas 
Sent: Wednesday, October 18, 2023 2:04 PM
To: users@tomcat.apache.org
Subject: Re: Question about releases available for download

On 18/10/2023 18:29, Mcalexander, Jon J. wrote:

Hi Mark, et-al,

With the recursion error with these releases in mind, should 8.5.94, 9.0.81,

and 10.1.15 be available for download via the archives? Should they not be
removed and a not placed in the location that they have been removed due
to introduced issues?

Recursion error?

Regardless, all ASF releases will always be available from the ASF archives.
That is ASF policv and I don't see it changing.

Yes, old releases have bugs and/or security issues. Yes, some of those bugs /
security issues are quite nasty. That is why we always recommend using the
latest release of a current supported major version.

Maven Central has a similar policy. Once a release is published to Maven
Central it is pretty much impossible to get it removed.

Mark



Just asking,

Thanks.

Dream * Excel * Explore * Inspire
Jon McAlexander
Senior Infrastructure Engineer
Asst. Vice President
He/His

Middleware Product Engineering
Enterprise CIO | EAS | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508



jonmcalexan...@wellsfargo.com

This message may contain confidential and/or privileged information. If you

are not the addressee or authorized to receive this for the addressee, you
must not use, copy, disclose, or take any action based on this message or any
information herein. If you have received this message in error, please advise
the sender immediately by reply e-mail and delete this message. Thank you
for your cooperation.





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Question about releases available for download

2023-10-19 Thread Christopher Schultz


Jon,

On 10/19/23 11:33, Mcalexander, Jon J. wrote:

Ding Ding Ding. Chris wins! Yes, that was the word.


https://www.youtube.com/watch?v=NtfVgzXTp7Q

-chris


-Original Message-
From: Christopher Schultz 
Sent: Wednesday, October 18, 2023 9:42 PM
To: users@tomcat.apache.org
Subject: Re: Question about releases available for download

Jon,

On 10/18/23 15:39, Mcalexander, Jon J. wrote:

Thanks Mark. I'm sorry if I stated it incorrectly. I meant the issue
with JDBC being broken, etc. the stuff that prompted the immediate new
releases.

I think the word you were looking for was "regression", not "recursion" ;)

-chris


-Original Message-
From: Mark Thomas 
Sent: Wednesday, October 18, 2023 2:04 PM
To: users@tomcat.apache.org
Subject: Re: Question about releases available for download

On 18/10/2023 18:29, Mcalexander, Jon J. wrote:

Hi Mark, et-al,

With the recursion error with these releases in mind, should 8.5.94,
9.0.81,

and 10.1.15 be available for download via the archives? Should they
not be removed and a not placed in the location that they have been
removed due to introduced issues?

Recursion error?

Regardless, all ASF releases will always be available from the ASF archives.
That is ASF policv and I don't see it changing.

Yes, old releases have bugs and/or security issues. Yes, some of
those bugs / security issues are quite nasty. That is why we always
recommend using the latest release of a current supported major version.

Maven Central has a similar policy. Once a release is published to
Maven Central it is pretty much impossible to get it removed.

Mark



Just asking,

Thanks.

Dream * Excel * Explore * Inspire
Jon McAlexander
Senior Infrastructure Engineer
Asst. Vice President
He/His

Middleware Product Engineering
Enterprise CIO | EAS | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508





jonmcalexan...@wellsfargo.com<mailto:jonmcalexan...@wellsfargo.com>

This message may contain confidential and/or privileged information.
If you

are not the addressee or authorized to receive this for the
addressee, you must not use, copy, disclose, or take any action based
on this message or any information herein. If you have received this
message in error, please advise the sender immediately by reply
e-mail and delete this message. Thank you for your cooperation.





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Dealing with an insecure Struts application on Tomcat

2023-10-19 Thread Christopher Schultz


Alan,

On 10/19/23 12:44, Alan F wrote:

I am looking at security steps to mitigate issues with a 1.x Struts based app.


Is this from a "Struts 1 is vulnerable" perspective? Because -- on paper 
-- it is. Vulnerable that is. But that doesn't necessarily mean that 
your application is vulnerable. I encourage you to read the CVEs 
associated with Struts 1 to see if they apply to you.



I have recommended the following until an upgrade resource is available

Remove application from current shared datasource
Remediate high risk CVE scored vulnerabilities (x4 with high EPSS rating)
Reduce exposure to internal audience.
Create new db and instance for above isolated datasource

Would you take it further and ensure this runs on it's own separate Tomcat 
instance?
Any other recommendations?


This depends upon what your threat model is. If the application seems 
like it's vulnerable, then isolating it from other applications may make 
some sense. But if your primary concern is access to the underlying 
data, then isolating the application won't protect the data.


I'm not sure what you mean by "shared data source". If you have a 
server-defined data source that is being shared by individual 
applications, then you probably just shouldn't be doing that in general.


Note that upgrading from Struts 1 to Struts 2 will probably require a 
complete rewrite of your application. :/


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat minor update

2023-10-20 Thread Christopher Schultz


Aditya,

On 10/19/23 14:42, Aditya Shastri wrote:

The way I do the start.sh in my Catalina base is:

BASEDIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )/..
export CATALINA_BASE=$(realpath ${BASEDIR})

/opt/tomcat/tomcat-9/tomcat-9-latest/bin/startup.sh

I could just say $(realpath /opt/tomcat/tomcat-9/tomcat-9-latest)

The old version is we get rid off after 7 days just in case a rollback is 
needed.


Depending upon the system, `realpath` may or may not resolve all 
symlinks. On Linux, the man page indicates that the -P switch can be 
used to resolve them, but that appears to be a GNU-only option. 
`realpath` on my Linux system seems to replace symlinks with their 
physical resolution.


You didn't give an example, but yes, you could say

CATALINA_BASE=$( realpath ${whatever} )

And probably get the behavior I'm describing.

It may be appropriate for Tomcat to try to do that kind of thing for 
you, but maybe this would be better to simply "recommend" in cases like 
yours. It's very easy to do this properly for a single environment by 
just sticking something into bin/setenv.sh to resolve the path to a 
physical one. It may be more difficult for Tomcat to solve that for all 
potential environments where it may run.


> As for Windows, I was under the impression that they are not big fans 
of symlinks anyway so maybe that's ok? :-D


You might be surprised. There are soft-ish-links and hard-ish-links in 
NTFS plus some other hand-wavy things in DOS/Windows land. I don't 
pretend to understand them all, but I've been surprised to discover that 
various weird combinations of special symbols can be used along with 
environment variables to get paths in certain forms under Windows.


-chris


-Original Message-
From: Christopher Schultz 
Sent: Wednesday, October 18, 2023 7:32 PM
To: users@tomcat.apache.org
Subject: Re: Tomcat minor update

Mark and Aditya,

On 10/18/23 04:21, Mark Thomas wrote:

On 17/10/2023 22:47, Aditya Shastri wrote:

Hello,

We have several tomcat instances that use a single CATALINA_HOME which
is a symlink for a specific version. The Tomcat instance we use is
very barebones and doesn't have any of the apps that come with it.

For example,
The CATALINA_HOME points to a symlink
/opt/tomcat/tomcat-9/tomcat-9-latest ->
/opt/tomcat/tomcat-9/apache-tomcat-9.0.80.

Now, if I want to upgrade to apache-tomcat-9.0.82, I normally do the
following steps:
1. Stop all the running instances of tomcat in the various
'CATALINA_BASE'.
2. Update the symlink /opt/tomcat/tomcat-9/tomcat-9-latest from
/opt/tomcat/tomcat-9/apache-tomcat-9.0.80 to
/opt/tomcat/tomcat-9/apache-tomcat-9.0.82.
3. Start all the instances.

This method appears to work, and I read it as the most appropriate
method.

My question is, can I change the symlink,
'/opt/tomcat/tomcat-9/tomcat-9-latest', while all the instances are
running and restart the instances when I have downtime?


Probably not. You might get away with it sometimes but sometimes you are
going to see errors.


Does Tomcat load all the CATALINA_HOME jar(s) (not including the
webapps folder) and config to memory thereby not caring if the
libraries have changed or does it realize that something has changed?


No. The JVM loads classes when they are first referenced.

The issue will be if you update the symlink, then Tomcat tries to load
another class and the class from the new version is not compatible with
the classes from the old version. A failure is unlikely but not
impossible. I wouldn't risk it.


I wonder if we could solve this, at least on *NIX, by resolving
CATALINA_HOME by using `readlink -f`. This would allow you to use a
symlink to point to Tomcat but after catalina.sh is invoked,
CATALINA_HOME could be replaced with a canonicalized one which does not
contain a symlink anymore. Maybe there is a similar
utility/command/path-mangling-magic available on Windows?

That would allow you to change the symlink and not disturb any
currently-running Tomcat instances. You would obviously not want to
remove the old version from the disk before shutting-down those
instances, of course.

-chris




The information in this electronic mail communication (e-mail) contains 
confidential information which is the property of the sender and may be 
protected by the attorney-client privilege and/or attorney work product 
doctrine. It is intended solely for the addressee. Access to this e-mail by 
anyone else is unauthorized by the sender. If you are not the intended 
recipient, you are hereby notified that any disclosure, copying, or 
distribution of the contents of this e-mail transmission or the taking or 
omission of any action in reliance thereon or pursuant thereto, is prohibited, 
and may b

Re: Dealing with an insecure Struts application on Tomcat

2023-10-20 Thread Christopher Schultz


Greg,

On 10/20/23 11:52, Greg Huber wrote:

Remember seeing this, a maintained version of Struts 1.  Might be work a
look.

https://github.com/weblegacy/struts1


This is interesting. I knew about this one:

https://github.com/kawasima/struts1-forever

But the weblegacy folks look *serious* about this: they even have a 
Jakarta EE-compatible release.


-chris


On Thu, 19 Oct 2023 at 17:45, Alan F  wrote:


I am looking at security steps to mitigate issues with a 1.x Struts based
app.

I have recommended the following until an upgrade resource is available

Remove application from current shared datasource
Remediate high risk CVE scored vulnerabilities (x4 with high EPSS rating)
Reduce exposure to internal audience.
Create new db and instance for above isolated datasource

Would you take it further and ensure this runs on it's own separate Tomcat
instance?
Any other recommendations?








-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: CredentialHandler tomcat 7

2023-10-23 Thread Christopher Schultz

Chuck,

On 10/22/23 13:55, Chuck Caldarale wrote:

On Oct 22, 2023, at 10:02, Усманов Азат Анварович wrote:

Hi everyone! I'm trying to use CredentialHandler with tomcat to increase
security since our db at $work still has pwd stored as md5 hashes. Some of our
servers still use tomcat 7.092/ I was looking at this presentation by
Christopher Shultz
http://people.apache.org/~schultz/ApacheCon%20NA%202017/Seamless%20Upgrades%20for%20Credential%20Security%20in%20Apache%20Tomcat.pdf
it mentions that Credention handler should be available to a web app in
Tomcat 7.0.70+ But then I looked up source code for catalina.jar in 7.0.92 and
7.0.109-src I cant find class Named CredentialHandler.Am I looking at the
wrong place or is it just not available in tomcat 7 ? Also tomcat docs for 7
doesn't seem to mention CredentialHandler at all..

Looks like the CredentialHandler mechanism was introduced in 8.0.15 (November
2014), with no indication that it would ever be retrofitted to any 7.0.x
version. (The footnote on slide 30 of the cited presentation appears to be in
error.)

Yeah, I have no idea where I got the 7.0.70 version number from. Maybe I
guessed it while drafting and never confirmed it. Sorry,

Азат, it looks like I got that one wrong.

Given that Tomcat 7.0 has not been supported for over two years and numerous
issues have been addressed in the intervening time period, it might be time to
upgrade…

At this point, 7.0 is essentially 2 versions back form the
currently-supported version of Tomcat (8.5.x) which itself is scheduled
to be retired at the end of this coming March -- a mere 5 months from now.

I don't see any appetite for anybody -- myself included -- working on a
back-port for this to Tomcat 7.

I would encourage you to upgrade to Tomcat 9. I suspect you'll find that
your application runs with very few if any issues if you just upgrade in
a development environment and run a test.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Need Help : Tomcat 9.0.75 not honoring session timeout configured in tomcat web.xml for FORM Authentication

2023-10-27 Thread Christopher Schultz


Channa,

On 10/27/23 00:07, Channa Puchakayala wrote:

Tomcat Version : 9.0.75

Operating System: Windows and Linux

Bits: 64

Tomcat 9.0.75 not honoring  session timeout configured in 
tomcat/conf/web.xml for FORM Authentication and it is effecting customers.


==

    

     30 // 30 minutes

     

=

Verified the Tomcat source code

-FormAuthenticator overriding above configured session timeout setting 
(30 minutes)  with value (120 seconds)


-As per FormAuthenticator.Java, this change/issue started from Tomcat 
Version : 9.0.74 for FORM Authentication and it overwrites the original 
session-timeout value


-This issue/behavior not observed in 9.0.73

Verified the Tomcat documentation

-Verified the tomcat changelog, there is a fix/change went in Tomcat 
9.0.74 below related to FORM Based Authentication Session @ 
https://tomcat.apache.org/tomcat-9.0-doc/changelog.html 
, looks which 
is causing this issue.



Can you please state clearly what the issue actually is? This is 
documented behavior of Tomcat. There is a well-documented setting that 
you can adjust if necessary.


Are you reporting a problem? If so, it is not clear from your message above.

What test did you perform?
What did you expect to happen?
What actually happened that was different from your expectation?

-chris


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to custom java program to decrypt keystore password in Tomcat 10.1.15

2023-10-27 Thread Christopher Schultz


yanyizhong and Mark,

On 10/27/23 04:44, Mark Thomas wrote:

On 26/10/2023 11:05, yanyizhong wrote:



Hi Tomcat team,
Version: Tomcat 10.1.15


I am trying to upgrade Tomcat from version 9.0.56 into 10.1.15, and 
found that there is no setKeystorePass(String) method in tomcat 10.1.15.



As we want to use the custom keystore encryption password in 
server.xml like this:



chiphhers="TLS_ECDHE_RSA_WITH_AES_123_GCM_SHA256"

   keystoreFile="E:\tes.jks"
   keystorePass="xsdfdfdsfdfxdf(encryption password)"
   keystoreType"JKS" />


And this "encrypted" password is "decrypted" how?
https://cwiki.apache.org/confluence/display/TOMCAT/Password
(Hint: this is a waste of time from a security perspective.)

If you can find a way to make this work then you are welcome to use it 
but I am sure as I can be that if source code changes are required in 
Tomcat to make this work they won't be happening.


I suspect the way to do this (if you really must) would be via a custom 
PropertySource. If you look at the existing implementations then you 
should have enough hints to put together an implementation that looks 
for "enc:" and "decrypts" what it finds.


Note that org.apache.tomcat.util.digester.PROPERTY_SOURCE multiple 
values, separated by commas.


I've been experimenting with the ServiceBindingPropertySource lately 
and, IMHO, improving it. It was contributed to the project some time ago 
and is woefully under-documented. I'm looking to change that. At first, 
I was thinking about a full hour-ish presentation, but it looks like 
it's better as a short webinar or even just patches to the existing 
documentation.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Accessing Credential handler inside the web application always returns null

2023-10-30 Thread Christopher Schultz


Азат,

On 10/29/23 20:45, Усманов Азат Анварович wrote:

Hi everyone!I'm trying to test CredentialHandeler functionality onour test 
server (Tomcat 9.0.64) inside the web-app
I Our realm is defined as follows( excerpt from server.xml
)
 


  
  

Currently pwd  column defined as  Oracle (RAW) only stores md5 hashes, I was 
hoping to upgrade to PBKDF2 using tomcat ?so  here is the relevant part basic  
login  controller code  (LoginCheckServlet)
LoginCheckServlet

  protected void doGet(HttpServletRequest request, HttpServletResponse 
response) throws ServletException, IOException {
...
  String userName = request.getParameter("j_username");
String password = request.getParameter("j_password");
  HttpSession session = request.getSession();

   UserRecord user=... //load data from db
if 
(user.checkCorrectPassword(password,session.getServletContext())) {
  CredentialHandler 
cr=Security.getCredentialHandler(getServletContext());
  System.out.println(cr.mutate(password));// hoping 
to see my password displayed as pbkdf2 hash

.
}

Security.getCredentialHandler

  public static CredentialHandler getCredentialHandler(final ServletContext 
context) {
System.out.println("context"+context) ;// prints 
contextorg.apache.catalina.core.ApplicationContextFacade@33f1f7c7
System.out.println("context vs"+context.getMajorVersion()); // 
prints 4

System.out.println("ATRIB"+context.getAttribute(Globals.CREDENTIAL_HANDLER));//always
  prints ATRIB null
return (CredentialHandler) 
context.getAttribute(Globals.CREDENTIAL_HANDLER);
}


Your code and configuration looks reasonable to me.


So basically it always  return null  when trying to access
CredentialHandler attribute inside Security.getCredentialHandler
method,Any idea why it might be the case ?
Are you able to re-try with Tomcat 9.0.70 or later? There is a 
changelog[1] entry which may be important for you:


"
Fix: Improve the behavior of the credential handler attribute that is 
set in the Servlet context so that it actually reflects what is used 
during authentication. (remm)

"

There was a problem specifically with the NestedCredentialHandler, I 
think, which was not working as expected. 9.0.70 includes a fix that 
should improve things for you.


-chris


[1] 
https://tomcat.apache.org/tomcat-9.0-doc/changelog.html#Tomcat_9.0.70_(remm)


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Verifying Tomcat downloads

2023-11-03 Thread Christopher Schultz


James, Mark,

On 11/3/23 12:33, Mark Thomas wrote:

On 03/11/2023 15:45, James H. H. Lampert wrote:
Forgive me if this might be a bit off-topic. But I haven't found a lot 
of resources on the subject (and that includes a search of List 
archives).


For years now, I've been ignoring the note on the Tomcat download 
pages to verify the downloads, preferably by their PGP signatures, 
before putting them into service.


This time, though, I decided to follow the instructions. I installed 
GPG, imported the KEYS file, and checked the signatures.


But everything I've read about GPG, and PGP signature checking, says 
it's meaningless unless the keys are verified as genuine.


Is there a procedure for doing this? A few days ago, I privately 
emailed a well-known Tomcat developer, one who has helped me with 
technical matters in the past, asking for a fingerprint verification. 
I've heard nothing back (then again, he hasn't been heard from on-List 
in a few days, so he may be away).


(Sorry... Upcoming production release.)

From the ASF perspective, if the signing key is in the KEYS file then 
it can be considered valid. The KEYS file is protected to the same 
degree that the source code is protected.


+1

The KEYS file in in revision control. If someone can overwrite that, 
they can overwrite the release, anyway, so trust kind of begins there.


The ASF release manager keys should be in the web of trust. Those trust 
relationships *typically* mean that the person trusting the key has seen 
and validated government issued photo ID of the key holder. But there is 
zero guarantee of that.


You can see the signatures on my key if you ask your PGP key manager to 
download all signatories of my (or any other) key. You can use those to 
determine whether or not you trust the whole group of signers, including 
the signed key. For example, my key has been signed by a number of 
notable members of the ASF. If my key were to have been signed by a 
handful of people you had never heard of before, you might want to be 
wary. Because it's a "web", you can follow the relationships until you 
get to Kevin Bacon at which point I think you *have* to trust it.


A better way to validate the binaries is to reproduce the build. With 
recent releases, as long as you build with the same Ant and Java 
version/vendor as the release manager (details are in the 
build.properties.release file in the root of the source archive), you 
should be able to build a bit-for-bit identical binary - i.e. the 
SHSA-512s will match. If a few folks do that and post their results - 
ideally as part of the release vote - that adds significant weight to 
the validity of the released files.


Caveats
- the Javadoc bundle will be reproducible if you use the same OS
- everything else *should* be OS independent
- reproducible builds are hard and easily broken - we might not have got
   it right for every file every time


+1 again

The next 11.0 release will include a verify-release ant target which 
should automate essentially all of that. It can easily be back-ported to 
the other branches as well.


Alternatively, come along to the next Community Over Code conference, 
take part in the key signing party and join the web of trust (or just 
use this as the excuse to come to the conference).


And as a final option (I've done it once in 20 years) you can always 
arrange to meet a release manager face to face to have your own 2-person 
key signing party. Offers of $beverages can help facilitate this ;)


I live near Washington, DC if anybody wants to buy me a beer.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Accessing Credential handler inside the web application always returns null


Азат,

On 10/31/23 13:53, Усманов Азат Анварович wrote:

Hi everyone! CredentialHandler became not null, as soon as I
transferred Realm definition from server.xml to context.xml(after
checking the source code) .I've been able to see the new pbkdf2
version of the given clear text password even with old  9.0.64
version. I was wondering is the necessity to have realm defined
inside context. xml for accessing CredentialHandler a design decision
or a possible  bug in tomcat itself?. It wasn't mentioned in tomcat
documentation. Perhaps it should be added in the docs.
Hmm... it shouldn't matter if you define your  in server.xml or 
in app/META-INF/context.xml. Are you sure that was the only difference 
between working/not-working configurations?


Thanks,
-chris



От: Усманов Азат Анварович 
Отправлено: 30 октября 2023 г. 20:25
Кому: users@tomcat.apache.org 
Тема: RE: Accessing Credential handler inside the web application always 
returns null

I did recheck using 9.0.82, unfortunately nothing has changed CredentialHandler 
is still null
____
От: Christopher Schultz 
Отправлено: 30 октября 2023 г. 18:52
Кому: Tomcat Users List ; Усманов Азат Анварович 

Тема: Re: Accessing Credential handler inside the web application always 
returns null

Азат,

On 10/29/23 20:45, Усманов Азат Анварович wrote:

Hi everyone!I'm trying to test CredentialHandeler functionality onour test 
server (Tomcat 9.0.64) inside the web-app
I Our realm is defined as follows( excerpt from server.xml
)
  
 

   
   
 
Currently pwd  column defined as  Oracle (RAW) only stores md5 hashes, I was 
hoping to upgrade to PBKDF2 using tomcat ?so  here is the relevant part basic  
login  controller code  (LoginCheckServlet)
LoginCheckServlet

  protected void doGet(HttpServletRequest request, HttpServletResponse 
response) throws ServletException, IOException {
...
  String userName = request.getParameter("j_username");
String password = request.getParameter("j_password");
  HttpSession session = request.getSession();

    UserRecord user=... //load data from db
if 
(user.checkCorrectPassword(password,session.getServletContext())) {
  CredentialHandler 
cr=Security.getCredentialHandler(getServletContext());
  System.out.println(cr.mutate(password));// hoping 
to see my password displayed as pbkdf2 hash

.
}

Security.getCredentialHandler

  public static CredentialHandler getCredentialHandler(final ServletContext 
context) {
System.out.println("context"+context) ;// prints 
contextorg.apache.catalina.core.ApplicationContextFacade@33f1f7c7
System.out.println("context vs"+context.getMajorVersion()); // 
prints 4

System.out.println("ATRIB"+context.getAttribute(Globals.CREDENTIAL_HANDLER));//always
  prints ATRIB null
return (CredentialHandler) 
context.getAttribute(Globals.CREDENTIAL_HANDLER);
}


Your code and configuration looks reasonable to me.


So basically it always  return null  when trying to access
CredentialHandler attribute inside Security.getCredentialHandler
method,Any idea why it might be the case ?

Are you able to re-try with Tomcat 9.0.70 or later? There is a
changelog[1] entry which may be important for you:

"
Fix: Improve the behavior of the credential handler attribute that is
set in the Servlet context so that it actually reflects what is used
during authentication. (remm)
"

There was a problem specifically with the NestedCredentialHandler, I
think, which was not working as expected. 9.0.70 includes a fix that
should improve things for you.

-chris


[1]
https://tomcat.apache.org/tomcat-9.0-doc/changelog.html#Tomcat_9.0.70_(remm)


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: FIPS Configuration for Java 11/17 and Tomcat 9


Amit,

On 11/2/23 21:18, Amit Pande wrote:

Please refer to the link below in case you are interested in configuring FIPS 
for Tomcat 9 running on Java 17.

https://github.com/amitlpande/tomcat-9-fips/wiki/Java-11-17-Tomcat-9-FIPS-Configuration-Using-Bouncy-Castle

I have tested steps for Java 11 and even Java 8 too. But there are different 
ways for Java 8 at least  (https://github.com/amitlpande/tomcat-9-fips).

Appreciate reviews and any other feedback.


Thanks for doing this. The guide is easy to understand and explains both 
the "how" /and/ the "why" for everything.


Nicely done.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Admin password for Tomcat


Jerry,

On 11/4/23 20:17, Jerry Malcolm wrote:
My support team needs to be able to log in to our site as various users 
(on behalf of...) to be able to see exactly what they are seeing since 
roles, access groups, history is different for different users.  I would 
like to implement an admin password where I can log in as any userId 
with this password.  I totally realize the security risks involved in 
this.  But I am handling the security risks with additional 
authorizations.  I simply need to make every user have two passwords... 
their real personal password, and the admin password.  The only 
alternative I have right now is to save off the user's password hash in 
the USERS table, replace it with my password hash, then restore the 
user's original password when I'm done.  I'm not thrilled with that 
solution first because it's a pain and error prone, and also because the 
user can no longer log in while their password is replaced with my 
password.


  I figure this function is buried in the authenticator code somewhere. 
But I'd first like to see if anybody has done anything like this 
already.  If not, could somebody point me in the right direction to the 
tomcat source file that I'm going to need to modify and also what's 
involved in making authentication use my updated class instead of the 
default.


Suggestions?


This sounds like "impersonation" to me, which, I think, can be done 
differently. If you are indeed describing an X-Y problem above, then 
might I suggest the following?


Instead of figuring out how to "add" a second password to a user, what 
about allowing you to login as e.g. "jerry" and then assume the identity 
of the user "tom"? You should be able to do this by changing the 
UserPrincipal in the session to have a different username.


Which application are you trying to do this with? Your own application, 
or one which ships with Tomcat (e.g. Manager)?


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re:


Greg and Mark,

On 11/5/23 09:31, Mark Thomas wrote:

On 05/11/2023 10:18, Greg Huber wrote:
OK thanks, the docs mention "static resource cache" but I could not 
find info on what it actually is.


It caches the content of static resources in memory and uses that rather 
than accessing disk.



I am loading maven jars and /target/classes.

eg:








As its purely for development guess it makes no difference?


I doubt you'll notice if you disable it.


+1

Since you are using JAR files, the caching won't matter once the classes 
themselves are loaded-into memory. So you may observe some slowness 
early in the lifetime of the web application after deployment, but at 
long as your code, etc. isn't trying to re-scan JAR files all the time, 
etc. then you should be fine.


-chris


On 05/11/2023 10:02, Mark Thomas wrote:

On 04/11/2023 11:03, Greg Huber wrote:

Hello,

I am using the  and  to run tomcat for 
debugging my app (and it is pretty awesome).  I am getting the cache 
warning limit, as it is 10mb, what effect would it have if I turned 
off the cache ie cachingAllowed="false" rather than having to 
increase the limit all the time?


This is one of those "it depends" questions. There are lots of 
factors that will influence how effective the cache is. You could try 
and reason what the impact would be but you will likely get a more 
accurate answer, faster by just trying it and measuring the impact.


Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: tomcat 10


直以来,

On 11/6/23 06:25, 一直以来 wrote:

What can I do to see that the request is reused, using what settings?


What problem are you trying to solve?

-chris


-- Original --
From: Mark Thomas 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Vulnerabilities Patches


All,

On 11/6/23 20:32, James H. H. Lampert wrote:

On 11/6/23 5:21 PM, Nithiyanandam BALASUBRAMANIYAN (Oneberry) wrote:
I am using Tomcat Apache Version 8.5.94 in Windows server 2012. 
Recently received following vulnerabilities alert to fix :


Short answer: you're already there. And the latest Tomcat 8 (which I 
just bumped a customer up to) is 8.5.95.


On an IBM Midrange box, I just manually copy the keystore, our webapps, 
and certain configuration settings over from the old version to the new 
version, then find a good time to switch the customer over (which 
involves shutting down the old Tomcat, renaming the old and new Tomcat 
directories, and restarting it with the new version in place. Piece of 
cake.


I understand that Linux, WinDoze, and Mac have ways to bump up the 
Tomcat version that are even easier.


https://tomcat.apache.org/presentations.html#latest-split-installation

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Admin password for Tomcat


Jerry,

On 11/6/23 23:22, Jerry Malcolm wrote:


On 11/5/2023 11:54 AM, Jerry Malcolm wrote:


On 11/5/2023 9:26 AM, Christopher Schultz wrote:

Jerry,

On 11/4/23 20:17, Jerry Malcolm wrote:
My support team needs to be able to log in to our site as various 
users (on behalf of...) to be able to see exactly what they are 
seeing since roles, access groups, history is different for 
different users.  I would like to implement an admin password where 
I can log in as any userId with this password.  I totally realize 
the security risks involved in this.  But I am handling the security 
risks with additional authorizations.  I simply need to make every 
user have two passwords... their real personal password, and the 
admin password.  The only alternative I have right now is to save 
off the user's password hash in the USERS table, replace it with my 
password hash, then restore the user's original password when I'm 
done.  I'm not thrilled with that solution first because it's a pain 
and error prone, and also because the user can no longer log in 
while their password is replaced with my password.


  I figure this function is buried in the authenticator code 
somewhere. But I'd first like to see if anybody has done anything 
like this already.  If not, could somebody point me in the right 
direction to the tomcat source file that I'm going to need to modify 
and also what's involved in making authentication use my updated 
class instead of the default.


Suggestions?


This sounds like "impersonation" to me, which, I think, can be done 
differently. If you are indeed describing an X-Y problem above, then 
might I suggest the following?


Instead of figuring out how to "add" a second password to a user, 
what about allowing you to login as e.g. "jerry" and then assume the 
identity of the user "tom"? You should be able to do this by changing 
the UserPrincipal in the session to have a different username.


Which application are you trying to do this with? Your own 
application, or one which ships with Tomcat (e.g. Manager)?


-chris


Hi Chris, it's my own webapp.  Changing user principal is exactly what 
I'm trying to do.  I wasn't aware that the user principal could be 
easily swapped.  Where can I learn more about how to do that?


Chris, I'm not having any luck googling info on how to replace the user 
principal object in the session object.  This is exactly what I need to 
do.  But looks like I'm going to need a little bit of guidance to figure 
out how to implement it.


I forgot that "we" are using our own custom principal and actually not 
using Tomcat's authentication and authorization. So we do things 
differently.


If you are using FORM authentication, then I think this is a little easier.

You may have to do a nasty bit of casting to internal Tomcat classes 
and/or use reflection, but you can simply call:


org.apache.catalina.Session.setPrincipal(java.security.Principal)

The StandardSession class you probably are already getting from Tomcat 
implements that interface, so you should be able to call that.


I think while Tomcat will accept any java.security.Principal, in 
practice, you'll want to use org.apache.catalina.realm.GenericPrincipal.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Are there any known class loader leaks in Tomcat 9?


William,

On 11/7/23 05:59, William Crowell wrote:

Olaf and Sevendu,

Thank you for your replies.  Correct, I sincerely doubt this is a Tomcat class 
loading bug.

I am using Tomcat’s normal class loader (webapp/WAR) to load the classes into 
memory, and it is a single class loader.

I am going to periodically run: jcmd  GC.class_stats

I am only having the issue in general operation and not on a redeploy, and I 
have to restart the server daily.  I will find out more details as to how these 
classes are loaded into memory.

Here is what is going on…

I have a 16GB Linux instance and one Apache Tomcat 9.0.78 instance running on 
it.  It is running JDK 1.8.0_371-b25.  I have min and max JVM heap setting at 
8GB.  JVM arguments are:

-XX:CICompilerCount=3 -XX:ConcGCThreads=1 -XX:+DisableExplicitGC 
-XX:G1HeapRegionSize=4194304 -XX:GCLogFileSize=3145728 
-XX:+HeapDumpOnOutOfMemoryError -XX:InitialHeapSize=8589934592 
-XX:MarkStackSize=4194304 -XX:MaxHeapSize=8589934592 -XX:MaxNewSize=5150605312 
-XX:MinHeapDeltaBytes=4194304 -XX:NativeMemoryTracking=detail 
-XX:NumberOfGCLogFiles=25 -XX:+PrintGC -XX:+PrintGCApplicationConcurrentTime 
-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails 
-XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops 
-XX:+UseFastUnorderedTimeStamps -XX:+UseG1GC -XX:+UseGCLogFileRotation

The MaxMetaspaceSize is not set, so this means it is unlimited.

After the server comes up and after a short period of time I get a native out 
of memory condition:

# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 8589934592 bytes for committing 
reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may 
be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2798), pid=191803, tid=0x14fff94b7700
#
# JRE version:  (8.0_371) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.371-b25 mixed mode linux-amd64 
compressed oops)
# Core dump written. Default location: /hosting/core or core.191803
#


Enable heap dumps on OOME and look at them with a heap analyzer. You may 
find that you have some huge things in memory that aren't being released 
for other reasons.


If you have to bounce your server every day, I suspect that you have a 
"known resource leak" in your web application already. It's very 
unlikely to be Tomcat doing this.


What you should NOT do is just keep raising the heap size until it stops 
failing, because you will never find out what is taking up all that heap 
space.


If you find that nothing is out of place in that heap dump then AND ONLY 
then should you raise the heap size. Sometimes you really just need more 
heap.


-chri

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Vulnerabilities Patches

2023-11-09 Thread Christopher Schultz


Nithiyanandam,

On 11/8/23 22:06, Nithiyanandam BALASUBRAMANIYAN (Oneberry) wrote:

I want to upgrade form 8.5.94 to 8.5.95. is it the easiest way to
upgrade ? like I seen the jar file copy from old version to new
version. Sorry I am new to apache


I would highly recommend against simply copying "the JARs". You may 
sometimes be able to get away with that, but it's best to leave Tomcat's 
installation directory untouched to make sure you have all the resources 
Tomcat is expecting to find there.


-chris


-Original Message-----
From: Christopher Schultz 
Sent: Thursday, November 9, 2023 4:34 AM
To: users@tomcat.apache.org
Subject: Re: Vulnerabilities Patches

All,

On 11/6/23 20:32, James H. H. Lampert wrote:

On 11/6/23 5:21 PM, Nithiyanandam BALASUBRAMANIYAN (Oneberry) wrote:

I am using Tomcat Apache Version 8.5.94 in Windows server 2012.
Recently received following vulnerabilities alert to fix :


Short answer: you're already there. And the latest Tomcat 8 (which I
just bumped a customer up to) is 8.5.95.

On an IBM Midrange box, I just manually copy the keystore, our
webapps, and certain configuration settings over from the old version
to the new version, then find a good time to switch the customer over
(which involves shutting down the old Tomcat, renaming the old and new
Tomcat directories, and restarting it with the new version in place.
Piece of cake.

I understand that Linux, WinDoze, and Mac have ways to bump up the
Tomcat version that are even easier.


https://tomcat.apache.org/presentations.html#latest-split-installation

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: CredentialHandler not working for MD5


Peter,

On 11/10/23 13:27, Peter Otto wrote:

Logging into manager using MD5 works in 9.0.73 but now fails in 9.0.74->current
Steps to reproduce.

Step 1. Run C:\tomcat\bin> .\digest.bat -a md5 -s 0 -i 1 
tomcat:UserDatabase:nobueno

tomcat:UserDatabase:nobueno:bb6c1c32b9b6df4f707c0e58f2c900e0


Step 2. Use the digest # and place it in tomcat-users.xml





Step 3. Edit server.xml and add the CredentialHandler to use MD5









Step 4. Edit the web.xml in manager to say

 DIGEST
 UserDatabase
   

Step 5 start tomcat and try to access the manager.
On WIndows 2019 server/Chrome/OpenJDK11  type tomcat for the user
and nobueno for the password.

This would work on versions 9.0.73 and earlier

This stopped working from 9.0.74 and onwards.
The way to access the manager from 9.0.74+ is to use 
bb6c1c32b9b6df4f707c0e58f2c900e0 as the password.
In other words the text in tomcat-user.xml is the password.

Anyone have any ideas how to fix this?  I have to use 9.0.74+ version of tomcat 
because of CVEs.


If you temporarily remove the LockOutRealm, does the correct password work?

If you upgrade to 9.0.82, does the correct password work?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Testing OpenSSL integration using the FFM API with Tomcat 11 on Windows 10


Mark,

On 11/10/23 10:27, Mark Thomas wrote:

On 10/11/2023 14:44, Eduardo Guadalupe wrote:

Thanks Mark,

I found the issue, I assumed OpenSSL was installed because I had seen in
some logs the message “OpenSSL successfully initialized [OpenSSL 
3.0.11 19

Sep 2023].”


That may be the OpenSSL version that is static linked to the Tomcat 
Native library. I don't think you can use that directly.


I would think that WOULD work (once loaded), except Tomcat is 
specifically attempting to load ssl.dll in this case. IMO it's probably 
not worth it to allow either libtcnative or libssl. I think you should 
pick one or the other and load the one you expect to use.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: FileUpload class not working with Tomcat 10.1


Mark,

On 11/10/23 12:53, Mark Foley wrote:

On Fri, 10 Nov 2023 17:11:59 Mark Thomas 

On 10/11/2023 16:49, Mark Foley wrote:

I recently upgraded from Tomcat 10.0.17 to 10.1.13.  When I previously upgraded
from 9.0.41 to 10.0.17 (back in 2/22) the FileUpload class broke. I fixed that
thanks to postings on stackoverflow, but now that I've
upgraded to 10.1.13 it is broken again! Here's the error I get:

An error occurred at line: [40] in the jsp file: [/schDistImportResults.jsp]
The method isMultipartContent(ServletRequestContext) is undefined for the type 
FileUpload


Tomcat's internal fork of Commons FileUpload isn't intended for
applications to use. It is not a full fork - just a limited subset of
the functionality Tomcat needs to implement the Servley upload API.

If you want to use Commons File Upload, add the JAR to your web
application and use it.

Alternatively, if you want to use the Servlet upload API then use that.

If the javax.sevlet -> jakarta.servlet transition means you can't use
your preferred version of Commons File Upload in Tomcat 10.1.x (very
likely) then run your preferred version of Commons File Upload through
Tomcat's migration tool for Jakarta EE and use the converted version of
Commons File Upload in your web application.

Depending on Tomcat internals is very likely to lead to breakage.

Mark


Thanks for your quick reply. Whatever I've been using keeps breaking. I had it
working in 9.0.14 and earlier, then it broke with 10.0.17 and I fixed that, now
it's broken again with 10.1.13. So, my "prefered" solution is whatever is
recommended and is likely to continue to be supported without breaking in future
Tomcats.

What do you recommend? And do you have a quickie template somewhere which shows
the basic include(s) and method I need? I really don't do anything very fancy.
My current "basic" implementation is:

<%@ page import="org.apache.tomcat.util.http.fileupload.*,
 org.apache.tomcat.util.http.fileupload.disk.*,
 org.apache.tomcat.util.http.fileupload.servlet.*,
 org.apache.commons.io.*" %>

DiskFileItemFactory factory = new DiskFileItemFactory();
ServletFileUpload upload = new ServletFileUpload(factory);
List items = upload.parseRequest(new ServletRequestContext(request));
Iterator iter = items.iterator();
FileItem item = null;

while (iter.hasNext())
{
 item = (FileItem) iter.next();

 resultsFile = new File(getServletContext().getRealPath("") + 
"/tmp/schTaxResults.txt");

 try { item.write(resultsFile); }
 catch ( Exception e) { out.println("Exception: " + e); }
}

If you could tell me what the officially prefered Apache Tomcat FileUpload
mechanism is, and what the correct jar and functions are to accomplish the 
above, I'd be
very grateful!



No offense, but the above is horrifying. All that Java code in a JSP 
makes me cringe. You can do this however you want, but I'd recommend 
putting Java code into a proper servlet and letting the JSP handle 
display only.


Anyway, I'll get off my soapbox.

The easiest thing IMO for you to do is stop trying to parse the upload 
yourself and use the container. You must have migrated this application 
forward for like 10 years or something if you are still using a separate 
library to handle multipart-form-uploads. This has been a part of the 
code servlet API for some time, now, and you should use it:


import jakarta.servlet.http.Part;

...

String contentType = request.getContentType();
if(null == contentType || !contentType.startsWith("multipart/form-data;")) {
logger.warn("Received non-multipart request");

throw new IllegalStateException("Expected multi-part");
}

java.io.File tmpDir = 
(java.io.File)request.getServletContext().getAttribute("javax.servlet.context.tempdir");


java.io.File targetFile = new java.io.File(tmpDir, "schTaxResults.txt");

Part fileUpload = request.getPart("param-name");

if(null != fileUpload) {
fileUpload.write(targetFile.getAbsolutePath());
}

I have made some obvious and not-so-obvious changes, here. First, you 
don't need a separate library: you are relying on the container for the 
multi-part handling.


Second, I have changed from uploading the file directly into the servlet 
context (the live running application~) into a temporary directory. If 
you want to serve this file back out to clients, you may want to use 
WebDAV or some other protocol rather than file-upload, or maybe not.


If you want to serve this file back to clients, I *highly* recommend 
creating a directory OUTSIDE your web application where you can push 
uploaded files, and then use something like  to allow 
Tomcat to load content from that location, mounted on a path that won't 
allow users to upload binaries, etc. that might get loaded by the 
application.


You may want to be careful about how you are writing. If two requests 
come-in at the same time, thee files may overwrite each other in 
unpredictable ways.


-chris

Re: CredentialHandler not working for MD5


Peter,

On 11/10/23 16:30, Peter Otto wrote:

With 9.0.82, and the latest version 10, I get the same problem.
So I assume it stopped working since 9.0.74 all the way up to 9.0.82

Removing the Realm LockOutRealm did not work either.


Thanks for double-checking both of those.

I don't see anything in the changelog that seems like it would be 
related. Thing I suspect are related were in an earlier release.


Are you able to run under a debugger, and are you comfortable doing 
that? It's pretty easy to set a breakpoint in the Realm and/or 
CredentialHandler to see what's being done when you try to authenticate.


-chris


From: Christopher Schultz 
Date: Friday, November 10, 2023 at 12:35 PM
To: users@tomcat.apache.org 
Subject: Re: CredentialHandler not working for MD5
Peter,

On 11/10/23 13:27, Peter Otto wrote:

Logging into manager using MD5 works in 9.0.73 but now fails in 9.0.74->current
Steps to reproduce.

Step 1. Run C:\tomcat\bin> .\digest.bat -a md5 -s 0 -i 1 
tomcat:UserDatabase:nobueno

tomcat:UserDatabase:nobueno:bb6c1c32b9b6df4f707c0e58f2c900e0


Step 2. Use the digest # and place it in tomcat-users.xml





Step 3. Edit server.xml and add the CredentialHandler to use MD5









Step 4. Edit the web.xml in manager to say

  DIGEST
  UserDatabase


Step 5 start tomcat and try to access the manager.
On WIndows 2019 server/Chrome/OpenJDK11  type tomcat for the user
and nobueno for the password.

This would work on versions 9.0.73 and earlier

This stopped working from 9.0.74 and onwards.
The way to access the manager from 9.0.74+ is to use 
bb6c1c32b9b6df4f707c0e58f2c900e0 as the password.
In other words the text in tomcat-user.xml is the password.

Anyone have any ideas how to fix this?  I have to use 9.0.74+ version of tomcat 
because of CVEs.


If you temporarily remove the LockOutRealm, does the correct password work?

If you upgrade to 9.0.82, does the correct password work?

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
This e-mail and any files transmitted with it are the property of Arthrex, Inc. 
and/or its affiliates, are confidential, and are intended solely for the use of 
the individual or entity to whom this e-mail is addressed. If you are not one 
of the named recipient(s) or otherwise have reason to believe that you have 
received this message in error, please notify the sender at 239-643-5553 and 
delete this message immediately from your computer. Any other use, retention, 
dissemination forwarding, printing or copying of this e-mail is strictly 
prohibited. Please note that any views or opinions presented in this email are 
solely those of the author and do not necessarily represent those of the 
company. Finally, while Arthrex uses virus protection, the recipient should 
check this email and any attachments for the presence of viruses. The company 
accepts no liability for any damage caused by any virus transmitted by this 
email.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Is the HTTP/2 Rapid Reset Exploit still possible on 2.4.58?

2023-11-14 Thread Christopher Schultz


All,

On 11/13/23 17:36, Chuck Caldarale wrote:

You may have the wrong mailing list - this one is for Tomcat, but your query 
seems to be solely about Apache httpd.


Also, the httpd project has stated that they were never vulnerable to 
CVE-2023-44487.


https://github.com/icing/blog/blob/main/h2-rapid-reset.md

To be fair, this is not an "official" statement by the httpd team.

With httpd 5.4.58, you should be covered for not only CVE-2023-44487 (h2 
rapid reset, which was never really a problem) but also CVE-2023-45802 
which was exposed by testing httpd for CVE-2023-44487, but is in fact a 
separate issue, now fixed in 5.4.88.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[ANN] Apache Tomcat 8.5.96 available

2023-11-14 Thread Christopher Schultz


The Apache Tomcat team announces the immediate availability of Apache
Tomcat 8.5.96.

Apache Tomcat 8 is an open source software implementation of the Java
Servlet, JavaServer Pages, Java Unified Expression Language, Java
WebSocket and JASPIC technologies.

Apache Tomcat 8.5.96 is a bugfix and feature release. The notable
changes compared to 8.5.95 include:

- Fix reloading TLS configuration could cause the Connector to
  refuse new connections or the JVM to crash.

- Ensure that an IOException during the reading of the request
  always triggers error handling, regardless of whether the
  application swallows the exception.

- The status manager servlet can now output statistics as json.

Please refer to the change log for the complete list of changes:
https://tomcat.apache.org/tomcat-8.5-doc/changelog.html

Downloads:
https://tomcat.apache.org/download-80.cgi

Migration guides from Apache Tomcat 7.x and 8.0:
https://tomcat.apache.org/migration.html

Please note that Tomcat 8.5.x will reach End-of-life (EOL) on 31 March 
2024. For more information please visit 
https://tomcat.apache.org/tomcat-85-eol.html


Enjoy!

- The Apache Tomcat team

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Wondering about tomcat-users.xml could not be found


Christoph,

On 11/15/23 10:32, Christoph Kukulies wrote:

I'm running tomcat9 under Ubuntu 22.04 with an haproxy 2.8 in front of it.

I'm wondering about the following in the logs:

Nov 15 16:19:23 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source 
[file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:23 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:33 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source 
[file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:33 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:43 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source 
[file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:43 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:53 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source 
[file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:53 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found




File /var/lib/tomcat9/conf/tomcat-users.xml is definitely there.

It occurs every 10 seconds.

Don't know who is causing this and why. Permissions? Ownership wrong?

-rw-r- 1 root root   2756 Jan 15  2022 tomcat-users.xml

Believe the ownership was wrong. Maybe it came from migrating an old 
installation.


What are the correct perms/ownership in /var/lib/tomcat9 and below?


What is the user-owner of the JVM process?

Check that all of the above would be both readable and executable by 
that user:


 ls -ld /var
 ls -ld /var/lib
 ls -ld /var/lib/tomcat9
 ls -ld /var/lib/tomcat9/conf

... and of course that the JVM user can read 
/var/lib/tomcat9/conf/tomcat-users.xml which I assume is true since you 
said you already checked it.


What is the cwd of the JVM process?

The first message ("reloading") has the full path, and the second 
message ("file not found") only mentions a relative path. I wonder if 
that is the difference.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Partitioned cookies


Adam,

On 11/15/23 09:06, Adam Warfield wrote:

The Rfc6265CookieProcessor supports setting the SameSite cookie
attribute but starting in 2024, browsers will begin enforcing the
newer "Partitioned" attribute for third-party cookies.

Is there a way to set this attribute within Tomcat for things like the
JSESSIONID and XSRF-TOKEN cookies?


Wait... are you using cookies for CSRF tokens? That doesn't really 
provide much protection. Your CSRF cookie will be transmitted along with 
any request, even "forged" requests.


Are you responsible for the primary web application, here, or are you 
responsible for a third-party site such as an advertiser, back-end 
service, etc.?



This affects any webapps that are embedded within iframes across
domains where those cookies will be rejected if not partitioned.


If you migrate to Tomcat 10.1 or later (with Jakarta Servlet APIs), 
there is Cookie.setAttributeString name, String value)[1]


If you cannot upgrade to Tomcat 10 in time, then you can simply resort 
to setting the headers directly:


response.addHeader("Set-Cookie", "XSRF-TOKEN=foo; Partitioned");

-chris

[1] 
https://jakarta.ee/specifications/servlet/6.0/apidocs/jakarta.servlet/jakarta/servlet/http/cookie#setAttribute(java.lang.String,java.lang.String)


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: AW: FileUpload class not working with Tomcat 10.1


Mark,

Apologies for not replying earlier; looks like you have made good 
progress. See below.


On 11/14/23 12:19, Mark Foley wrote:

Anyway, enough griping! I have gotten it partially working thanks to your
suggested link, and particulary you suggestion to put the servlet info in
web.xml.  I've put the following in WEB-INF/web.xml:


   uploadfile
 /schDistImportResultsX.jsp
 
   /tmp
   20848820
   418018841
   1048576
 


   uploadfile
   /schDistImportResultsX.jsp


I've only changed the  and  tags above. The others are
as monkey-typed from your link example. I'll research the other parameters
later.

My jsp code is now:

<%@ page import="javax.servlet.annotation.MultipartConfig.*" %>


Nope, not for Tomcat 10. You need to use the jakarta package names. 
Besides, you don't need the MultipartConfig in your code, anyway.


You need /either/ an annotation (dicey in JSP code) /or/ an XML config, 
so the XML should be sufficient. (But you should switch to a proper 
servlet. DO IT! :)



if((contentType != null) && contentType.startsWith("multipart/form-data;"))
{
 InputStream inp = null;
 DataInputStream ins = null;

 Part fileUpload = request.getPart("taxResults");

 if(fileUpload != null)
 {
 inp = fileUpload.getInputStream();
 ins = new DataInputStream(inp);
 }
  
while ((inp != null) && (ins.available() != 0))

{
 String  transaction = ins.readLine();
 out.println("" + transaction);
}

ins.close();
inp.close();


I would use try-with-resources like this:

try (InputStream in = , DataInputStream ins = ...) {
}

Since you have no try/catch, your code can leak file handles and stuff 
like that. Yuck. With try-with-resources, you don't even need the calls 
to InputStream.close.



This actually worked I will experiment with it more and may be back with
more questions (e.g. do I really need the web.xml? Could I not do:
"inp = fileUpload.getInputStream(mypath);"). But ... maybe later.

Vielen Dank!!! --Mark


Na klar

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Wondering about tomcat-users.xml could not be found


Peter,

On 11/16/23 14:19, l...@kreuser.name wrote:

Hi Chris*,



Am 16.11.2023 um 20:12 schrieb Christopher Schultz 
:

Christoph,

On 11/15/23 10:32, Christoph Kukulies wrote:

I'm running tomcat9 under Ubuntu 22.04 with an haproxy 2.8 in front of it.
I'm wondering about the following in the logs:
Nov 15 16:19:23 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source [file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:23 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:33 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source [file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:33 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:43 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source [file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:43 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
Nov 15 16:19:53 mail tomcat9[832]: Reloading memory user database 
[UserDatabase] from updated source [file:/var/lib/tomcat9/conf/tomcat-users.xml]
Nov 15 16:19:53 mail tomcat9[832]: The specified user database 
[conf/tomcat-users.xml] could not be found
File /var/lib/tomcat9/conf/tomcat-users.xml is definitely there.
It occurs every 10 seconds.
Don't know who is causing this and why. Permissions? Ownership wrong?
-rw-r- 1 root root   2756 Jan 15  2022 tomcat-users.xml
Believe the ownership was wrong. Maybe it came from migrating an old 
installation.
What are the correct perms/ownership in /var/lib/tomcat9 and below?


What is the user-owner of the JVM process?

Check that all of the above would be both readable and executable by that user:

ls -ld /var
ls -ld /var/lib
ls -ld /var/lib/tomcat9
ls -ld /var/lib/tomcat9/conf

... and of course that the JVM user can read 
/var/lib/tomcat9/conf/tomcat-users.xml which I assume is true since you said 
you already checked it.

What is the cwd of the JVM process?

The first message ("reloading") has the full path, and the second message ("file not 
found") only mentions a relative path. I wonder if that is the difference.




Could it be that the second path relates to a missing env-Variable 
$CATALINA_BASE or $CATALINA_HOME?


Unlikely. Tomcat always determines the values for catalina.home and 
catalina.base before launching the JVM. After that, only those system 
properties are consulted.


But it's possible there is some sloppy code somewhere that is using 
cwd-relative paths.


-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Java/Tomcat is being killed by the Linux OOM killer for using a huge amount of RAM. How can I know what was going on inside my app (& Tomcat & the JVM) to make that happen?

Brian,

On 11/16/23 15:26, Brian Braun wrote:

First of all, this is my stack:

- Ubuntu 22.04.3 on x86/64 with 2GM of physical RAM that has been enough
for years.
- Java 11.0.20.1+1-post-Ubuntu-0ubuntu122.04 / openjdk 11.0.20.1 2023-08-24
- Tomcat 9.0.58 (JAVA_OPTS="-Djava.awt.headless=true -Xmx900m -Xms16m
..")

Don't bother setting a 16M initial heap and a maximum of 900M. Just set
them both to 900M. That will cause the JVM to request all of that heap
up front and lessen the chances of a native OOME.

There are certainly still plenty of reasons the process could use more
heap than that, of course.

- My app, which I developed myself, and has been running without any OOM
crashes for years

Well, a couple of weeks ago my website started crushing about every 5-7
days. Between crashes the RAM usage is fine and very steady (as it has been
for years) and it uses just about 50% of the "Max memory" (according to
what the Tomcat Manager server status shows). The 3 types of G1 heap are
steady and low. And there are no leaks as far as I can tell. And I haven't
made any significant changes to my app in the last months.

I think your problem is native-heap and not Java-heap.

What does 'top' say? You are looking for the "RES" (Resident Size) and
"VIRT" (Virtual Size) numbers. That's what the process is REALLY using.

How big is your physical RAM? What does this output while running your
application (after fixing the heap at 900M)?

$ free -m

What else is running on the machine?

Do you have swap enabled?

When my website crashes, I can see on the Ubuntu log that some process has
invoked the "oom-killer" and that this killer investigates which process is
using most of the RAM and it is Tomcat/Java so it kills it. This is what I
see on the log when it was Nginx that invoked the OOM-killer:

Nov 15 15:23:54 ip-172-31-89-211 kernel: [366008.597771]
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=nginx.service,mems_allowed=0,global_oom,task_memcg=/system.slice/tomcat9.service,task=java,pid=470,uid=998
Nov 15 15:23:54 ip-172-31-89-211 kernel: [366008.597932] Out of memory:
Killed process 470 (java) total-vm:4553056kB, anon-rss:1527944kB,
file-rss:2872kB, shmem-rss:0kB, UID:998 pgtables:3628kB oom_score_adj:0

I would like to be able to know what was happening inside the JVM when it
was using too much RAM and deserved to be killed. Was it a problem in Java
not associated with Tomcat or my app? Was it Tomcat itself that ate too
much RAM? I doubt it. Was it my application? If it was my application (and
I have to assume it was), how/why was it using all that RAM? What were the
objects, threads, etc that were involved in the crash? What part of the
heap memory was using all that RAM?

Probably native heap. Java 11 is mature and there are likely no leaks in
the JVM itself. If your code was using too much Java heap, you'd get
OutOfMemoryErrors thrown in the JVM but not Linux oom-killer.

But certain native libraries can leak. I seem to recall libgzip or
something like that can leak if you aren't careful. My guess is that you
are actually just running very very close to what your hardware can support.

Do you actually need 900M of heap to run your application? We ran for
years at $work with a 64M heap and only expended it when we started
getting enough concurrent users to /have/ to expand the heap.

This can happen at any time, like at 4am so I can not run to the computer
to see what was going on at that moment. I need some way to get a detailed
log of what was going on when the crush took place.

So my question is, what tool should I use to investigate these crashes? I
have started trying to make "New Relic" work since it seems that this
service could help me, but I am having some problems making it work and I
still don't know if this would be a solution in the first place. So, while
I struggle with New Relic, I would appreciate your suggestions.

You can get a lot of information by configuring your application to dump
the heap on OOME, but you aren't getting an OOME so that's kind of off
the table.

I would enable GC logging for sure. That will tell you the status of the
Java heap, but not the native memory spaces. But you may find that the
process is performing a GC when it dies or you can see what was
happening up to the point of the kill.

Is there any pattern to when it crashes? For example... is 04:00 a
popular time for it to die? Maybe you have a process that runs
periodically that needs a lot of RAM temporarily.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: CredentialHandler not working for MD5